__ __ / \ / \ ______ _ _ __ __ __ / /\ / /\ \ / __ \ | | | | / \ \ \ / / / / / / \ \ | | | | | | | | / /\ \ \ / \ \ \ \ / / | |__| | | |__| | / ____ \ | | \ \/ \ \/ / \_ ___/ \____/ /_/ \_\ |_| \__/ \__/ \ \__ \___\ by Red Hat Build, Store, and Distribute your Containers Running all default registry services Running init script '/quay-registry/conf/init/02_get_kube_certs.sh' Running on kubernetes, attempting to retrieve extra certs from secret /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) Running init script '/quay-registry/conf/init/certs_create.sh' Generating a 4096 bit RSA private key .............................................................................................................................++ .....................................................++ writing new private key to 'mitm-key.pem' ----- Running init script '/quay-registry/conf/init/certs_install.sh' Installing extra certificates found in /conf/stack/extra_ca_certs directory Running init script '/quay-registry/conf/init/copy_config_files.sh' Running init script '/quay-registry/conf/init/nginx_conf_create.sh' /quay-registry/conf/init/nginx_conf_create.py:130: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. config = yaml.load(f) Running init script '/quay-registry/conf/init/runmigration.sh' No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" 08:49:51 INFO [alembic.runtime.migration] Context impl PostgresqlImpl. 08:49:51 INFO [alembic.runtime.migration] Will assume non-transactional DDL. Running init script '/quay-registry/conf/init/supervisord_conf_create.sh' Running init script '/quay-registry/conf/init/zz_boot.sh' No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:49:54,893 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. 2020-03-31 08:49:54,906 INFO RPC interface 'supervisor' initialized 2020-03-31 08:49:54,906 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2020-03-31 08:49:54,906 INFO supervisord started with pid 161 2020-03-31 08:49:55,909 INFO spawned: 'stdout' with pid 164 2020-03-31 08:49:55,912 INFO spawned: 'servicekey' with pid 165 2020-03-31 08:49:55,914 INFO spawned: 'exportactionlogsworker' with pid 166 2020-03-31 08:49:55,917 INFO spawned: 'gunicorn-registry' with pid 167 2020-03-31 08:49:55,920 INFO spawned: 'security_notification_worker' with pid 168 2020-03-31 08:49:55,922 INFO spawned: 'prometheus-aggregator' with pid 169 2020-03-31 08:49:55,924 INFO spawned: 'queuecleanupworker' with pid 170 2020-03-31 08:49:55,926 INFO spawned: 'gunicorn-web' with pid 171 2020-03-31 08:49:55,929 INFO spawned: 'repositoryactioncounter' with pid 172 2020-03-31 08:49:55,931 INFO spawned: 'buildlogsarchiver' with pid 173 2020-03-31 08:49:55,934 INFO spawned: 'gcworker' with pid 174 2020-03-31 08:49:55,936 INFO spawned: 'notificationworker' with pid 175 2020-03-31 08:49:55,938 INFO spawned: 'teamsyncworker' with pid 176 2020-03-31 08:49:55,941 INFO spawned: 'logrotateworker' with pid 177 2020-03-31 08:49:55,943 INFO spawned: 'tagbackfillworker' with pid 178 2020-03-31 08:49:55,946 INFO spawned: 'blobuploadcleanupworker' with pid 179 2020-03-31 08:49:55,949 INFO spawned: 'nginx' with pid 180 2020-03-31 08:49:55,952 INFO spawned: 'gunicorn-secscan' with pid 181 2020-03-31 08:49:55,954 INFO spawned: 'memcache' with pid 182 2020-03-31 08:49:55,957 INFO spawned: 'chunkcleanupworker' with pid 183 2020-03-31 08:49:55,959 INFO spawned: 'expiredappspecifictokenworker' with pid 184 2020-03-31 08:49:55,962 INFO spawned: 'labelbackfillworker' with pid 185 2020-03-31 08:49:55,964 INFO spawned: 'gunicorn-verbs' with pid 186 2020-03-31 08:49:55,966 INFO spawned: 'securityworker' with pid 187 2020-03-31 08:49:55,969 INFO spawned: 'globalpromstats' with pid 188 2020-03-31 08:49:55,971 INFO spawned: 'builder' with pid 189 2020-03-31 08:49:55,974 INFO spawned: 'dnsmasq' with pid 190 2020-03-31 08:49:55,976 INFO spawned: 'storagereplication' with pid 191 2020-03-31 08:49:55,979 INFO spawned: 'namespacegcworker' with pid 192 2020-03-31 08:49:55,981 INFO spawned: 'jwtproxy' with pid 193 time="2020-03-31T08:49:56Z" level=info msg="listening on :9092" time="2020-03-31T08:49:56Z" level=info msg="No claims verifiers specified, upstream should be configured to verify authorization" time="2020-03-31T08:49:56Z" level=info msg="Starting reverse proxy (Listening on 'unix:/tmp/jwtproxy_secscan.sock')" time="2020-03-31T08:49:56Z" level=info msg="Starting forward proxy (Listening on ':8081')" 2020/03/31 08:49:56 [alert] 209#0: setpriority(-10) failed (13: Permission denied) 2020/03/31 08:49:56 [alert] 210#0: setpriority(-10) failed (13: Permission denied) 2020/03/31 08:49:56 [alert] 211#0: setpriority(-10) failed (13: Permission denied) 2020/03/31 08:49:56 [alert] 212#0: setpriority(-10) failed (13: Permission denied) 2020-03-31 08:49:57,239 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,240 INFO success: servicekey entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,240 INFO success: exportactionlogsworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,240 INFO success: gunicorn-registry entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,240 INFO success: security_notification_worker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,240 INFO success: prometheus-aggregator entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,240 INFO success: queuecleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: gunicorn-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: repositoryactioncounter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: buildlogsarchiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: gcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: notificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: teamsyncworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: logrotateworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: tagbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: blobuploadcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: gunicorn-secscan entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: memcache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,241 INFO success: chunkcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: expiredappspecifictokenworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: labelbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: gunicorn-verbs entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: securityworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: globalpromstats entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: builder entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: storagereplication entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: namespacegcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2020-03-31 08:49:57,242 INFO success: jwtproxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) prometheus-aggregator stderr | time="2020-03-31T08:49:56Z" level=info msg="listening on :9092" jwtproxy stderr | time="2020-03-31T08:49:56Z" level=info msg="No claims verifiers specified, upstream should be configured to verify authorization" time="2020-03-31T08:49:56Z" level=info msg="Starting reverse proxy (Listening on 'unix:/tmp/jwtproxy_secscan.sock')" jwtproxy stderr | time="2020-03-31T08:49:56Z" level=info msg="Starting forward proxy (Listening on ':8081')" nginx stdout | 2020/03/31 08:49:56 [alert] 209#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2020/03/31 08:49:56 [alert] 210#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2020/03/31 08:49:56 [alert] 211#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2020/03/31 08:49:56 [alert] 212#0: setpriority(-10) failed (13: Permission denied) 2020/03/31 08:49:58 [crit] 210#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.131.0.21:8443" 2020/03/31 08:49:58 [crit] 210#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.131.0.21:8443" 10.131.0.1 () - - [31/Mar/2020:08:49:58 +0000] "GET /health/instance HTTP/2.0" 502 173 "-" "kube-probe/1.16+" (0.000 49 0.000 : 0.000) nginx stdout | 2020/03/31 08:49:58 [crit] 210#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.131.0.21:8443" 2020/03/31 08:49:58 [crit] 210#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.131.0.21:8443" 10.131.0.1 () - - [31/Mar/2020:08:49:58 +0000] "GET /health/instance HTTP/2.0" 502 173 "-" "kube-probe/1.16+" (0.000 49 0.000 : 0.000) dnsmasq: started, version 2.76 cachesize 150 dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-DNSSEC loop-detect inotify dnsmasq: reading /etc/resolv.conf dnsmasq: using nameserver 172.30.0.10#53 dnsmasq: read /etc/hosts - 7 addresses dnsmasq stderr | dnsmasq: started, version 2.76 cachesize 150 dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-DNSSEC loop-detect inotify dnsmasq: reading /etc/resolv.conf dnsmasq: using nameserver 172.30.0.10#53 dnsmasq: read /etc/hosts - 7 addresses No handlers could be found for logger "util.config.provider.baseprovider" builder stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:07,999 [189] [DEBUG] [__main__] Building is disabled. Please enable the feature flag builder stdout | 2020-03-31 08:50:07,999 [189] [DEBUG] [__main__] Building is disabled. Please enable the feature flag 2020/03/31 08:50:08 [crit] 212#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.131.0.21:8443" 2020/03/31 08:50:08 [crit] 212#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.131.0.21:8443" 10.131.0.1 () - - [31/Mar/2020:08:50:08 +0000] "GET /health/instance HTTP/2.0" 502 173 "-" "kube-probe/1.16+" (0.000 49 0.000 : 0.000) nginx stdout | 2020/03/31 08:50:08 [crit] 212#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.131.0.21:8443" 2020/03/31 08:50:08 [crit] 212#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.131.0.21:8443" 10.131.0.1 () - - [31/Mar/2020:08:50:08 +0000] "GET /health/instance HTTP/2.0" 502 173 "-" "kube-probe/1.16+" (0.000 49 0.000 : 0.000) No handlers could be found for logger "util.config.provider.baseprovider" labelbackfillworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" tagbackfillworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" servicekey stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" chunkcleanupworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" gcworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" globalpromstats stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" notificationworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:13,090 [181] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2020-03-31 08:50:13,090 [181] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:13,090 [181] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:13,091 [181] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2020-03-31 08:50:13,090 [181] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:13,091 [181] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:13,150 [165] [DEBUG] [workers.worker] Scheduling worker. servicekey stdout | 2020-03-31 08:50:13,150 [165] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:13,152 [165] [INFO] [apscheduler.scheduler] Scheduler started servicekey stdout | 2020-03-31 08:50:13,152 [165] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:13,154 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:13,154 [165] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added servicekey stdout | 2020-03-31 08:50:13,154 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:13,154 [165] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:13,154 [165] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:54:39.155593 servicekey stdout | 2020-03-31 08:50:13,154 [165] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:54:39.155593 No handlers could be found for logger "util.config.provider.baseprovider" expiredappspecifictokenworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:13,390 [165] [INFO] [apscheduler.scheduler] Added job "_refresh_service_key" to job store "default" servicekey stdout | 2020-03-31 08:50:13,390 [165] [INFO] [apscheduler.scheduler] Added job "_refresh_service_key" to job store "default" 2020-03-31 08:50:13,391 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:13,391 [165] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:39.155593+00:00 (in 265.764193 seconds) servicekey stdout | 2020-03-31 08:50:13,391 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:13,391 [165] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:39.155593+00:00 (in 265.764193 seconds) No handlers could be found for logger "util.config.provider.baseprovider" blobuploadcleanupworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" security_notification_worker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" repositoryactioncounter stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:14,813 [181] [DEBUG] [app] Loading default config. gunicorn-secscan stdout | 2020-03-31 08:50:14,813 [181] [DEBUG] [app] Loading default config. 2020-03-31 08:50:14,814 [181] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-secscan stdout | 2020-03-31 08:50:14,814 [181] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml No handlers could be found for logger "util.config.provider.baseprovider" logrotateworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:15,058 [174] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:15,058 [174] [INFO] [apscheduler.scheduler] Scheduler started gcworker stdout | 2020-03-31 08:50:15,058 [174] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:15,058 [174] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:15,063 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:50:15,063 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:15,063 [174] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:45.064514 gcworker stdout | 2020-03-31 08:50:15,063 [174] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:45.064514 2020-03-31 08:50:15,067 [174] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added gcworker stdout | 2020-03-31 08:50:15,067 [174] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:15,151 [174] [INFO] [apscheduler.scheduler] Added job "_garbage_collection_repos" to job store "default" gcworker stdout | 2020-03-31 08:50:15,151 [174] [INFO] [apscheduler.scheduler] Added job "_garbage_collection_repos" to job store "default" 2020-03-31 08:50:15,152 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:15,152 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:45.064514+00:00 (in 29.912036 seconds) gcworker stdout | 2020-03-31 08:50:15,152 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:15,152 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:45.064514+00:00 (in 29.912036 seconds) 2020-03-31 08:50:15,395 [181] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None gunicorn-secscan stdout | 2020-03-31 08:50:15,395 [181] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None 2020-03-31 08:50:15,409 [181] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:15,409 [181] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:15,409 [181] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2020-03-31 08:50:15,409 [181] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:15,409 [181] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:15,409 [181] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:15,418 [181] [DEBUG] [app] Loaded config 2020-03-31 08:50:15,418 [181] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 gunicorn-secscan stdout | 2020-03-31 08:50:15,418 [181] [DEBUG] [app] Loaded config 2020-03-31 08:50:15,418 [181] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 2020-03-31 08:50:15,424 [181] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 gunicorn-secscan stdout | 2020-03-31 08:50:15,424 [181] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 2020-03-31 08:50:15,436 [181] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-secscan stdout | 2020-03-31 08:50:15,436 [181] [INFO] [util.ipresolver] Loading AWS IP ranges from disk No handlers could be found for logger "util.config.provider.baseprovider" exportactionlogsworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" storagereplication stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:15,611 [181] [INFO] [util.ipresolver] Building AWS IP ranges gunicorn-secscan stdout | 2020-03-31 08:50:15,611 [181] [INFO] [util.ipresolver] Building AWS IP ranges 2020-03-31 08:50:15,673 [184] [DEBUG] [__main__] Starting expired app specific token GC worker expiredappspecifictokenworker stdout | 2020-03-31 08:50:15,673 [184] [DEBUG] [__main__] Starting expired app specific token GC worker No handlers could be found for logger "util.config.provider.baseprovider" teamsyncworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:15,860 [184] [DEBUG] [__main__] Found expiration window: 1d expiredappspecifictokenworker stdout | 2020-03-31 08:50:15,860 [184] [DEBUG] [__main__] Found expiration window: 1d 2020-03-31 08:50:15,880 [184] [DEBUG] [workers.worker] Scheduling worker. expiredappspecifictokenworker stdout | 2020-03-31 08:50:15,880 [184] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:15,895 [184] [INFO] [apscheduler.scheduler] Scheduler started expiredappspecifictokenworker stdout | 2020-03-31 08:50:15,895 [184] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:15,904 [184] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:15,904 [184] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added expiredappspecifictokenworker stdout | 2020-03-31 08:50:15,904 [184] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:15,904 [184] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:15,906 [184] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:04:22.907313 expiredappspecifictokenworker stdout | 2020-03-31 08:50:15,906 [184] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:04:22.907313 No handlers could be found for logger "util.config.provider.baseprovider" buildlogsarchiver stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:16,077 [184] [INFO] [apscheduler.scheduler] Added job "_gc_expired_tokens" to job store "default" expiredappspecifictokenworker stdout | 2020-03-31 08:50:16,077 [184] [INFO] [apscheduler.scheduler] Added job "_gc_expired_tokens" to job store "default" 2020-03-31 08:50:16,079 [184] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:16,079 [184] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:22.907313+00:00 (in 846.827830 seconds) expiredappspecifictokenworker stdout | 2020-03-31 08:50:16,079 [184] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:16,079 [184] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:22.907313+00:00 (in 846.827830 seconds) No handlers could be found for logger "util.config.provider.baseprovider" queuecleanupworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" No handlers could be found for logger "util.config.provider.baseprovider" namespacegcworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:17,290 [177] [DEBUG] [__main__] Action log rotation worker not enabled; skipping logrotateworker stdout | 2020-03-31 08:50:17,290 [177] [DEBUG] [__main__] Action log rotation worker not enabled; skipping 2020-03-31 08:50:17,328 [168] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:17,328 [168] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:17,328 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:17,329 [168] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:17,329 [168] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:33.330112 security_notification_worker stdout | 2020-03-31 08:50:17,328 [168] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:17,328 [168] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:17,328 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:17,329 [168] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added security_notification_worker stdout | 2020-03-31 08:50:17,329 [168] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:33.330112 2020-03-31 08:50:17,409 [168] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:17,409 [168] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:54:23.410371 2020-03-31 08:50:17,409 [168] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:17,409 [168] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:24.410927 security_notification_worker stdout | 2020-03-31 08:50:17,409 [168] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:17,409 [168] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:54:23.410371 2020-03-31 08:50:17,409 [168] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:17,409 [168] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:24.410927 2020-03-31 08:50:17,413 [168] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:17,413 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:17,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:24.410927+00:00 (in 6.997289 seconds) security_notification_worker stdout | 2020-03-31 08:50:17,413 [168] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:17,413 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:17,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:24.410927+00:00 (in 6.997289 seconds) 2020/03/31 08:50:18 [crit] 212#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.131.0.21:8443" 2020/03/31 08:50:18 [crit] 212#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.131.0.21:8443" 10.131.0.1 () - - [31/Mar/2020:08:50:18 +0000] "GET /health/instance HTTP/2.0" 502 173 "-" "kube-probe/1.16+" (0.000 49 0.000 : 0.000) nginx stdout | 2020/03/31 08:50:18 [crit] 212#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.131.0.21:8443" 2020/03/31 08:50:18 [crit] 212#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.131.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.131.0.21:8443" 10.131.0.1 () - - [31/Mar/2020:08:50:18 +0000] "GET /health/instance HTTP/2.0" 502 173 "-" "kube-probe/1.16+" (0.000 49 0.000 : 0.000) No handlers could be found for logger "util.config.provider.baseprovider" securityworker stderr | No handlers could be found for logger "util.config.provider.baseprovider" 2020-03-31 08:50:18,758 [179] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:18,758 [179] [INFO] [apscheduler.scheduler] Scheduler started blobuploadcleanupworker stdout | 2020-03-31 08:50:18,758 [179] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:18,758 [179] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:18,834 [179] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:18,834 [179] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:18,834 [179] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:09:41.835723 blobuploadcleanupworker stdout | 2020-03-31 08:50:18,834 [179] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:18,834 [179] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added blobuploadcleanupworker stdout | 2020-03-31 08:50:18,834 [179] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:09:41.835723 2020-03-31 08:50:19,159 [183] [DEBUG] [__main__] Swift storage not detected; sleeping chunkcleanupworker stdout | 2020-03-31 08:50:19,159 [183] [DEBUG] [__main__] Swift storage not detected; sleeping 2020-03-31 08:50:19,306 [166] [DEBUG] [__main__] Starting export action logs worker exportactionlogsworker stdout | 2020-03-31 08:50:19,306 [166] [DEBUG] [__main__] Starting export action logs worker 2020-03-31 08:50:19,316 [175] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:19,317 [175] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stdout | 2020-03-31 08:50:19,316 [175] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:19,317 [175] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:19,320 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,320 [175] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added notificationworker stdout | 2020-03-31 08:50:19,320 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,320 [175] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:19,322 [175] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:24.323352 notificationworker stdout | 2020-03-31 08:50:19,322 [175] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:24.323352 2020-03-31 08:50:19,357 [166] [DEBUG] [workers.worker] Scheduling worker. exportactionlogsworker stdout | 2020-03-31 08:50:19,357 [166] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:19,361 [166] [INFO] [apscheduler.scheduler] Scheduler started exportactionlogsworker stdout | 2020-03-31 08:50:19,361 [166] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:19,363 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,363 [166] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added exportactionlogsworker stdout | 2020-03-31 08:50:19,363 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,363 [166] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:19,365 [166] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:33.366791 exportactionlogsworker stdout | 2020-03-31 08:50:19,365 [166] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:33.366791 2020-03-31 08:50:19,366 [178] [DEBUG] [__main__] Tag backfill disabled; skipping tagbackfillworker stdout | 2020-03-31 08:50:19,366 [178] [DEBUG] [__main__] Tag backfill disabled; skipping 2020-03-31 08:50:19,432 [179] [INFO] [apscheduler.scheduler] Added job "_cleanup_uploads" to job store "default" blobuploadcleanupworker stdout | 2020-03-31 08:50:19,432 [179] [INFO] [apscheduler.scheduler] Added job "_cleanup_uploads" to job store "default" 2020-03-31 08:50:19,433 [179] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,433 [179] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:41.835723+00:00 (in 1162.402332 seconds) blobuploadcleanupworker stdout | 2020-03-31 08:50:19,433 [179] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,433 [179] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:41.835723+00:00 (in 1162.402332 seconds) 2020-03-31 08:50:19,496 [175] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:19,497 [175] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:52:31.498397 2020-03-31 08:50:19,497 [175] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:19,497 [175] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:49.498966 notificationworker stdout | 2020-03-31 08:50:19,496 [175] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:19,497 [175] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:52:31.498397 2020-03-31 08:50:19,497 [175] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:19,497 [175] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:49.498966 2020-03-31 08:50:19,500 [175] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:19,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:24.323352+00:00 (in 4.822581 seconds) notificationworker stdout | 2020-03-31 08:50:19,500 [175] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:19,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:24.323352+00:00 (in 4.822581 seconds) 2020-03-31 08:50:19,538 [166] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:19,538 [166] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:55:14.539408 2020-03-31 08:50:19,538 [166] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:19,539 [166] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:43.539999 exportactionlogsworker stdout | 2020-03-31 08:50:19,538 [166] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:19,538 [166] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:55:14.539408 2020-03-31 08:50:19,538 [166] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:19,539 [166] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:43.539999 2020-03-31 08:50:19,542 [166] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:19,542 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:33.366791+00:00 (in 13.824020 seconds) exportactionlogsworker stdout | 2020-03-31 08:50:19,542 [166] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:19,542 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:33.366791+00:00 (in 13.824020 seconds) 2020-03-31 08:50:19,628 [188] [DEBUG] [workers.worker] Scheduling worker. globalpromstats stdout | 2020-03-31 08:50:19,628 [188] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:19,629 [188] [INFO] [apscheduler.scheduler] Scheduler started globalpromstats stdout | 2020-03-31 08:50:19,629 [188] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:19,637 [188] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,637 [188] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added globalpromstats stdout | 2020-03-31 08:50:19,637 [188] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,637 [188] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:19,638 [188] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:48:57.639223 globalpromstats stdout | 2020-03-31 08:50:19,638 [188] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:48:57.639223 2020-03-31 08:50:19,708 [170] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:19,709 [170] [INFO] [apscheduler.scheduler] Scheduler started queuecleanupworker stdout | 2020-03-31 08:50:19,708 [170] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:19,709 [170] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:19,712 [170] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2020-03-31 08:50:19,712 [170] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,712 [170] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:30:53.713440 queuecleanupworker stdout | 2020-03-31 08:50:19,712 [170] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 09:30:53.713440 2020-03-31 08:50:19,714 [170] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added queuecleanupworker stdout | 2020-03-31 08:50:19,714 [170] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:19,797 [170] [INFO] [apscheduler.scheduler] Added job "_cleanup_queue" to job store "default" 2020-03-31 08:50:19,798 [170] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,798 [170] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:53.713440+00:00 (in 2433.915081 seconds) queuecleanupworker stdout | 2020-03-31 08:50:19,797 [170] [INFO] [apscheduler.scheduler] Added job "_cleanup_queue" to job store "default" queuecleanupworker stdout | 2020-03-31 08:50:19,798 [170] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,798 [170] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:53.713440+00:00 (in 2433.915081 seconds) 2020-03-31 08:50:19,921 [188] [INFO] [apscheduler.scheduler] Added job "_try_report_stats" to job store "default" globalpromstats stdout | 2020-03-31 08:50:19,921 [188] [INFO] [apscheduler.scheduler] Added job "_try_report_stats" to job store "default" 2020-03-31 08:50:19,922 [188] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,922 [188] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:57.639223+00:00 (in 3517.716823 seconds) globalpromstats stdout | 2020-03-31 08:50:19,922 [188] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:19,922 [188] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:57.639223+00:00 (in 3517.716823 seconds) 2020-03-31 08:50:20,083 [192] [DEBUG] [__main__] Starting namespace GC worker namespacegcworker stdout | 2020-03-31 08:50:20,083 [192] [DEBUG] [__main__] Starting namespace GC worker 2020-03-31 08:50:20,107 [192] [DEBUG] [workers.worker] Scheduling worker. namespacegcworker stdout | 2020-03-31 08:50:20,107 [192] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:20,108 [192] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:20,109 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,109 [192] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added namespacegcworker stdout | 2020-03-31 08:50:20,108 [192] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:20,109 [192] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:51.110610 namespacegcworker stdout | 2020-03-31 08:50:20,109 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,109 [192] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added namespacegcworker stdout | 2020-03-31 08:50:20,109 [192] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:51.110610 2020-03-31 08:50:20,328 [171] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:20,328 [171] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:20,329 [171] [INFO] [data.registry_model] =============================== gunicorn-web stdout | 2020-03-31 08:50:20,328 [171] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:20,328 [171] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:20,329 [171] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:20,334 [173] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:20,334 [173] [INFO] [apscheduler.scheduler] Scheduler started buildlogsarchiver stdout | 2020-03-31 08:50:20,334 [173] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:20,334 [173] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:20,336 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,336 [173] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added buildlogsarchiver stdout | 2020-03-31 08:50:20,336 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,336 [173] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:20,336 [173] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:30.337699 buildlogsarchiver stdout | 2020-03-31 08:50:20,336 [173] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:30.337699 2020-03-31 08:50:20,419 [173] [INFO] [apscheduler.scheduler] Added job "_archive_redis_buildlogs" to job store "default" 2020-03-31 08:50:20,420 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,420 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:30.337699+00:00 (in 9.917284 seconds) buildlogsarchiver stdout | 2020-03-31 08:50:20,419 [173] [INFO] [apscheduler.scheduler] Added job "_archive_redis_buildlogs" to job store "default" buildlogsarchiver stdout | 2020-03-31 08:50:20,420 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,420 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:30.337699+00:00 (in 9.917284 seconds) 2020-03-31 08:50:20,431 [185] [DEBUG] [__main__] Manifest label backfill disabled; skipping labelbackfillworker stdout | 2020-03-31 08:50:20,431 [185] [DEBUG] [__main__] Manifest label backfill disabled; skipping 2020-03-31 08:50:20,438 [172] [DEBUG] [workers.worker] Scheduling worker. repositoryactioncounter stdout | 2020-03-31 08:50:20,438 [172] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:20,439 [172] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:20,439 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,439 [172] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositoryactioncounter stdout | 2020-03-31 08:50:20,439 [172] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:20,439 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,439 [172] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:20,439 [172] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:21.440707 repositoryactioncounter stdout | 2020-03-31 08:50:20,439 [172] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:21.440707 2020-03-31 08:50:20,488 [172] [INFO] [apscheduler.scheduler] Added job "wrapper" to job store "default" 2020-03-31 08:50:20,488 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,488 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:21.440707+00:00 (in 0.951965 seconds) repositoryactioncounter stdout | 2020-03-31 08:50:20,488 [172] [INFO] [apscheduler.scheduler] Added job "wrapper" to job store "default" 2020-03-31 08:50:20,488 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,488 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:21.440707+00:00 (in 0.951965 seconds) 2020-03-31 08:50:20,552 [192] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" namespacegcworker stdout | 2020-03-31 08:50:20,552 [192] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:20,555 [192] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:54:53.556282 namespacegcworker stdout | 2020-03-31 08:50:20,555 [192] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:54:53.556282 2020-03-31 08:50:20,555 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:50:20,555 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,557 [192] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:20,559 [192] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:59.560326 2020-03-31 08:50:20,559 [192] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" namespacegcworker stdout | 2020-03-31 08:50:20,557 [192] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:20,560 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:51.110610+00:00 (in 30.553355 seconds) 2020-03-31 08:50:20,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,560 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:51.110610+00:00 (in 30.550092 seconds) namespacegcworker stdout | 2020-03-31 08:50:20,559 [192] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:59.560326 2020-03-31 08:50:20,559 [192] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:20,561 [171] [DEBUG] [app] Loading default config. namespacegcworker stdout | 2020-03-31 08:50:20,560 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:51.110610+00:00 (in 30.553355 seconds) 2020-03-31 08:50:20,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,560 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:51.110610+00:00 (in 30.550092 seconds) 2020-03-31 08:50:20,562 [171] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-web stdout | 2020-03-31 08:50:20,561 [171] [DEBUG] [app] Loading default config. gunicorn-web stdout | 2020-03-31 08:50:20,562 [171] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml 2020-03-31 08:50:20,616 [171] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None gunicorn-web stdout | 2020-03-31 08:50:20,616 [171] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None 2020-03-31 08:50:20,617 [171] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:20,617 [171] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:20,618 [171] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:20,618 [171] [DEBUG] [app] Loaded config gunicorn-web stdout | 2020-03-31 08:50:20,617 [171] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:20,617 [171] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:20,618 [171] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:20,618 [171] [DEBUG] [app] Loaded config 2020-03-31 08:50:20,619 [171] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 2020-03-31 08:50:20,619 [171] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 gunicorn-web stdout | 2020-03-31 08:50:20,619 [171] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 2020-03-31 08:50:20,619 [171] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 2020-03-31 08:50:20,621 [171] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-web stdout | 2020-03-31 08:50:20,621 [171] [INFO] [util.ipresolver] Loading AWS IP ranges from disk 2020-03-31 08:50:20,634 [171] [INFO] [util.ipresolver] Building AWS IP ranges gunicorn-web stdout | 2020-03-31 08:50:20,634 [171] [INFO] [util.ipresolver] Building AWS IP ranges 2020-03-31 08:50:20,747 [181] [INFO] [util.ipresolver] Finished building AWS IP ranges gunicorn-secscan stdout | 2020-03-31 08:50:20,747 [181] [INFO] [util.ipresolver] Finished building AWS IP ranges 2020-03-31 08:50:20,759 [181] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured gunicorn-secscan stdout | 2020-03-31 08:50:20,759 [181] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured 2020-03-31 08:50:20,895 [191] [DEBUG] [__main__] Starting replication worker storagereplication stdout | 2020-03-31 08:50:20,895 [191] [DEBUG] [__main__] Starting replication worker 2020-03-31 08:50:20,920 [191] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:20,920 [191] [INFO] [apscheduler.scheduler] Scheduler started storagereplication stdout | 2020-03-31 08:50:20,920 [191] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:20,920 [191] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:20,923 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:50:20,923 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:20,923 [191] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:26.924497 storagereplication stdout | 2020-03-31 08:50:20,923 [191] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:26.924497 2020-03-31 08:50:20,924 [191] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added storagereplication stdout | 2020-03-31 08:50:20,924 [191] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:21,009 [191] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:21,009 [191] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:52:03.010318 2020-03-31 08:50:21,009 [191] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:21,009 [191] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:38.010934 storagereplication stdout | 2020-03-31 08:50:21,009 [191] [INFO] [apscheduler.scheduler] Added job "poll_queue" to job store "default" 2020-03-31 08:50:21,009 [191] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:52:03.010318 2020-03-31 08:50:21,009 [191] [INFO] [apscheduler.scheduler] Added job "update_queue_metrics" to job store "default" 2020-03-31 08:50:21,009 [191] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:38.010934 2020-03-31 08:50:21,012 [191] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" storagereplication stdout | 2020-03-31 08:50:21,012 [191] [INFO] [apscheduler.scheduler] Added job "run_watchdog" to job store "default" 2020-03-31 08:50:21,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:21,013 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:26.924497+00:00 (in 5.911141 seconds) storagereplication stdout | 2020-03-31 08:50:21,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:21,013 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:26.924497+00:00 (in 5.911141 seconds) 2020-03-31 08:50:21,108 [187] [DEBUG] [workers.worker] Scheduling worker. securityworker stdout | 2020-03-31 08:50:21,108 [187] [DEBUG] [workers.worker] Scheduling worker. 2020-03-31 08:50:21,110 [187] [INFO] [apscheduler.scheduler] Scheduler started securityworker stdout | 2020-03-31 08:50:21,110 [187] [INFO] [apscheduler.scheduler] Scheduler started 2020-03-31 08:50:21,111 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:21,111 [187] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityworker stdout | 2020-03-31 08:50:21,111 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:21,111 [187] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added 2020-03-31 08:50:21,112 [187] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:22.113221 securityworker stdout | 2020-03-31 08:50:21,112 [187] [DEBUG] [workers.worker] First run scheduled for 2020-03-31 08:50:22.113221 2020-03-31 08:50:21,231 [176] [DEBUG] [__main__] Team syncing is disabled; sleeping teamsyncworker stdout | 2020-03-31 08:50:21,231 [176] [DEBUG] [__main__] Team syncing is disabled; sleeping 2020-03-31 08:50:21,255 [187] [INFO] [apscheduler.scheduler] Added job "_index_images" to job store "default" securityworker stdout | 2020-03-31 08:50:21,255 [187] [INFO] [apscheduler.scheduler] Added job "_index_images" to job store "default" 2020-03-31 08:50:21,256 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:21,256 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:22.113221+00:00 (in 0.856852 seconds) securityworker stdout | 2020-03-31 08:50:21,256 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:21,256 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:22.113221+00:00 (in 0.856852 seconds) 2020-03-31 08:50:21,435 [181] [DEBUG] [data.database] Configuring database gunicorn-secscan stdout | 2020-03-31 08:50:21,435 [181] [DEBUG] [data.database] Configuring database 2020-03-31 08:50:21,438 [181] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:50:21,438 [181] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:21,439 [181] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2020-03-31 08:50:21,438 [181] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:50:21,439 [181] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:21,439 [181] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2020-03-31 08:50:21,438 [181] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:21,439 [181] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-secscan stdout | 2020-03-31 08:50:21,439 [181] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:21,439 [181] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:21,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:21 UTC)" (scheduled at 2020-03-31 08:50:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:50:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:50:21,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:21 UTC)" (scheduled at 2020-03-31 08:50:21.440707+00:00) 2020-03-31 08:50:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:50:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:50:21,449 [181] [DEBUG] [__config__] Starting secscan gunicorn with 4 workers and gevent worker class gunicorn-secscan stdout | 2020-03-31 08:50:21,449 [181] [DEBUG] [__config__] Starting secscan gunicorn with 4 workers and gevent worker class 2020-03-31 08:50:21,451 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:31.440707+00:00 (in 9.999241 seconds) repositoryactioncounter stdout | 2020-03-31 08:50:21,451 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:31.440707+00:00 (in 9.999241 seconds) 2020-03-31 08:50:21,471 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:50:21,471 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 08:50:21,471 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:50:21,471 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:21,471 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:50:21,471 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:31 UTC)" executed successfully 2020-03-31 08:50:21,537 [171] [INFO] [util.ipresolver] Finished building AWS IP ranges gunicorn-web stdout | 2020-03-31 08:50:21,537 [171] [INFO] [util.ipresolver] Finished building AWS IP ranges 2020-03-31 08:50:21,543 [171] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured gunicorn-web stdout | 2020-03-31 08:50:21,543 [171] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured 2020-03-31 08:50:21,813 [171] [DEBUG] [data.database] Configuring database gunicorn-web stdout | 2020-03-31 08:50:21,813 [171] [DEBUG] [data.database] Configuring database 2020-03-31 08:50:21,817 [171] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:50:21,817 [171] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:50:21,818 [171] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:21,818 [171] [INFO] [data.logs_model] =============================== gunicorn-web stdout | 2020-03-31 08:50:21,818 [171] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:21,818 [171] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:21,818 [171] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:21,819 [171] [INFO] [data.logs_model] =============================== gunicorn-web stdout | 2020-03-31 08:50:21,818 [171] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:21,819 [171] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:22 UTC)" (scheduled at 2020-03-31 08:50:22.113221+00:00) securityworker stdout | 2020-03-31 08:50:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:50:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:22 UTC)" (scheduled at 2020-03-31 08:50:22.113221+00:00) 2020-03-31 08:50:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:50:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:50:22,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:52.113221+00:00 (in 29.999504 seconds) securityworker stdout | 2020-03-31 08:50:22,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:52.113221+00:00 (in 29.999504 seconds) 2020-03-31 08:50:22,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:50:22,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:50:22,137 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:50:22,137 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:22,137 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:50:22,137 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:50:22,137 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:22,137 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:52 UTC)" executed successfully 2020-03-31 08:50:22,676 [171] [DEBUG] [__config__] Starting web gunicorn with 8 workers and gevent worker class gunicorn-web stdout | 2020-03-31 08:50:22,676 [171] [DEBUG] [__config__] Starting web gunicorn with 8 workers and gevent worker class 2020-03-31 08:50:23,274 [186] [INFO] [data.registry_model] =============================== gunicorn-verbs stdout | 2020-03-31 08:50:23,274 [186] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,274 [186] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:23,275 [186] [INFO] [data.registry_model] =============================== gunicorn-verbs stdout | 2020-03-31 08:50:23,274 [186] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:23,275 [186] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,336 [167] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2020-03-31 08:50:23,336 [167] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,336 [167] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:23,336 [167] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2020-03-31 08:50:23,336 [167] [INFO] [data.registry_model] Using registry model `` 2020-03-31 08:50:23,336 [167] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,456 [186] [DEBUG] [app] Loading default config. gunicorn-verbs stdout | 2020-03-31 08:50:23,456 [186] [DEBUG] [app] Loading default config. 2020-03-31 08:50:23,457 [186] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-verbs stdout | 2020-03-31 08:50:23,457 [186] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml 2020-03-31 08:50:23,492 [186] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None gunicorn-verbs stdout | 2020-03-31 08:50:23,492 [186] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None 2020-03-31 08:50:23,493 [186] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,493 [186] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:23,493 [186] [INFO] [data.registry_model] =============================== gunicorn-verbs stdout | 2020-03-31 08:50:23,493 [186] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,493 [186] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:23,493 [186] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,493 [186] [DEBUG] [app] Loaded config 2020-03-31 08:50:23,494 [186] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 2020-03-31 08:50:23,494 [186] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 gunicorn-verbs stdout | 2020-03-31 08:50:23,493 [186] [DEBUG] [app] Loaded config 2020-03-31 08:50:23,494 [186] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 2020-03-31 08:50:23,494 [186] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 2020-03-31 08:50:23,496 [186] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-verbs stdout | 2020-03-31 08:50:23,496 [186] [INFO] [util.ipresolver] Loading AWS IP ranges from disk 2020-03-31 08:50:23,507 [167] [DEBUG] [app] Loading default config. gunicorn-registry stdout | 2020-03-31 08:50:23,507 [167] [DEBUG] [app] Loading default config. 2020-03-31 08:50:23,508 [167] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-registry stdout | 2020-03-31 08:50:23,508 [167] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml 2020-03-31 08:50:23,512 [186] [INFO] [util.ipresolver] Building AWS IP ranges gunicorn-verbs stdout | 2020-03-31 08:50:23,512 [186] [INFO] [util.ipresolver] Building AWS IP ranges 2020-03-31 08:50:23,562 [167] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None gunicorn-registry stdout | 2020-03-31 08:50:23,562 [167] [ERROR] [util.config.provider.baseprovider] Could not validate config schema Traceback (most recent call last): File "/quay-registry/util/config/provider/baseprovider.py", line 42, in import_yaml validate(config_obj, CONFIG_SCHEMA) File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/jsonschema/validators.py", line 899, in validate raise error ValidationError: None is not of type 'string' Failed validating 'type' in schema['properties']['REPO_MIRROR_SERVER_HOSTNAME']: {'description': 'Replaces the SERVER_HOSTNAME as the destination for mirroring. Defaults to unset', 'type': 'string', 'x-example': 'openshift-quay-service'} On instance['REPO_MIRROR_SERVER_HOSTNAME']: None 2020-03-31 08:50:23,564 [167] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,564 [167] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:23,564 [167] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2020-03-31 08:50:23,564 [167] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,564 [167] [INFO] [data.registry_model] Full V2_2 + OCI model is enabled 2020-03-31 08:50:23,564 [167] [INFO] [data.registry_model] =============================== 2020-03-31 08:50:23,564 [167] [DEBUG] [app] Loaded config gunicorn-registry stdout | 2020-03-31 08:50:23,564 [167] [DEBUG] [app] Loaded config 2020-03-31 08:50:23,565 [167] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 2020-03-31 08:50:23,565 [167] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 gunicorn-registry stdout | 2020-03-31 08:50:23,565 [167] [DEBUG] [util.metrics.prometheus] Initializing prometheus with aggregator url: http://localhost:9092 2020-03-31 08:50:23,565 [167] [DEBUG] [util.metrics.prometheus] Prometheus aggregator sending to http://localhost:9092 2020-03-31 08:50:23,567 [167] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-registry stdout | 2020-03-31 08:50:23,567 [167] [INFO] [util.ipresolver] Loading AWS IP ranges from disk 2020-03-31 08:50:23,581 [167] [INFO] [util.ipresolver] Building AWS IP ranges gunicorn-registry stdout | 2020-03-31 08:50:23,581 [167] [INFO] [util.ipresolver] Building AWS IP ranges 2020-03-31 08:50:23,991 [186] [INFO] [util.ipresolver] Finished building AWS IP ranges gunicorn-verbs stdout | 2020-03-31 08:50:23,991 [186] [INFO] [util.ipresolver] Finished building AWS IP ranges 2020-03-31 08:50:23,995 [186] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured gunicorn-verbs stdout | 2020-03-31 08:50:23,995 [186] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured 2020-03-31 08:50:24,027 [167] [INFO] [util.ipresolver] Finished building AWS IP ranges gunicorn-registry stdout | 2020-03-31 08:50:24,027 [167] [INFO] [util.ipresolver] Finished building AWS IP ranges 2020-03-31 08:50:24,032 [167] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured gunicorn-registry stdout | 2020-03-31 08:50:24,032 [167] [DEBUG] [util.saas.cloudwatch] CloudWatch not configured 2020-03-31 08:50:24,307 [186] [DEBUG] [data.database] Configuring database gunicorn-verbs stdout | 2020-03-31 08:50:24,307 [186] [DEBUG] [data.database] Configuring database 2020-03-31 08:50:24,309 [186] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-verbs stdout | 2020-03-31 08:50:24,309 [186] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:50:24,309 [186] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:24,309 [186] [INFO] [data.logs_model] =============================== gunicorn-verbs stdout | 2020-03-31 08:50:24,309 [186] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:24,309 [186] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:24,309 [186] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:24,310 [186] [INFO] [data.logs_model] =============================== gunicorn-verbs stdout | 2020-03-31 08:50:24,309 [186] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:24,310 [186] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:50:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:24 UTC)" (scheduled at 2020-03-31 08:50:24.323352+00:00) 2020-03-31 08:50:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:50:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:24 UTC)" (scheduled at 2020-03-31 08:50:24.323352+00:00) 2020-03-31 08:50:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 24, 324728), True, datetime.datetime(2020, 3, 31, 8, 50, 24, 324728), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:50:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 24, 324728), True, datetime.datetime(2020, 3, 31, 8, 50, 24, 324728), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:50:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:34.323352+00:00 (in 9.999533 seconds) notificationworker stdout | 2020-03-31 08:50:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:34.323352+00:00 (in 9.999533 seconds) 2020-03-31 08:50:24,348 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:50:24,348 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:24,348 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:24,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:50:24,348 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:50:24,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:34 UTC)" executed successfully 2020-03-31 08:50:24,384 [186] [DEBUG] [__config__] Starting verbs gunicorn with 8 workers and sync worker class gunicorn-verbs stdout | 2020-03-31 08:50:24,384 [186] [DEBUG] [__config__] Starting verbs gunicorn with 8 workers and sync worker class 2020-03-31 08:50:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:50:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:24,413 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:50:24 UTC)" (scheduled at 2020-03-31 08:50:24.410927+00:00) security_notification_worker stdout | 2020-03-31 08:50:24,413 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:50:24 UTC)" (scheduled at 2020-03-31 08:50:24.410927+00:00) 2020-03-31 08:50:24,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:33.330112+00:00 (in 8.918742 seconds) 2020-03-31 08:50:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:24,414 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:50:24,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:33.330112+00:00 (in 8.918742 seconds) 2020-03-31 08:50:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:24,414 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:24 UTC)" executed successfully 2020-03-31 08:50:24,473 [167] [DEBUG] [data.database] Configuring database gunicorn-registry stdout | 2020-03-31 08:50:24,473 [167] [DEBUG] [data.database] Configuring database 2020-03-31 08:50:24,475 [167] [INFO] [data.database] Connection pooling enabled for postgresql; stale timeout: None; max connection count: None gunicorn-registry stdout | 2020-03-31 08:50:24,475 [167] [INFO] [data.database] Connection pooling enabled for postgresql; stale timeout: None; max connection count: None 2020-03-31 08:50:24,476 [167] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:24,476 [167] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2020-03-31 08:50:24,476 [167] [DEBUG] [data.logs_model] Configuring log lodel 2020-03-31 08:50:24,476 [167] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:24,476 [167] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:24,476 [167] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2020-03-31 08:50:24,476 [167] [INFO] [data.logs_model] Using logs model `` 2020-03-31 08:50:24,476 [167] [INFO] [data.logs_model] =============================== 2020-03-31 08:50:24,702 [167] [DEBUG] [__config__] Starting registry gunicorn with 16 workers and gevent worker class gunicorn-registry stdout | 2020-03-31 08:50:24,702 [167] [DEBUG] [__config__] Starting registry gunicorn with 16 workers and gevent worker class 2020-03-31 08:50:26,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:50:26,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:26 UTC)" (scheduled at 2020-03-31 08:50:26.924497+00:00) storagereplication stdout | 2020-03-31 08:50:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:26 UTC)" (scheduled at 2020-03-31 08:50:26.924497+00:00) 2020-03-31 08:50:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:50:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:26,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:36.924497+00:00 (in 9.999536 seconds) storagereplication stdout | 2020-03-31 08:50:26,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:36.924497+00:00 (in 9.999536 seconds) 2020-03-31 08:50:26,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 26, 926741), True, datetime.datetime(2020, 3, 31, 8, 50, 26, 926741), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:50:26,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 26, 926741), True, datetime.datetime(2020, 3, 31, 8, 50, 26, 926741), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:50:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:50:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:36 UTC)" executed successfully 2020-03-31 08:50:28,280 [519] [DEBUG] [app] Starting request: urn:request:946e5a44-1e07-4a84-a81d-997ca528724e (/health/instance) gunicorn-web stdout | 2020-03-31 08:50:28,280 [519] [DEBUG] [app] Starting request: urn:request:946e5a44-1e07-4a84-a81d-997ca528724e (/health/instance) 2020-03-31 08:50:28,290 [519] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 08:50:28,290 [519] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 08:50:28,301 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:28,301 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:28,307 [555] [DEBUG] [app] Starting request: urn:request:d1eeaf82-31a1-41d4-b100-7e6e67f47f60 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:28,307 [555] [DEBUG] [app] Starting request: urn:request:d1eeaf82-31a1-41d4-b100-7e6e67f47f60 (/v1/_internal_ping) 2020-03-31 08:50:28,310 [555] [DEBUG] [app] Ending request: urn:request:d1eeaf82-31a1-41d4-b100-7e6e67f47f60 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:28,310 [555] [DEBUG] [app] Ending request: urn:request:d1eeaf82-31a1-41d4-b100-7e6e67f47f60 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:50:28,312 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:50:28,312 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:28,314 [519] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:50:28,314 [519] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:50:28,311 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 08:50:28,311 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:28,323 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:50:28,323 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:50:28,327 [519] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:50:28,327 [519] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:50:28,329 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 108 gunicorn-registry stdout | 2020-03-31 08:50:28,329 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 108 2020-03-31 08:50:28,332 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:50:28,332 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:50:28,335 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:50:28,335 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:50:28,335 [555] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 09:50:28.335893 gunicorn-registry stdout | 2020-03-31 08:50:28,335 [555] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 09:50:28.335893 2020-03-31 08:50:28,343 [519] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:50:28,343 [519] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:50:28,349 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:28,349 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:28,356 [518] [DEBUG] [app] Starting request: urn:request:635877a1-31cc-4808-81c1-39c6fed0664f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:28,356 [518] [DEBUG] [app] Starting request: urn:request:635877a1-31cc-4808-81c1-39c6fed0664f (/_internal_ping) 2020-03-31 08:50:28,359 [518] [DEBUG] [app] Ending request: urn:request:635877a1-31cc-4808-81c1-39c6fed0664f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:28,359 [518] [DEBUG] [app] Ending request: urn:request:635877a1-31cc-4808-81c1-39c6fed0664f (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 159 0.010) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 159 0.010) 2020-03-31 08:50:28,361 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:50:28,361 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:28,364 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:50:28,364 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:28,365 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:28,365 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:28,366 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:50:28,366 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:50:28,369 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:28,369 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:28,370 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:50:28,370 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:50:28,374 [519] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 50, 28, 372388), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 50, 28, 372456), u'quay', None, datetime.datetime(2020, 3, 30, 8, 50, 28, 372496), datetime.datetime(2020, 3, 24, 8, 50, 28, 372525), None]) gunicorn-web stdout | 2020-03-31 08:50:28,374 [519] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 50, 28, 372388), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 50, 28, 372456), u'quay', None, datetime.datetime(2020, 3, 30, 8, 50, 28, 372496), datetime.datetime(2020, 3, 24, 8, 50, 28, 372525), None]) 2020-03-31 08:50:28,387 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:28,387 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:28,391 [537] [DEBUG] [app] Starting request: urn:request:8afc4a0b-6722-45ed-943b-8779cbfcc757 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:28,391 [537] [DEBUG] [app] Starting request: urn:request:8afc4a0b-6722-45ed-943b-8779cbfcc757 (/c1/_internal_ping) 2020-03-31 08:50:28,395 [537] [DEBUG] [app] Ending request: urn:request:8afc4a0b-6722-45ed-943b-8779cbfcc757 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:28,395 [537] [DEBUG] [app] Ending request: urn:request:8afc4a0b-6722-45ed-943b-8779cbfcc757 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:50:28,397 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:50:28,397 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:28,400 [519] [DEBUG] [app] Ending request: urn:request:946e5a44-1e07-4a84-a81d-997ca528724e (/health/instance) gunicorn-web stdout | 2020-03-31 08:50:28,400 [519] [DEBUG] [app] Ending request: urn:request:946e5a44-1e07-4a84-a81d-997ca528724e (/health/instance) 2020-03-31 08:50:28,400 [519] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:50:28,400 [519] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:28,401 [519] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.129 49 0.129) 2020-03-31 08:50:28,396 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:50:28,401 [519] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:50:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.129 49 0.129) gunicorn-verbs stdout | 2020-03-31 08:50:28,396 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:50:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:30 UTC)" (scheduled at 2020-03-31 08:50:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:50:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:30 UTC)" (scheduled at 2020-03-31 08:50:30.337699+00:00) 2020-03-31 08:50:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:00.337699+00:00 (in 29.999464 seconds) buildlogsarchiver stdout | 2020-03-31 08:50:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:00.337699+00:00 (in 29.999464 seconds) 2020-03-31 08:50:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 50, 30, 338936), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:50:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 50, 30, 338936), False, 50, 1, 0]) 2020-03-31 08:50:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:50:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:50:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:50:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:00 UTC)" executed successfully 2020-03-31 08:50:31,451 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:50:31,451 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:31,451 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:41.440707+00:00 (in 9.989398 seconds) 2020-03-31 08:50:31,451 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:41 UTC)" (scheduled at 2020-03-31 08:50:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:50:31,451 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:41.440707+00:00 (in 9.989398 seconds) 2020-03-31 08:50:31,451 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:41 UTC)" (scheduled at 2020-03-31 08:50:31.440707+00:00) 2020-03-31 08:50:31,454 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:50:31,454 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:50:31,479 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:50:31,479 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:50:31,480 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:31,480 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:50:31,480 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:31,480 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:41 UTC)" executed successfully 2020-03-31 08:50:33,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:50:33,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:33,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:03 UTC)" (scheduled at 2020-03-31 08:50:33.330112+00:00) 2020-03-31 08:50:33,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:03.330112+00:00 (in 29.996800 seconds) 2020-03-31 08:50:33,334 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:50:33,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:03 UTC)" (scheduled at 2020-03-31 08:50:33.330112+00:00) 2020-03-31 08:50:33,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:03.330112+00:00 (in 29.996800 seconds) 2020-03-31 08:50:33,334 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:33,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 33, 334564), True, datetime.datetime(2020, 3, 31, 8, 50, 33, 334564), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:50:33,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 33, 334564), True, datetime.datetime(2020, 3, 31, 8, 50, 33, 334564), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:50:33,353 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:50:33,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:33,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:50:33,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:03 UTC)" executed successfully 2020-03-31 08:50:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:50:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:43.539999+00:00 (in 10.172791 seconds) exportactionlogsworker stdout | 2020-03-31 08:50:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:43.539999+00:00 (in 10.172791 seconds) 2020-03-31 08:50:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:33 UTC)" (scheduled at 2020-03-31 08:50:33.366791+00:00) 2020-03-31 08:50:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:50:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:33 UTC)" (scheduled at 2020-03-31 08:50:33.366791+00:00) 2020-03-31 08:50:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 33, 369086), True, datetime.datetime(2020, 3, 31, 8, 50, 33, 369086), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:50:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 33, 369086), True, datetime.datetime(2020, 3, 31, 8, 50, 33, 369086), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:50:33,386 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:50:33,386 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:33,386 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 08:50:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:50:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:33 UTC)" executed successfully 2020-03-31 08:50:34,329 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:50:34,329 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:34,330 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:44.323352+00:00 (in 9.993711 seconds) 2020-03-31 08:50:34,330 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:44 UTC)" (scheduled at 2020-03-31 08:50:34.323352+00:00) 2020-03-31 08:50:34,330 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:50:34,330 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:44.323352+00:00 (in 9.993711 seconds) 2020-03-31 08:50:34,330 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:44 UTC)" (scheduled at 2020-03-31 08:50:34.323352+00:00) 2020-03-31 08:50:34,330 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:34,333 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 34, 330778), True, datetime.datetime(2020, 3, 31, 8, 50, 34, 330778), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:50:34,333 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 34, 330778), True, datetime.datetime(2020, 3, 31, 8, 50, 34, 330778), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:50:34,350 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:50:34,350 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:34,350 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:34,350 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:50:34,350 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:34,350 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:44 UTC)" executed successfully 2020-03-31 08:50:36,929 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:50:36,929 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:36,930 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:38.010934+00:00 (in 1.081373 seconds) 2020-03-31 08:50:36,930 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:46 UTC)" (scheduled at 2020-03-31 08:50:36.924497+00:00) 2020-03-31 08:50:36,930 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:50:36,930 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:38.010934+00:00 (in 1.081373 seconds) 2020-03-31 08:50:36,930 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:46 UTC)" (scheduled at 2020-03-31 08:50:36.924497+00:00) 2020-03-31 08:50:36,930 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:36,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 36, 930885), True, datetime.datetime(2020, 3, 31, 8, 50, 36, 930885), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:50:36,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 36, 930885), True, datetime.datetime(2020, 3, 31, 8, 50, 36, 930885), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:50:36,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:36,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:36,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:50:36,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:36,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:36,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:46 UTC)" executed successfully 2020-03-31 08:50:38,012 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:50:38,012 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:46.924497+00:00 (in 8.912021 seconds) 2020-03-31 08:50:38,013 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:38 UTC)" (scheduled at 2020-03-31 08:50:38.010934+00:00) 2020-03-31 08:50:38,013 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:38,013 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:50:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:46.924497+00:00 (in 8.912021 seconds) 2020-03-31 08:50:38,013 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:38 UTC)" (scheduled at 2020-03-31 08:50:38.010934+00:00) 2020-03-31 08:50:38,013 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:38,013 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:38 UTC)" executed successfully 2020-03-31 08:50:38,273 [518] [DEBUG] [app] Starting request: urn:request:b1f844ba-c874-4331-aa4a-e8ba7c61f641 (/health/instance) gunicorn-web stdout | 2020-03-31 08:50:38,273 [518] [DEBUG] [app] Starting request: urn:request:b1f844ba-c874-4331-aa4a-e8ba7c61f641 (/health/instance) 2020-03-31 08:50:38,280 [518] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 08:50:38,280 [518] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 08:50:38,289 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:38,289 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:38,295 [538] [DEBUG] [app] Starting request: urn:request:41a2154e-e7df-46b6-b4c8-d9ebb89e8996 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:38,295 [538] [DEBUG] [app] Starting request: urn:request:41a2154e-e7df-46b6-b4c8-d9ebb89e8996 (/v1/_internal_ping) 2020-03-31 08:50:38,298 [538] [DEBUG] [app] Ending request: urn:request:41a2154e-e7df-46b6-b4c8-d9ebb89e8996 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:38,298 [538] [DEBUG] [app] Ending request: urn:request:41a2154e-e7df-46b6-b4c8-d9ebb89e8996 (/v1/_internal_ping) 2020-03-31 08:50:38,300 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-web stdout | 2020-03-31 08:50:38,300 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:38,300 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-registry stdout | 2020-03-31 08:50:38,300 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:38,302 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:50:38,302 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:50:38,307 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:50:38,307 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:50:38,313 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:50:38,313 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:50:38,314 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:50:38,314 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:50:38,319 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:50:38,319 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:50:38,332 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:50:38,332 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:50:38,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:38,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:38,338 [518] [DEBUG] [app] Starting request: urn:request:2f31c110-f819-4a1c-9d15-887c970758bd (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:38,338 [518] [DEBUG] [app] Starting request: urn:request:2f31c110-f819-4a1c-9d15-887c970758bd (/_internal_ping) 2020-03-31 08:50:38,339 [518] [DEBUG] [app] Ending request: urn:request:2f31c110-f819-4a1c-9d15-887c970758bd (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:38,339 [518] [DEBUG] [app] Ending request: urn:request:2f31c110-f819-4a1c-9d15-887c970758bd (/_internal_ping) 2020-03-31 08:50:38,340 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:50:38,340 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:38,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:50:38,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:38,344 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:38,344 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:38,345 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:50:38,345 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:50:38,348 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:38,348 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:50:38,349 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:50:38,349 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:50:38,352 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 50, 38, 351030), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 50, 38, 351079), u'quay', None, datetime.datetime(2020, 3, 30, 8, 50, 38, 351105), datetime.datetime(2020, 3, 24, 8, 50, 38, 351122), None]) gunicorn-web stdout | 2020-03-31 08:50:38,352 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 50, 38, 351030), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 50, 38, 351079), u'quay', None, datetime.datetime(2020, 3, 30, 8, 50, 38, 351105), datetime.datetime(2020, 3, 24, 8, 50, 38, 351122), None]) 2020-03-31 08:50:38,365 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:38,365 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:38,366 [537] [DEBUG] [app] Starting request: urn:request:a11df707-5fe4-4eb5-8908-7a3dc18e3704 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:38,366 [537] [DEBUG] [app] Starting request: urn:request:a11df707-5fe4-4eb5-8908-7a3dc18e3704 (/c1/_internal_ping) 2020-03-31 08:50:38,367 [537] [DEBUG] [app] Ending request: urn:request:a11df707-5fe4-4eb5-8908-7a3dc18e3704 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:38,367 [537] [DEBUG] [app] Ending request: urn:request:a11df707-5fe4-4eb5-8908-7a3dc18e3704 (/c1/_internal_ping) 2020-03-31 08:50:38,368 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:50:38,368 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:50:38,368 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:50:38,368 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:38,369 [518] [DEBUG] [app] Ending request: urn:request:b1f844ba-c874-4331-aa4a-e8ba7c61f641 (/health/instance) 2020-03-31 08:50:38,369 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:50:38,369 [518] [DEBUG] [app] Ending request: urn:request:b1f844ba-c874-4331-aa4a-e8ba7c61f641 (/health/instance) 2020-03-31 08:50:38,369 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:38,370 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.098 49 0.098) gunicorn-web stdout | 2020-03-31 08:50:38,370 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:50:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.098 49 0.098) 2020-03-31 08:50:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:50:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:51.440707+00:00 (in 9.999283 seconds) 2020-03-31 08:50:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:51 UTC)" (scheduled at 2020-03-31 08:50:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:50:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:51.440707+00:00 (in 9.999283 seconds) 2020-03-31 08:50:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:51 UTC)" (scheduled at 2020-03-31 08:50:41.440707+00:00) 2020-03-31 08:50:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:50:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:50:41,471 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:50:41,471 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:50:41,472 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:41,472 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:50:41,472 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:41,472 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:51 UTC)" executed successfully 2020-03-31 08:50:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:50:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:43,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:33.366791+00:00 (in 49.824967 seconds) 2020-03-31 08:50:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:43 UTC)" (scheduled at 2020-03-31 08:50:43.539999+00:00) 2020-03-31 08:50:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:50:43,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:33.366791+00:00 (in 49.824967 seconds) 2020-03-31 08:50:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:43 UTC)" (scheduled at 2020-03-31 08:50:43.539999+00:00) 2020-03-31 08:50:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:43 UTC)" executed successfully 2020-03-31 08:50:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:50:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:49.498966+00:00 (in 5.174267 seconds) 2020-03-31 08:50:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:54 UTC)" (scheduled at 2020-03-31 08:50:44.323352+00:00) notificationworker stdout | 2020-03-31 08:50:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:49.498966+00:00 (in 5.174267 seconds) 2020-03-31 08:50:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:54 UTC)" (scheduled at 2020-03-31 08:50:44.323352+00:00) 2020-03-31 08:50:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:50:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 44, 325900), True, datetime.datetime(2020, 3, 31, 8, 50, 44, 325900), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:50:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 44, 325900), True, datetime.datetime(2020, 3, 31, 8, 50, 44, 325900), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:50:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:50:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:50:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:54 UTC)" executed successfully 2020-03-31 08:50:45,064 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:45 UTC)" (scheduled at 2020-03-31 08:50:45.064514+00:00) gcworker stdout | 2020-03-31 08:50:45,064 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:50:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:50:45 UTC)" (scheduled at 2020-03-31 08:50:45.064514+00:00) 2020-03-31 08:50:45,066 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) 2020-03-31 08:50:45,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:15.064514+00:00 (in 29.999367 seconds) gcworker stdout | 2020-03-31 08:50:45,066 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) 2020-03-31 08:50:45,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:15.064514+00:00 (in 29.999367 seconds) 2020-03-31 08:50:45,082 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435045, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:50:45,082 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435045, 1209600, 500, 1, 0]) 2020-03-31 08:50:45,088 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:50:45,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:50:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:50:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:15 UTC)" executed successfully 2020-03-31 08:50:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:50:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:56.924497+00:00 (in 9.999204 seconds) 2020-03-31 08:50:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:56 UTC)" (scheduled at 2020-03-31 08:50:46.924497+00:00) 2020-03-31 08:50:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:50:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:56.924497+00:00 (in 9.999204 seconds) 2020-03-31 08:50:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:56 UTC)" (scheduled at 2020-03-31 08:50:46.924497+00:00) 2020-03-31 08:50:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 46, 925957), True, datetime.datetime(2020, 3, 31, 8, 50, 46, 925957), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:50:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 46, 925957), True, datetime.datetime(2020, 3, 31, 8, 50, 46, 925957), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:50:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:50:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:50:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:50:56 UTC)" executed successfully 2020-03-31 08:50:48,272 [518] [DEBUG] [app] Starting request: urn:request:29e3c57b-091a-4547-941d-a91eaef84786 (/health/instance) gunicorn-web stdout | 2020-03-31 08:50:48,272 [518] [DEBUG] [app] Starting request: urn:request:29e3c57b-091a-4547-941d-a91eaef84786 (/health/instance) 2020-03-31 08:50:48,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:50:48,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:50:48,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:48,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:48,280 [555] [DEBUG] [app] Starting request: urn:request:24b21248-a6ae-438d-9a09-3a76470f9bc7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:48,280 [555] [DEBUG] [app] Starting request: urn:request:24b21248-a6ae-438d-9a09-3a76470f9bc7 (/v1/_internal_ping) 2020-03-31 08:50:48,281 [555] [DEBUG] [app] Ending request: urn:request:24b21248-a6ae-438d-9a09-3a76470f9bc7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:48,281 [555] [DEBUG] [app] Ending request: urn:request:24b21248-a6ae-438d-9a09-3a76470f9bc7 (/v1/_internal_ping) 2020-03-31 08:50:48,281 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:50:48,281 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:50:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:50:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:48,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:50:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:50:48,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:50:48,285 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:50:48,285 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:50:48,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:50:48,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:50:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:50:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:50:48,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:50:48,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:50:48,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:48,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:48,315 [518] [DEBUG] [app] Starting request: urn:request:9d83bfa7-1cdd-461c-843a-7d65236c93ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:48,315 [518] [DEBUG] [app] Starting request: urn:request:9d83bfa7-1cdd-461c-843a-7d65236c93ac (/_internal_ping) 2020-03-31 08:50:48,316 [518] [DEBUG] [app] Ending request: urn:request:9d83bfa7-1cdd-461c-843a-7d65236c93ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:48,316 [518] [DEBUG] [app] Ending request: urn:request:9d83bfa7-1cdd-461c-843a-7d65236c93ac (/_internal_ping) 2020-03-31 08:50:48,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:50:48,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:50:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:50:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:50:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:50:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:50:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:50:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:48,328 [537] [DEBUG] [app] Starting request: urn:request:6b5d61ac-21ee-4e97-9bf0-c9eda3c1ab26 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:48,328 [537] [DEBUG] [app] Starting request: urn:request:6b5d61ac-21ee-4e97-9bf0-c9eda3c1ab26 (/c1/_internal_ping) 2020-03-31 08:50:48,329 [537] [DEBUG] [app] Ending request: urn:request:6b5d61ac-21ee-4e97-9bf0-c9eda3c1ab26 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:48,329 [537] [DEBUG] [app] Ending request: urn:request:6b5d61ac-21ee-4e97-9bf0-c9eda3c1ab26 (/c1/_internal_ping) 2020-03-31 08:50:48,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:50:48,330 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:50:48,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:50:48,330 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:48,331 [518] [DEBUG] [app] Ending request: urn:request:29e3c57b-091a-4547-941d-a91eaef84786 (/health/instance) 2020-03-31 08:50:48,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:48,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:50:48,331 [518] [DEBUG] [app] Ending request: urn:request:29e3c57b-091a-4547-941d-a91eaef84786 (/health/instance) 2020-03-31 08:50:48,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:50:48,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:50:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 08:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:54.323352+00:00 (in 4.822821 seconds) 2020-03-31 08:50:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:49 UTC)" (scheduled at 2020-03-31 08:50:49.498966+00:00) 2020-03-31 08:50:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:54.323352+00:00 (in 4.822821 seconds) 2020-03-31 08:50:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:49 UTC)" (scheduled at 2020-03-31 08:50:49.498966+00:00) 2020-03-31 08:50:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:49 UTC)" executed successfully 2020-03-31 08:50:51,110 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:50:51,110 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:59.560326+00:00 (in 8.449293 seconds) 2020-03-31 08:50:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:51 UTC)" (scheduled at 2020-03-31 08:50:51.110610+00:00) 2020-03-31 08:50:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:50:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:50:59.560326+00:00 (in 8.449293 seconds) 2020-03-31 08:50:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:51 UTC)" (scheduled at 2020-03-31 08:50:51.110610+00:00) 2020-03-31 08:50:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 51, 112294), True, datetime.datetime(2020, 3, 31, 8, 50, 51, 112294), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:50:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 51, 112294), True, datetime.datetime(2020, 3, 31, 8, 50, 51, 112294), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:50:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:50:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:50:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:51 UTC)" executed successfully 2020-03-31 08:50:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:50:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:01.440707+00:00 (in 9.999002 seconds) 2020-03-31 08:50:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:01 UTC)" (scheduled at 2020-03-31 08:50:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:50:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:01.440707+00:00 (in 9.999002 seconds) 2020-03-31 08:50:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:01 UTC)" (scheduled at 2020-03-31 08:50:51.440707+00:00) 2020-03-31 08:50:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:50:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:50:51,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:50:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:50:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:50:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:01 UTC)" executed successfully 2020-03-31 08:50:52,117 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:50:52,117 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:52,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:22.113221+00:00 (in 29.995791 seconds) 2020-03-31 08:50:52,117 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:22 UTC)" (scheduled at 2020-03-31 08:50:52.113221+00:00) securityworker stdout | 2020-03-31 08:50:52,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:22.113221+00:00 (in 29.995791 seconds) 2020-03-31 08:50:52,117 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:22 UTC)" (scheduled at 2020-03-31 08:50:52.113221+00:00) 2020-03-31 08:50:52,118 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:50:52,118 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:50:52,134 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:50:52,134 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:50:52,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:50:52,136 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 08:50:52,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:50:52,136 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:52,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:50:52,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:22 UTC)" executed successfully 2020-03-31 08:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:04.323352+00:00 (in 9.998879 seconds) 2020-03-31 08:50:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:04 UTC)" (scheduled at 2020-03-31 08:50:54.323352+00:00) 2020-03-31 08:50:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:04.323352+00:00 (in 9.998879 seconds) 2020-03-31 08:50:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:04 UTC)" (scheduled at 2020-03-31 08:50:54.323352+00:00) 2020-03-31 08:50:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 54, 325392), True, datetime.datetime(2020, 3, 31, 8, 50, 54, 325392), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:50:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 54, 325392), True, datetime.datetime(2020, 3, 31, 8, 50, 54, 325392), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:50:54,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:50:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:50:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:04 UTC)" executed successfully 2020-03-31 08:50:56,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:06.924497+00:00 (in 9.999355 seconds) storagereplication stdout | 2020-03-31 08:50:56,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:50:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:06.924497+00:00 (in 9.999355 seconds) 2020-03-31 08:50:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:06 UTC)" (scheduled at 2020-03-31 08:50:56.924497+00:00) 2020-03-31 08:50:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:50:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:06 UTC)" (scheduled at 2020-03-31 08:50:56.924497+00:00) 2020-03-31 08:50:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:50:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 56, 926927), True, datetime.datetime(2020, 3, 31, 8, 50, 56, 926927), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:50:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 50, 56, 926927), True, datetime.datetime(2020, 3, 31, 8, 50, 56, 926927), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:50:56,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:50:56,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:50:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:50:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:50:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:06 UTC)" executed successfully 2020-03-31 08:50:58,272 [518] [DEBUG] [app] Starting request: urn:request:9548d720-13c9-4291-ba75-b7c46db720c4 (/health/instance) gunicorn-web stdout | 2020-03-31 08:50:58,272 [518] [DEBUG] [app] Starting request: urn:request:9548d720-13c9-4291-ba75-b7c46db720c4 (/health/instance) 2020-03-31 08:50:58,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:50:58,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:50:58,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:58,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:58,283 [553] [DEBUG] [app] Starting request: urn:request:2ce386d2-9d2d-4e26-8a52-7ae91b896d00 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:58,283 [553] [DEBUG] [app] Starting request: urn:request:2ce386d2-9d2d-4e26-8a52-7ae91b896d00 (/v1/_internal_ping) 2020-03-31 08:50:58,286 [553] [DEBUG] [app] Ending request: urn:request:2ce386d2-9d2d-4e26-8a52-7ae91b896d00 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:50:58,286 [553] [DEBUG] [app] Ending request: urn:request:2ce386d2-9d2d-4e26-8a52-7ae91b896d00 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) 2020-03-31 08:50:58,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:50:58,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:58,288 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 08:50:58,288 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:58,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:50:58,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:50:58,294 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:50:58,294 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:50:58,299 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:50:58,299 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:50:58,300 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:50:58,300 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:50:58,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:50:58,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:50:58,321 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:50:58,321 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:50:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:58,327 [518] [DEBUG] [app] Starting request: urn:request:3503cf9c-c63b-4897-a296-61799d6eeeea (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:58,327 [518] [DEBUG] [app] Starting request: urn:request:3503cf9c-c63b-4897-a296-61799d6eeeea (/_internal_ping) 2020-03-31 08:50:58,328 [518] [DEBUG] [app] Ending request: urn:request:3503cf9c-c63b-4897-a296-61799d6eeeea (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:50:58,328 [518] [DEBUG] [app] Ending request: urn:request:3503cf9c-c63b-4897-a296-61799d6eeeea (/_internal_ping) 2020-03-31 08:50:58,329 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:50:58,329 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:50:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:50:58,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:58,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:58,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:50:58,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:50:58,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:58,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:50:58,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:50:58,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:50:58,339 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:50:58,339 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:50:58,341 [537] [DEBUG] [app] Starting request: urn:request:47dc98bc-dfc5-4928-beb0-2078030c4a28 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:58,341 [537] [DEBUG] [app] Starting request: urn:request:47dc98bc-dfc5-4928-beb0-2078030c4a28 (/c1/_internal_ping) 2020-03-31 08:50:58,342 [537] [DEBUG] [app] Ending request: urn:request:47dc98bc-dfc5-4928-beb0-2078030c4a28 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:50:58,342 [537] [DEBUG] [app] Ending request: urn:request:47dc98bc-dfc5-4928-beb0-2078030c4a28 (/c1/_internal_ping) 2020-03-31 08:50:58,343 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:50:58,343 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:50:58,343 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:50:58,343 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:50:58,344 [518] [DEBUG] [app] Ending request: urn:request:9548d720-13c9-4291-ba75-b7c46db720c4 (/health/instance) 2020-03-31 08:50:58,344 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:50:58,344 [518] [DEBUG] [app] Ending request: urn:request:9548d720-13c9-4291-ba75-b7c46db720c4 (/health/instance) 2020-03-31 08:50:58,344 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:50:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) 2020-03-31 08:50:58,345 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:50:58,345 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:50:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:51.110610+00:00 (in 51.549154 seconds) 2020-03-31 08:50:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:59 UTC)" (scheduled at 2020-03-31 08:50:59.560326+00:00) 2020-03-31 08:50:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:51.110610+00:00 (in 51.549154 seconds) 2020-03-31 08:50:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:59 UTC)" (scheduled at 2020-03-31 08:50:59.560326+00:00) 2020-03-31 08:50:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:50:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:51:59 UTC)" executed successfully 2020-03-31 08:51:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:51:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:30.337699+00:00 (in 29.997903 seconds) 2020-03-31 08:51:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:30 UTC)" (scheduled at 2020-03-31 08:51:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:51:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:30.337699+00:00 (in 29.997903 seconds) 2020-03-31 08:51:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:30 UTC)" (scheduled at 2020-03-31 08:51:00.337699+00:00) 2020-03-31 08:51:00,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 51, 0, 340729), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:51:00,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 51, 0, 340729), False, 50, 1, 0]) 2020-03-31 08:51:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:51:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:51:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:51:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:30 UTC)" executed successfully 2020-03-31 08:51:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:51:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:11.440707+00:00 (in 9.999146 seconds) 2020-03-31 08:51:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:11 UTC)" (scheduled at 2020-03-31 08:51:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:51:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:11.440707+00:00 (in 9.999146 seconds) 2020-03-31 08:51:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:11 UTC)" (scheduled at 2020-03-31 08:51:01.440707+00:00) 2020-03-31 08:51:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:51:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:51:01,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:51:01,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:51:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:51:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:11 UTC)" executed successfully 2020-03-31 08:51:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:51:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:24.410927+00:00 (in 21.079269 seconds) 2020-03-31 08:51:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:33 UTC)" (scheduled at 2020-03-31 08:51:03.330112+00:00) 2020-03-31 08:51:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:51:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:24.410927+00:00 (in 21.079269 seconds) 2020-03-31 08:51:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:33 UTC)" (scheduled at 2020-03-31 08:51:03.330112+00:00) 2020-03-31 08:51:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 3, 332680), True, datetime.datetime(2020, 3, 31, 8, 51, 3, 332680), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:51:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 3, 332680), True, datetime.datetime(2020, 3, 31, 8, 51, 3, 332680), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:51:03,354 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:51:03,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:51:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:33 UTC)" executed successfully 2020-03-31 08:51:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:51:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:14 UTC)" (scheduled at 2020-03-31 08:51:04.323352+00:00) 2020-03-31 08:51:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:51:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:14 UTC)" (scheduled at 2020-03-31 08:51:04.323352+00:00) 2020-03-31 08:51:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 4, 325270), True, datetime.datetime(2020, 3, 31, 8, 51, 4, 325270), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:51:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 4, 325270), True, datetime.datetime(2020, 3, 31, 8, 51, 4, 325270), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:51:04,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:14.323352+00:00 (in 9.998966 seconds) notificationworker stdout | 2020-03-31 08:51:04,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:14.323352+00:00 (in 9.998966 seconds) 2020-03-31 08:51:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:51:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:51:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:14 UTC)" executed successfully 2020-03-31 08:51:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:51:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:06 UTC)" (scheduled at 2020-03-31 08:51:06.924497+00:00) 2020-03-31 08:51:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:51:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:06 UTC)" (scheduled at 2020-03-31 08:51:06.924497+00:00) 2020-03-31 08:51:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 6, 925869), True, datetime.datetime(2020, 3, 31, 8, 51, 6, 925869), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:51:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 6, 925869), True, datetime.datetime(2020, 3, 31, 8, 51, 6, 925869), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:51:06,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:16.924497+00:00 (in 9.999331 seconds) storagereplication stdout | 2020-03-31 08:51:06,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:16.924497+00:00 (in 9.999331 seconds) 2020-03-31 08:51:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:51:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:51:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:16 UTC)" executed successfully 2020-03-31 08:51:08,277 [526] [DEBUG] [app] Starting request: urn:request:75c3fe32-4da2-472a-a399-6353ea263296 (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:08,277 [526] [DEBUG] [app] Starting request: urn:request:75c3fe32-4da2-472a-a399-6353ea263296 (/health/instance) 2020-03-31 08:51:08,285 [526] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 08:51:08,285 [526] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 08:51:08,295 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:08,295 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:08,297 [538] [DEBUG] [app] Starting request: urn:request:b41918ff-9ffc-427e-bfcc-d61b480d80f8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:08,297 [538] [DEBUG] [app] Starting request: urn:request:b41918ff-9ffc-427e-bfcc-d61b480d80f8 (/v1/_internal_ping) 2020-03-31 08:51:08,298 [538] [DEBUG] [app] Ending request: urn:request:b41918ff-9ffc-427e-bfcc-d61b480d80f8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:08,298 [538] [DEBUG] [app] Ending request: urn:request:b41918ff-9ffc-427e-bfcc-d61b480d80f8 (/v1/_internal_ping) 2020-03-31 08:51:08,299 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:51:08,299 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:08,299 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 08:51:08,299 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:08,301 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:08,301 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:51:08,302 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:51:08,302 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:08,302 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:51:08,302 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:51:08,304 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:08,304 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:08,313 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:51:08,313 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:08,331 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:08,331 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:08,335 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:08,335 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:08,341 [527] [DEBUG] [app] Starting request: urn:request:090d7b5e-583d-4620-a3ec-55160a278cb3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:08,341 [527] [DEBUG] [app] Starting request: urn:request:090d7b5e-583d-4620-a3ec-55160a278cb3 (/_internal_ping) 2020-03-31 08:51:08,344 [527] [DEBUG] [app] Ending request: urn:request:090d7b5e-583d-4620-a3ec-55160a278cb3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:08,344 [527] [DEBUG] [app] Ending request: urn:request:090d7b5e-583d-4620-a3ec-55160a278cb3 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 159 0.010) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 159 0.010) 2020-03-31 08:51:08,346 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:08,345 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:08,346 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:08,345 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:08,349 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:08,349 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:08,349 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:51:08,349 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:51:08,352 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:08,352 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:51:08,353 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:51:08,353 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:51:08,356 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 51, 8, 354738), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 51, 8, 354782), u'quay', None, datetime.datetime(2020, 3, 30, 8, 51, 8, 354807), datetime.datetime(2020, 3, 24, 8, 51, 8, 354824), None]) gunicorn-web stdout | 2020-03-31 08:51:08,356 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 51, 8, 354738), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 51, 8, 354782), u'quay', None, datetime.datetime(2020, 3, 30, 8, 51, 8, 354807), datetime.datetime(2020, 3, 24, 8, 51, 8, 354824), None]) 2020-03-31 08:51:08,369 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:08,369 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:08,371 [537] [DEBUG] [app] Starting request: urn:request:abefbea3-48a9-4b9f-97b3-76ee761c7a05 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:08,371 [537] [DEBUG] [app] Starting request: urn:request:abefbea3-48a9-4b9f-97b3-76ee761c7a05 (/c1/_internal_ping) 2020-03-31 08:51:08,372 [537] [DEBUG] [app] Ending request: urn:request:abefbea3-48a9-4b9f-97b3-76ee761c7a05 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:08,372 [537] [DEBUG] [app] Ending request: urn:request:abefbea3-48a9-4b9f-97b3-76ee761c7a05 (/c1/_internal_ping) 2020-03-31 08:51:08,373 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:51:08,372 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:08,373 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:51:08,372 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:08,375 [526] [DEBUG] [app] Ending request: urn:request:75c3fe32-4da2-472a-a399-6353ea263296 (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:08,375 [526] [DEBUG] [app] Ending request: urn:request:75c3fe32-4da2-472a-a399-6353ea263296 (/health/instance) 2020-03-31 08:51:08,375 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:51:08,375 [526] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.105 49 0.105) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.105 49 0.105) 2020-03-31 08:51:08,376 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:51:08,376 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:51:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:51:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:21.440707+00:00 (in 9.999073 seconds) 2020-03-31 08:51:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:21 UTC)" (scheduled at 2020-03-31 08:51:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:51:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:21.440707+00:00 (in 9.999073 seconds) 2020-03-31 08:51:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:21 UTC)" (scheduled at 2020-03-31 08:51:11.440707+00:00) 2020-03-31 08:51:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:51:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:51:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:51:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:51:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:51:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:21 UTC)" executed successfully 2020-03-31 08:51:14,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:51:14,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:14,329 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:24.323352+00:00 (in 9.994617 seconds) 2020-03-31 08:51:14,329 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:24 UTC)" (scheduled at 2020-03-31 08:51:14.323352+00:00) 2020-03-31 08:51:14,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:51:14,329 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:24.323352+00:00 (in 9.994617 seconds) 2020-03-31 08:51:14,329 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:24 UTC)" (scheduled at 2020-03-31 08:51:14.323352+00:00) 2020-03-31 08:51:14,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 14, 329823), True, datetime.datetime(2020, 3, 31, 8, 51, 14, 329823), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:51:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 14, 329823), True, datetime.datetime(2020, 3, 31, 8, 51, 14, 329823), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:51:14,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:14,348 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:51:14,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:14,348 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:14,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:51:14,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:24 UTC)" executed successfully 2020-03-31 08:51:15,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:51:15,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:15,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:45.064514+00:00 (in 29.996931 seconds) 2020-03-31 08:51:15,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:45 UTC)" (scheduled at 2020-03-31 08:51:15.064514+00:00) gcworker stdout | 2020-03-31 08:51:15,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:45.064514+00:00 (in 29.996931 seconds) 2020-03-31 08:51:15,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:45 UTC)" (scheduled at 2020-03-31 08:51:15.064514+00:00) 2020-03-31 08:51:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435075, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:51:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435075, 1209600, 500, 1, 0]) 2020-03-31 08:51:15,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:51:15,086 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 08:51:15,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:51:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:51:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:45 UTC)" executed successfully 2020-03-31 08:51:16,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:51:16,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:16,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:26.924497+00:00 (in 9.995715 seconds) 2020-03-31 08:51:16,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:26 UTC)" (scheduled at 2020-03-31 08:51:16.924497+00:00) 2020-03-31 08:51:16,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:51:16,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:26.924497+00:00 (in 9.995715 seconds) 2020-03-31 08:51:16,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:26 UTC)" (scheduled at 2020-03-31 08:51:16.924497+00:00) 2020-03-31 08:51:16,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:16,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 16, 929774), True, datetime.datetime(2020, 3, 31, 8, 51, 16, 929774), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:51:16,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 16, 929774), True, datetime.datetime(2020, 3, 31, 8, 51, 16, 929774), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:51:16,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:51:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:51:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:26 UTC)" executed successfully 2020-03-31 08:51:18,272 [526] [DEBUG] [app] Starting request: urn:request:2d02b1f4-979b-4fb3-a5c9-009d8d1c1055 (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:18,272 [526] [DEBUG] [app] Starting request: urn:request:2d02b1f4-979b-4fb3-a5c9-009d8d1c1055 (/health/instance) 2020-03-31 08:51:18,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:18,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:18,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:18,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:18,283 [550] [DEBUG] [app] Starting request: urn:request:f20912b1-eef8-45b3-931b-b08719b4f0df (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:18,283 [550] [DEBUG] [app] Starting request: urn:request:f20912b1-eef8-45b3-931b-b08719b4f0df (/v1/_internal_ping) 2020-03-31 08:51:18,287 [550] [DEBUG] [app] Ending request: urn:request:f20912b1-eef8-45b3-931b-b08719b4f0df (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:18,287 [550] [DEBUG] [app] Ending request: urn:request:f20912b1-eef8-45b3-931b-b08719b4f0df (/v1/_internal_ping) 2020-03-31 08:51:18,288 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-web stdout | 2020-03-31 08:51:18,288 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:18,288 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:18,289 [526] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-registry stdout | 2020-03-31 08:51:18,288 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:18,289 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:18,295 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:18,295 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:51:18,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:51:18,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:18,300 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:51:18,300 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:51:18,301 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:18,301 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:18,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:18,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:18,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:18,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:18,324 [523] [DEBUG] [app] Starting request: urn:request:851f2177-9abe-4bc4-a9e3-94a9381d4298 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:18,324 [523] [DEBUG] [app] Starting request: urn:request:851f2177-9abe-4bc4-a9e3-94a9381d4298 (/_internal_ping) 2020-03-31 08:51:18,327 [523] [DEBUG] [app] Ending request: urn:request:851f2177-9abe-4bc4-a9e3-94a9381d4298 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:18,327 [523] [DEBUG] [app] Ending request: urn:request:851f2177-9abe-4bc4-a9e3-94a9381d4298 (/_internal_ping) 2020-03-31 08:51:18,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 159 0.009) gunicorn-web stdout | 2020-03-31 08:51:18,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:18,329 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 159 0.009) gunicorn-web stdout | 2020-03-31 08:51:18,329 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:18,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:18,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:18,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:51:18,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:51:18,335 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:18,335 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:18,336 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:51:18,336 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:51:18,339 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:18,339 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:18,343 [539] [DEBUG] [app] Starting request: urn:request:a2f70747-d8b4-4099-b163-a98773979177 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:18,343 [539] [DEBUG] [app] Starting request: urn:request:a2f70747-d8b4-4099-b163-a98773979177 (/c1/_internal_ping) 2020-03-31 08:51:18,347 [539] [DEBUG] [app] Ending request: urn:request:a2f70747-d8b4-4099-b163-a98773979177 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:18,347 [539] [DEBUG] [app] Ending request: urn:request:a2f70747-d8b4-4099-b163-a98773979177 (/c1/_internal_ping) 2020-03-31 08:51:18,348 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) gunicorn-web stdout | 2020-03-31 08:51:18,348 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:18,348 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) gunicorn-verbs stdout | 2020-03-31 08:51:18,348 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:18,350 [526] [DEBUG] [app] Ending request: urn:request:2d02b1f4-979b-4fb3-a5c9-009d8d1c1055 (/health/instance) 2020-03-31 08:51:18,350 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:51:18,350 [526] [DEBUG] [app] Ending request: urn:request:2d02b1f4-979b-4fb3-a5c9-009d8d1c1055 (/health/instance) 2020-03-31 08:51:18,350 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:18,351 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.079 49 0.079) gunicorn-web stdout | 2020-03-31 08:51:18,351 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.079 49 0.079) 2020-03-31 08:51:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:51:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:31.440707+00:00 (in 9.998702 seconds) 2020-03-31 08:51:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:31 UTC)" (scheduled at 2020-03-31 08:51:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:51:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:31.440707+00:00 (in 9.998702 seconds) 2020-03-31 08:51:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:31 UTC)" (scheduled at 2020-03-31 08:51:21.440707+00:00) 2020-03-31 08:51:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:51:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:51:21,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:51:21,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:51:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:51:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:31 UTC)" executed successfully 2020-03-31 08:51:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:51:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:52.113221+00:00 (in 29.999191 seconds) 2020-03-31 08:51:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:52 UTC)" (scheduled at 2020-03-31 08:51:22.113221+00:00) securityworker stdout | 2020-03-31 08:51:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:52.113221+00:00 (in 29.999191 seconds) 2020-03-31 08:51:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:52 UTC)" (scheduled at 2020-03-31 08:51:22.113221+00:00) 2020-03-31 08:51:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:51:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:51:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:51:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:51:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:51:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:51:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:51:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:51:52 UTC)" executed successfully 2020-03-31 08:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:34.323352+00:00 (in 9.998923 seconds) 2020-03-31 08:51:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:34 UTC)" (scheduled at 2020-03-31 08:51:24.323352+00:00) 2020-03-31 08:51:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:34.323352+00:00 (in 9.998923 seconds) 2020-03-31 08:51:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:34 UTC)" (scheduled at 2020-03-31 08:51:24.323352+00:00) 2020-03-31 08:51:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 24, 325151), True, datetime.datetime(2020, 3, 31, 8, 51, 24, 325151), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:51:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 24, 325151), True, datetime.datetime(2020, 3, 31, 8, 51, 24, 325151), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:51:24,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:51:24,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:51:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:34 UTC)" executed successfully 2020-03-31 08:51:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:51:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:33.330112+00:00 (in 8.918096 seconds) 2020-03-31 08:51:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:24 UTC)" (scheduled at 2020-03-31 08:51:24.410927+00:00) 2020-03-31 08:51:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:51:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:33.330112+00:00 (in 8.918096 seconds) 2020-03-31 08:51:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:24 UTC)" (scheduled at 2020-03-31 08:51:24.410927+00:00) 2020-03-31 08:51:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:24 UTC)" executed successfully 2020-03-31 08:51:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:51:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:36.924497+00:00 (in 9.997283 seconds) 2020-03-31 08:51:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:36 UTC)" (scheduled at 2020-03-31 08:51:26.924497+00:00) 2020-03-31 08:51:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:51:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:36.924497+00:00 (in 9.997283 seconds) 2020-03-31 08:51:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:36 UTC)" (scheduled at 2020-03-31 08:51:26.924497+00:00) 2020-03-31 08:51:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 26, 927919), True, datetime.datetime(2020, 3, 31, 8, 51, 26, 927919), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:51:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 26, 927919), True, datetime.datetime(2020, 3, 31, 8, 51, 26, 927919), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:51:26,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:51:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:51:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:36 UTC)" executed successfully 2020-03-31 08:51:28,273 [518] [DEBUG] [app] Starting request: urn:request:c647028e-1cbb-4f86-a201-e53485df531c (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:28,273 [518] [DEBUG] [app] Starting request: urn:request:c647028e-1cbb-4f86-a201-e53485df531c (/health/instance) 2020-03-31 08:51:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:28,284 [542] [DEBUG] [app] Starting request: urn:request:3633843c-ab05-49a4-bba6-b0b2275532a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:28,284 [542] [DEBUG] [app] Starting request: urn:request:3633843c-ab05-49a4-bba6-b0b2275532a0 (/v1/_internal_ping) 2020-03-31 08:51:28,287 [542] [DEBUG] [app] Ending request: urn:request:3633843c-ab05-49a4-bba6-b0b2275532a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:28,287 [542] [DEBUG] [app] Ending request: urn:request:3633843c-ab05-49a4-bba6-b0b2275532a0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:51:28,289 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:28,289 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:51:28,289 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:28,290 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:51:28,289 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:51:28,290 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:28,296 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:28,296 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:51:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:51:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:28,301 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:51:28,301 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:51:28,302 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:28,302 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:28,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:28,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:28,319 [518] [DEBUG] [app] Starting request: urn:request:57721125-f5f7-43a8-8945-adbac5efc7c3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:28,319 [518] [DEBUG] [app] Starting request: urn:request:57721125-f5f7-43a8-8945-adbac5efc7c3 (/_internal_ping) 2020-03-31 08:51:28,320 [518] [DEBUG] [app] Ending request: urn:request:57721125-f5f7-43a8-8945-adbac5efc7c3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:28,320 [518] [DEBUG] [app] Ending request: urn:request:57721125-f5f7-43a8-8945-adbac5efc7c3 (/_internal_ping) 2020-03-31 08:51:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:51:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:51:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:28,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:28,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:28,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:51:28,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:51:28,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:51:28,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:51:28,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:28,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:28,333 [539] [DEBUG] [app] Starting request: urn:request:1b8e0e28-6ec1-4500-8033-b2d5c02781a7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:28,333 [539] [DEBUG] [app] Starting request: urn:request:1b8e0e28-6ec1-4500-8033-b2d5c02781a7 (/c1/_internal_ping) 2020-03-31 08:51:28,334 [539] [DEBUG] [app] Ending request: urn:request:1b8e0e28-6ec1-4500-8033-b2d5c02781a7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:28,334 [539] [DEBUG] [app] Ending request: urn:request:1b8e0e28-6ec1-4500-8033-b2d5c02781a7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:51:28,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:28,335 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:28,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:51:28,335 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:28,336 [518] [DEBUG] [app] Ending request: urn:request:c647028e-1cbb-4f86-a201-e53485df531c (/health/instance) 2020-03-31 08:51:28,337 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:51:28,336 [518] [DEBUG] [app] Ending request: urn:request:c647028e-1cbb-4f86-a201-e53485df531c (/health/instance) 2020-03-31 08:51:28,337 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:28,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 08:51:28,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 08:51:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:51:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:00.337699+00:00 (in 29.998907 seconds) 2020-03-31 08:51:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:00 UTC)" (scheduled at 2020-03-31 08:51:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:51:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:00.337699+00:00 (in 29.998907 seconds) 2020-03-31 08:51:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:00 UTC)" (scheduled at 2020-03-31 08:51:30.337699+00:00) 2020-03-31 08:51:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 51, 30, 339570), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:51:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 51, 30, 339570), False, 50, 1, 0]) 2020-03-31 08:51:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:51:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:51:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:51:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:00 UTC)" executed successfully 2020-03-31 08:51:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:51:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:41.440707+00:00 (in 9.998835 seconds) 2020-03-31 08:51:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:41 UTC)" (scheduled at 2020-03-31 08:51:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:51:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:41.440707+00:00 (in 9.998835 seconds) 2020-03-31 08:51:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:41 UTC)" (scheduled at 2020-03-31 08:51:31.440707+00:00) 2020-03-31 08:51:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:51:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:51:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:51:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:51:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:51:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:41 UTC)" executed successfully 2020-03-31 08:51:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:51:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:03.330112+00:00 (in 29.999002 seconds) 2020-03-31 08:51:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:03 UTC)" (scheduled at 2020-03-31 08:51:33.330112+00:00) 2020-03-31 08:51:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:51:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:03.330112+00:00 (in 29.999002 seconds) 2020-03-31 08:51:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:03 UTC)" (scheduled at 2020-03-31 08:51:33.330112+00:00) 2020-03-31 08:51:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 33, 332125), True, datetime.datetime(2020, 3, 31, 8, 51, 33, 332125), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:51:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 33, 332125), True, datetime.datetime(2020, 3, 31, 8, 51, 33, 332125), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:51:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:51:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:51:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:03 UTC)" executed successfully 2020-03-31 08:51:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:51:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:43.539999+00:00 (in 10.172484 seconds) 2020-03-31 08:51:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:33 UTC)" (scheduled at 2020-03-31 08:51:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 08:51:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:43.539999+00:00 (in 10.172484 seconds) 2020-03-31 08:51:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:33 UTC)" (scheduled at 2020-03-31 08:51:33.366791+00:00) 2020-03-31 08:51:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:51:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 33, 368495), True, datetime.datetime(2020, 3, 31, 8, 51, 33, 368495), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:51:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 33, 368495), True, datetime.datetime(2020, 3, 31, 8, 51, 33, 368495), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:51:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:51:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:51:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:33 UTC)" executed successfully 2020-03-31 08:51:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:51:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:34 UTC)" (scheduled at 2020-03-31 08:51:34.323352+00:00) 2020-03-31 08:51:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:51:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:34 UTC)" (scheduled at 2020-03-31 08:51:34.323352+00:00) 2020-03-31 08:51:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:44.323352+00:00 (in 9.999242 seconds) notificationworker stdout | 2020-03-31 08:51:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:44.323352+00:00 (in 9.999242 seconds) 2020-03-31 08:51:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 34, 324790), True, datetime.datetime(2020, 3, 31, 8, 51, 34, 324790), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:51:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 34, 324790), True, datetime.datetime(2020, 3, 31, 8, 51, 34, 324790), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:51:34,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:51:34,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:51:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:44 UTC)" executed successfully 2020-03-31 08:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:38.010934+00:00 (in 1.085656 seconds) 2020-03-31 08:51:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:46 UTC)" (scheduled at 2020-03-31 08:51:36.924497+00:00) 2020-03-31 08:51:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:38.010934+00:00 (in 1.085656 seconds) 2020-03-31 08:51:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:46 UTC)" (scheduled at 2020-03-31 08:51:36.924497+00:00) 2020-03-31 08:51:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 36, 926164), True, datetime.datetime(2020, 3, 31, 8, 51, 36, 926164), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:51:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 36, 926164), True, datetime.datetime(2020, 3, 31, 8, 51, 36, 926164), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:51:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:51:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:51:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:46 UTC)" executed successfully 2020-03-31 08:51:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:51:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:46.924497+00:00 (in 8.912787 seconds) 2020-03-31 08:51:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:38 UTC)" (scheduled at 2020-03-31 08:51:38.010934+00:00) 2020-03-31 08:51:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:51:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:46.924497+00:00 (in 8.912787 seconds) 2020-03-31 08:51:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:38 UTC)" (scheduled at 2020-03-31 08:51:38.010934+00:00) 2020-03-31 08:51:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:38 UTC)" executed successfully 2020-03-31 08:51:38,273 [518] [DEBUG] [app] Starting request: urn:request:064da999-8103-4587-b739-67dd0c44be34 (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:38,273 [518] [DEBUG] [app] Starting request: urn:request:064da999-8103-4587-b739-67dd0c44be34 (/health/instance) 2020-03-31 08:51:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:38,283 [554] [DEBUG] [app] Starting request: urn:request:149673ed-fa07-44ae-bb46-1fe1f02b44bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:38,283 [554] [DEBUG] [app] Starting request: urn:request:149673ed-fa07-44ae-bb46-1fe1f02b44bd (/v1/_internal_ping) 2020-03-31 08:51:38,288 [554] [DEBUG] [app] Ending request: urn:request:149673ed-fa07-44ae-bb46-1fe1f02b44bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:38,288 [554] [DEBUG] [app] Ending request: urn:request:149673ed-fa07-44ae-bb46-1fe1f02b44bd (/v1/_internal_ping) 2020-03-31 08:51:38,289 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 162 0.010) gunicorn-web stdout | 2020-03-31 08:51:38,289 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:38,289 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:38,290 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 162 0.010) gunicorn-registry stdout | 2020-03-31 08:51:38,289 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:38,290 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:38,296 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:38,296 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:51:38,301 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:51:38,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:38,302 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:38,301 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 08:51:38,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-registry stdout | 2020-03-31 08:51:38,302 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:38,317 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:38,317 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:38,323 [526] [DEBUG] [app] Starting request: urn:request:f72cd24f-1422-4fd7-92cf-1d9e7a70765e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:38,323 [526] [DEBUG] [app] Starting request: urn:request:f72cd24f-1422-4fd7-92cf-1d9e7a70765e (/_internal_ping) 2020-03-31 08:51:38,324 [526] [DEBUG] [app] Ending request: urn:request:f72cd24f-1422-4fd7-92cf-1d9e7a70765e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:38,324 [526] [DEBUG] [app] Ending request: urn:request:f72cd24f-1422-4fd7-92cf-1d9e7a70765e (/_internal_ping) 2020-03-31 08:51:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:38,324 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:51:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:38,324 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:51:38,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:38,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:51:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:51:38,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:38,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:51:38,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:51:38,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:51:38,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:38,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:38,338 [539] [DEBUG] [app] Starting request: urn:request:91197dc1-fe33-4bfd-9190-91b9f0d94d0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:38,338 [539] [DEBUG] [app] Starting request: urn:request:91197dc1-fe33-4bfd-9190-91b9f0d94d0b (/c1/_internal_ping) 2020-03-31 08:51:38,339 [539] [DEBUG] [app] Ending request: urn:request:91197dc1-fe33-4bfd-9190-91b9f0d94d0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:38,339 [539] [DEBUG] [app] Ending request: urn:request:91197dc1-fe33-4bfd-9190-91b9f0d94d0b (/c1/_internal_ping) 2020-03-31 08:51:38,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:51:38,340 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:38,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:51:38,340 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:38,341 [518] [DEBUG] [app] Ending request: urn:request:064da999-8103-4587-b739-67dd0c44be34 (/health/instance) 2020-03-31 08:51:38,341 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:51:38,341 [518] [DEBUG] [app] Ending request: urn:request:064da999-8103-4587-b739-67dd0c44be34 (/health/instance) 2020-03-31 08:51:38,341 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 08:51:38,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:51:38,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:51.440707+00:00 (in 9.999179 seconds) 2020-03-31 08:51:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:51 UTC)" (scheduled at 2020-03-31 08:51:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:51.440707+00:00 (in 9.999179 seconds) 2020-03-31 08:51:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:51 UTC)" (scheduled at 2020-03-31 08:51:41.440707+00:00) 2020-03-31 08:51:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:51:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:51:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:51:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:51:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:51:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:51 UTC)" executed successfully 2020-03-31 08:51:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:51:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:33.366791+00:00 (in 49.825916 seconds) 2020-03-31 08:51:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:43 UTC)" (scheduled at 2020-03-31 08:51:43.539999+00:00) 2020-03-31 08:51:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:51:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:33.366791+00:00 (in 49.825916 seconds) 2020-03-31 08:51:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:43 UTC)" (scheduled at 2020-03-31 08:51:43.539999+00:00) 2020-03-31 08:51:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:43 UTC)" executed successfully 2020-03-31 08:51:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:51:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:54 UTC)" (scheduled at 2020-03-31 08:51:44.323352+00:00) notificationworker stdout | 2020-03-31 08:51:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:54 UTC)" (scheduled at 2020-03-31 08:51:44.323352+00:00) 2020-03-31 08:51:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:49.498966+00:00 (in 5.172816 seconds) 2020-03-31 08:51:44,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:51:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:49.498966+00:00 (in 5.172816 seconds) 2020-03-31 08:51:44,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:44,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 44, 327357), True, datetime.datetime(2020, 3, 31, 8, 51, 44, 327357), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:51:44,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 44, 327357), True, datetime.datetime(2020, 3, 31, 8, 51, 44, 327357), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:51:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:51:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:51:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:54 UTC)" executed successfully 2020-03-31 08:51:44,446 [527] [DEBUG] [app] Starting request: urn:request:eaa91b3b-2c8c-4e28-86da-fa02ae82e7dc (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:44,446 [527] [DEBUG] [app] Starting request: urn:request:eaa91b3b-2c8c-4e28-86da-fa02ae82e7dc (/health/instance) 2020-03-31 08:51:44,452 [527] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 08:51:44,452 [527] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 08:51:44,462 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:44,462 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:44,463 [538] [DEBUG] [app] Starting request: urn:request:7a7ee3d7-d48d-4c55-8318-f732393860d7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:44,463 [538] [DEBUG] [app] Starting request: urn:request:7a7ee3d7-d48d-4c55-8318-f732393860d7 (/v1/_internal_ping) 2020-03-31 08:51:44,465 [538] [DEBUG] [app] Ending request: urn:request:7a7ee3d7-d48d-4c55-8318-f732393860d7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:44,465 [538] [DEBUG] [app] Ending request: urn:request:7a7ee3d7-d48d-4c55-8318-f732393860d7 (/v1/_internal_ping) 2020-03-31 08:51:44,465 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:44,466 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:51:44,465 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:44,466 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:51:44,468 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:44,468 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:51:44,469 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:44,470 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 08:51:44,469 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:44,471 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:44,470 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:51:44,471 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:44,482 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:51:44,482 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:44,494 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:44,494 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:44,499 [527] [DEBUG] [app] Starting request: urn:request:27ddbd36-44d1-42b1-bf83-ebd025e4b8d5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:44,499 [527] [DEBUG] [app] Starting request: urn:request:27ddbd36-44d1-42b1-bf83-ebd025e4b8d5 (/_internal_ping) 2020-03-31 08:51:44,499 [527] [DEBUG] [app] Ending request: urn:request:27ddbd36-44d1-42b1-bf83-ebd025e4b8d5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:44,499 [527] [DEBUG] [app] Ending request: urn:request:27ddbd36-44d1-42b1-bf83-ebd025e4b8d5 (/_internal_ping) 2020-03-31 08:51:44,500 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:51:44,500 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:51:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:44,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:44,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:44,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:51:44,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:44,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:44,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:51:44,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:51:44,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:51:44,514 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 51, 44, 512285), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 51, 44, 512355), u'quay', None, datetime.datetime(2020, 3, 30, 8, 51, 44, 512394), datetime.datetime(2020, 3, 24, 8, 51, 44, 512425), None]) gunicorn-web stdout | 2020-03-31 08:51:44,514 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 51, 44, 512285), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 51, 44, 512355), u'quay', None, datetime.datetime(2020, 3, 30, 8, 51, 44, 512394), datetime.datetime(2020, 3, 24, 8, 51, 44, 512425), None]) 2020-03-31 08:51:44,526 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:44,526 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:44,528 [539] [DEBUG] [app] Starting request: urn:request:56738c7a-6e5e-424f-8e5c-a0d290120e6f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:44,528 [539] [DEBUG] [app] Starting request: urn:request:56738c7a-6e5e-424f-8e5c-a0d290120e6f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:51:44,528 [539] [DEBUG] [app] Ending request: urn:request:56738c7a-6e5e-424f-8e5c-a0d290120e6f (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:51:44,529 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:44,529 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:51:44,528 [539] [DEBUG] [app] Ending request: urn:request:56738c7a-6e5e-424f-8e5c-a0d290120e6f (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:44,529 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:51:44,529 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:44,531 [527] [DEBUG] [app] Ending request: urn:request:eaa91b3b-2c8c-4e28-86da-fa02ae82e7dc (/health/instance) 2020-03-31 08:51:44,531 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:51:44,531 [527] [DEBUG] [app] Ending request: urn:request:eaa91b3b-2c8c-4e28-86da-fa02ae82e7dc (/health/instance) 2020-03-31 08:51:44,531 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:44,531 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.087 49 0.087) gunicorn-web stdout | 2020-03-31 08:51:44,531 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.087 49 0.087) 2020-03-31 08:51:45,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:51:45,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:45,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:15 UTC)" (scheduled at 2020-03-31 08:51:45.064514+00:00) gcworker stdout | 2020-03-31 08:51:45,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:15 UTC)" (scheduled at 2020-03-31 08:51:45.064514+00:00) 2020-03-31 08:51:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435105, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:51:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435105, 1209600, 500, 1, 0]) 2020-03-31 08:51:45,070 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:15.064514+00:00 (in 29.997775 seconds) gcworker stdout | 2020-03-31 08:51:45,070 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:15.064514+00:00 (in 29.997775 seconds) 2020-03-31 08:51:45,090 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:51:45,090 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:51:45,091 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:45,091 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:51:45,091 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:45,091 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:15 UTC)" executed successfully 2020-03-31 08:51:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:51:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:56.924497+00:00 (in 9.999270 seconds) 2020-03-31 08:51:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:56 UTC)" (scheduled at 2020-03-31 08:51:46.924497+00:00) 2020-03-31 08:51:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:51:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:56.924497+00:00 (in 9.999270 seconds) 2020-03-31 08:51:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:56 UTC)" (scheduled at 2020-03-31 08:51:46.924497+00:00) 2020-03-31 08:51:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 46, 925988), True, datetime.datetime(2020, 3, 31, 8, 51, 46, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:51:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 46, 925988), True, datetime.datetime(2020, 3, 31, 8, 51, 46, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:51:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:51:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:51:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:56 UTC)" executed successfully 2020-03-31 08:51:48,273 [518] [DEBUG] [app] Starting request: urn:request:c65d3370-1474-4729-8fe5-7a1041917f3c (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:48,273 [518] [DEBUG] [app] Starting request: urn:request:c65d3370-1474-4729-8fe5-7a1041917f3c (/health/instance) 2020-03-31 08:51:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:48,281 [550] [DEBUG] [app] Starting request: urn:request:83c91b5c-8014-49ab-aca5-51a524e95271 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:48,281 [550] [DEBUG] [app] Starting request: urn:request:83c91b5c-8014-49ab-aca5-51a524e95271 (/v1/_internal_ping) 2020-03-31 08:51:48,282 [550] [DEBUG] [app] Ending request: urn:request:83c91b5c-8014-49ab-aca5-51a524e95271 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:48,282 [550] [DEBUG] [app] Ending request: urn:request:83c91b5c-8014-49ab-aca5-51a524e95271 (/v1/_internal_ping) 2020-03-31 08:51:48,283 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:51:48,283 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:51:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:51:48,286 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:51:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:48,288 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:51:48,286 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:48,288 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:51:48,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:48,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:51:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:48,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:48,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:48,313 [519] [DEBUG] [app] Starting request: urn:request:a9314d2c-ceeb-45a3-87dc-35b02422baea (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:48,313 [519] [DEBUG] [app] Starting request: urn:request:a9314d2c-ceeb-45a3-87dc-35b02422baea (/_internal_ping) 2020-03-31 08:51:48,314 [519] [DEBUG] [app] Ending request: urn:request:a9314d2c-ceeb-45a3-87dc-35b02422baea (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:48,314 [519] [DEBUG] [app] Ending request: urn:request:a9314d2c-ceeb-45a3-87dc-35b02422baea (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:51:48,315 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:51:48,315 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:51:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:51:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:51:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:51:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:48,326 [539] [DEBUG] [app] Starting request: urn:request:464c4abf-7b4e-4132-99c8-443ce8924519 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:48,326 [539] [DEBUG] [app] Starting request: urn:request:464c4abf-7b4e-4132-99c8-443ce8924519 (/c1/_internal_ping) 2020-03-31 08:51:48,327 [539] [DEBUG] [app] Ending request: urn:request:464c4abf-7b4e-4132-99c8-443ce8924519 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:48,327 [539] [DEBUG] [app] Ending request: urn:request:464c4abf-7b4e-4132-99c8-443ce8924519 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:51:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:51:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:48,329 [518] [DEBUG] [app] Ending request: urn:request:c65d3370-1474-4729-8fe5-7a1041917f3c (/health/instance) 2020-03-31 08:51:48,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:51:48,329 [518] [DEBUG] [app] Ending request: urn:request:c65d3370-1474-4729-8fe5-7a1041917f3c (/health/instance) 2020-03-31 08:51:48,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:48,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 08:51:48,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 08:51:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:51:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:54.323352+00:00 (in 4.822832 seconds) 2020-03-31 08:51:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:49 UTC)" (scheduled at 2020-03-31 08:51:49.498966+00:00) 2020-03-31 08:51:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 08:51:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:54.323352+00:00 (in 4.822832 seconds) 2020-03-31 08:51:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:49 UTC)" (scheduled at 2020-03-31 08:51:49.498966+00:00) 2020-03-31 08:51:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:51:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:49 UTC)" executed successfully 2020-03-31 08:51:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:51:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:59.560326+00:00 (in 8.448485 seconds) 2020-03-31 08:51:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:51 UTC)" (scheduled at 2020-03-31 08:51:51.110610+00:00) 2020-03-31 08:51:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:51:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:51:59.560326+00:00 (in 8.448485 seconds) 2020-03-31 08:51:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:51 UTC)" (scheduled at 2020-03-31 08:51:51.110610+00:00) 2020-03-31 08:51:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 51, 112633), True, datetime.datetime(2020, 3, 31, 8, 51, 51, 112633), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:51:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 51, 112633), True, datetime.datetime(2020, 3, 31, 8, 51, 51, 112633), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:51:51,134 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:51:51,134 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:51,134 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:51:51,134 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:51 UTC)" executed successfully 2020-03-31 08:51:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:51:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:51 UTC)" (scheduled at 2020-03-31 08:51:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:51:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:51:51 UTC)" (scheduled at 2020-03-31 08:51:51.440707+00:00) 2020-03-31 08:51:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:51:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:51:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:01.440707+00:00 (in 9.998826 seconds) repositoryactioncounter stdout | 2020-03-31 08:51:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:01.440707+00:00 (in 9.998826 seconds) 2020-03-31 08:51:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:51:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:51:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:51:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:01 UTC)" executed successfully 2020-03-31 08:51:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:51:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:22.113221+00:00 (in 29.998951 seconds) 2020-03-31 08:51:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:22 UTC)" (scheduled at 2020-03-31 08:51:52.113221+00:00) securityworker stdout | 2020-03-31 08:51:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:22.113221+00:00 (in 29.998951 seconds) 2020-03-31 08:51:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:22 UTC)" (scheduled at 2020-03-31 08:51:52.113221+00:00) 2020-03-31 08:51:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:51:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:51:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:51:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:51:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:51:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:51:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:51:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:22 UTC)" executed successfully 2020-03-31 08:51:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:51:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:04.323352+00:00 (in 9.998870 seconds) 2020-03-31 08:51:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:04 UTC)" (scheduled at 2020-03-31 08:51:54.323352+00:00) 2020-03-31 08:51:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:51:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:04.323352+00:00 (in 9.998870 seconds) 2020-03-31 08:51:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:04 UTC)" (scheduled at 2020-03-31 08:51:54.323352+00:00) 2020-03-31 08:51:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 54, 325299), True, datetime.datetime(2020, 3, 31, 8, 51, 54, 325299), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:51:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 54, 325299), True, datetime.datetime(2020, 3, 31, 8, 51, 54, 325299), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:51:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:51:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:51:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:04 UTC)" executed successfully 2020-03-31 08:51:54,447 [527] [DEBUG] [app] Starting request: urn:request:ef2bb236-76fc-4916-b92f-593c5d277b10 (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:54,447 [527] [DEBUG] [app] Starting request: urn:request:ef2bb236-76fc-4916-b92f-593c5d277b10 (/health/instance) 2020-03-31 08:51:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:54,454 [553] [DEBUG] [app] Starting request: urn:request:e7e73ae1-bb2e-4ab3-a15e-2f4f86cedbe3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:54,454 [553] [DEBUG] [app] Starting request: urn:request:e7e73ae1-bb2e-4ab3-a15e-2f4f86cedbe3 (/v1/_internal_ping) 2020-03-31 08:51:54,456 [553] [DEBUG] [app] Ending request: urn:request:e7e73ae1-bb2e-4ab3-a15e-2f4f86cedbe3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:54,456 [553] [DEBUG] [app] Ending request: urn:request:e7e73ae1-bb2e-4ab3-a15e-2f4f86cedbe3 (/v1/_internal_ping) 2020-03-31 08:51:54,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:51:54,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:54,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:51:54,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:54,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:54,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:51:54,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:51:54,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:51:54,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:54,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:51:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:54,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:54,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:54,481 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:54,481 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:54,482 [527] [DEBUG] [app] Starting request: urn:request:44d3c839-3e04-41a0-8889-034f54cb7935 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:54,482 [527] [DEBUG] [app] Starting request: urn:request:44d3c839-3e04-41a0-8889-034f54cb7935 (/_internal_ping) 2020-03-31 08:51:54,483 [527] [DEBUG] [app] Ending request: urn:request:44d3c839-3e04-41a0-8889-034f54cb7935 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:54,483 [527] [DEBUG] [app] Ending request: urn:request:44d3c839-3e04-41a0-8889-034f54cb7935 (/_internal_ping) 2020-03-31 08:51:54,483 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:51:54,483 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:54,484 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:51:54,484 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:54,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:54,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:54,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:51:54,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:51:54,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:54,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:51:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:51:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:51:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:54,499 [535] [DEBUG] [app] Starting request: urn:request:c17015f5-d801-48de-90ee-7888a9897df6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:54,499 [535] [DEBUG] [app] Starting request: urn:request:c17015f5-d801-48de-90ee-7888a9897df6 (/c1/_internal_ping) 2020-03-31 08:51:54,503 [535] [DEBUG] [app] Ending request: urn:request:c17015f5-d801-48de-90ee-7888a9897df6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:54,503 [535] [DEBUG] [app] Ending request: urn:request:c17015f5-d801-48de-90ee-7888a9897df6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:51:54,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:54,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:54,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:51:54,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:54,506 [527] [DEBUG] [app] Ending request: urn:request:ef2bb236-76fc-4916-b92f-593c5d277b10 (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:54,506 [527] [DEBUG] [app] Ending request: urn:request:ef2bb236-76fc-4916-b92f-593c5d277b10 (/health/instance) 2020-03-31 08:51:54,506 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:51:54,506 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:54,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:51:54,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:51:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:51:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:03.010318+00:00 (in 6.084963 seconds) 2020-03-31 08:51:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:06 UTC)" (scheduled at 2020-03-31 08:51:56.924497+00:00) 2020-03-31 08:51:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:51:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:03.010318+00:00 (in 6.084963 seconds) 2020-03-31 08:51:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:06 UTC)" (scheduled at 2020-03-31 08:51:56.924497+00:00) 2020-03-31 08:51:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:51:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 56, 926423), True, datetime.datetime(2020, 3, 31, 8, 51, 56, 926423), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:51:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 51, 56, 926423), True, datetime.datetime(2020, 3, 31, 8, 51, 56, 926423), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:51:56,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:51:56,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:51:56,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:56,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:51:56,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:56,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:06 UTC)" executed successfully 2020-03-31 08:51:58,273 [518] [DEBUG] [app] Starting request: urn:request:bfd42eb6-ced8-44bb-a67d-025bd8778513 (/health/instance) gunicorn-web stdout | 2020-03-31 08:51:58,273 [518] [DEBUG] [app] Starting request: urn:request:bfd42eb6-ced8-44bb-a67d-025bd8778513 (/health/instance) 2020-03-31 08:51:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:58,283 [551] [DEBUG] [app] Starting request: urn:request:9cdf8d7b-24ec-4231-bf9c-27b11b507360 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:58,283 [551] [DEBUG] [app] Starting request: urn:request:9cdf8d7b-24ec-4231-bf9c-27b11b507360 (/v1/_internal_ping) 2020-03-31 08:51:58,287 [551] [DEBUG] [app] Ending request: urn:request:9cdf8d7b-24ec-4231-bf9c-27b11b507360 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:51:58,287 [551] [DEBUG] [app] Ending request: urn:request:9cdf8d7b-24ec-4231-bf9c-27b11b507360 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:51:58,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:51:58,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:58,288 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 08:51:58,288 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:58,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:51:58,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:51:58,295 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:51:58,295 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:51:58,299 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:51:58,299 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:51:58,300 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:51:58,300 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:51:58,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:51:58,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:51:58,315 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:51:58,315 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:51:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:58,322 [518] [DEBUG] [app] Starting request: urn:request:9485d75f-8878-4924-85ee-a41bcdbd8cba (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:58,322 [518] [DEBUG] [app] Starting request: urn:request:9485d75f-8878-4924-85ee-a41bcdbd8cba (/_internal_ping) 2020-03-31 08:51:58,323 [518] [DEBUG] [app] Ending request: urn:request:9485d75f-8878-4924-85ee-a41bcdbd8cba (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:51:58,323 [518] [DEBUG] [app] Ending request: urn:request:9485d75f-8878-4924-85ee-a41bcdbd8cba (/_internal_ping) 2020-03-31 08:51:58,323 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:51:58,323 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:51:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:51:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:51:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:51:58,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:58,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:51:58,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:51:58,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:51:58,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:51:58,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:51:58,338 [539] [DEBUG] [app] Starting request: urn:request:64d31e15-b8d4-4a93-8550-e3383d0f3cb1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:58,338 [539] [DEBUG] [app] Starting request: urn:request:64d31e15-b8d4-4a93-8550-e3383d0f3cb1 (/c1/_internal_ping) 2020-03-31 08:51:58,339 [539] [DEBUG] [app] Ending request: urn:request:64d31e15-b8d4-4a93-8550-e3383d0f3cb1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:51:58,339 [539] [DEBUG] [app] Ending request: urn:request:64d31e15-b8d4-4a93-8550-e3383d0f3cb1 (/c1/_internal_ping) 2020-03-31 08:51:58,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:51:58,340 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:51:58,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-verbs stdout | 2020-03-31 08:51:58,340 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:51:58,341 [518] [DEBUG] [app] Ending request: urn:request:bfd42eb6-ced8-44bb-a67d-025bd8778513 (/health/instance) 2020-03-31 08:51:58,342 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:51:58,341 [518] [DEBUG] [app] Ending request: urn:request:bfd42eb6-ced8-44bb-a67d-025bd8778513 (/health/instance) 2020-03-31 08:51:58,342 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:51:58,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 08:51:58,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:51:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:51:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 08:51:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:51:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:51:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:51.110610+00:00 (in 51.549469 seconds) 2020-03-31 08:51:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:59 UTC)" (scheduled at 2020-03-31 08:51:59.560326+00:00) 2020-03-31 08:51:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:51:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:51.110610+00:00 (in 51.549469 seconds) 2020-03-31 08:51:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:59 UTC)" (scheduled at 2020-03-31 08:51:59.560326+00:00) 2020-03-31 08:51:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:51:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:52:59 UTC)" executed successfully 2020-03-31 08:52:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:52:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:30.337699+00:00 (in 29.999009 seconds) 2020-03-31 08:52:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:30 UTC)" (scheduled at 2020-03-31 08:52:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:52:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:30.337699+00:00 (in 29.999009 seconds) 2020-03-31 08:52:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:30 UTC)" (scheduled at 2020-03-31 08:52:00.337699+00:00) 2020-03-31 08:52:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 52, 0, 339345), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:52:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 52, 0, 339345), False, 50, 1, 0]) 2020-03-31 08:52:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:52:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:52:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:52:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:30 UTC)" executed successfully 2020-03-31 08:52:01,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:52:01,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:11.440707+00:00 (in 9.995178 seconds) 2020-03-31 08:52:01,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:11 UTC)" (scheduled at 2020-03-31 08:52:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:52:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:11.440707+00:00 (in 9.995178 seconds) 2020-03-31 08:52:01,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:11 UTC)" (scheduled at 2020-03-31 08:52:01.440707+00:00) 2020-03-31 08:52:01,449 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:52:01,449 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:52:01,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:52:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:52:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:52:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:11 UTC)" executed successfully 2020-03-31 08:52:03,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:52:03,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:06.924497+00:00 (in 3.913111 seconds) 2020-03-31 08:52:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:03 UTC)" (scheduled at 2020-03-31 08:52:03.010318+00:00) storagereplication stdout | 2020-03-31 08:52:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:06.924497+00:00 (in 3.913111 seconds) 2020-03-31 08:52:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:03 UTC)" (scheduled at 2020-03-31 08:52:03.010318+00:00) 2020-03-31 08:52:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 08:52:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), u'imagestoragereplication/%']) 2020-03-31 08:52:03,029 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), True, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 08:52:03,029 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), True, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), 0, u'imagestoragereplication/%']) 2020-03-31 08:52:03,033 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), True, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 08:52:03,033 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), True, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 8, 52, 3, 12104), u'imagestoragereplication/%']) 2020-03-31 08:52:03,037 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 08:52:03,037 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:03,039 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:03,039 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:03 UTC)" executed successfully 2020-03-31 08:52:03,042 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 08:52:03,042 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:03,043 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 08:52:03,043 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:03,044 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 08:52:03,044 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 08:52:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:52:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:24.410927+00:00 (in 21.079859 seconds) 2020-03-31 08:52:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:33 UTC)" (scheduled at 2020-03-31 08:52:03.330112+00:00) 2020-03-31 08:52:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:52:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:24.410927+00:00 (in 21.079859 seconds) 2020-03-31 08:52:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:33 UTC)" (scheduled at 2020-03-31 08:52:03.330112+00:00) 2020-03-31 08:52:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 3, 331909), True, datetime.datetime(2020, 3, 31, 8, 52, 3, 331909), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:52:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 3, 331909), True, datetime.datetime(2020, 3, 31, 8, 52, 3, 331909), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:52:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:52:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:52:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:33 UTC)" executed successfully 2020-03-31 08:52:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:14.323352+00:00 (in 9.999196 seconds) 2020-03-31 08:52:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:14 UTC)" (scheduled at 2020-03-31 08:52:04.323352+00:00) 2020-03-31 08:52:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:52:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:14.323352+00:00 (in 9.999196 seconds) 2020-03-31 08:52:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:14 UTC)" (scheduled at 2020-03-31 08:52:04.323352+00:00) 2020-03-31 08:52:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 4, 325275), True, datetime.datetime(2020, 3, 31, 8, 52, 4, 325275), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:52:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 4, 325275), True, datetime.datetime(2020, 3, 31, 8, 52, 4, 325275), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:52:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:04,344 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:52:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:14 UTC)" executed successfully 2020-03-31 08:52:04,446 [518] [DEBUG] [app] Starting request: urn:request:b16f4499-2a2d-4e6a-a08a-4e2cff018c1f (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:04,446 [518] [DEBUG] [app] Starting request: urn:request:b16f4499-2a2d-4e6a-a08a-4e2cff018c1f (/health/instance) 2020-03-31 08:52:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:04,457 [556] [DEBUG] [app] Starting request: urn:request:16b6332d-b5a7-41dc-9e21-2c73389937e3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:04,457 [556] [DEBUG] [app] Starting request: urn:request:16b6332d-b5a7-41dc-9e21-2c73389937e3 (/v1/_internal_ping) 2020-03-31 08:52:04,461 [556] [DEBUG] [app] Ending request: urn:request:16b6332d-b5a7-41dc-9e21-2c73389937e3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:04,461 [556] [DEBUG] [app] Ending request: urn:request:16b6332d-b5a7-41dc-9e21-2c73389937e3 (/v1/_internal_ping) 2020-03-31 08:52:04,463 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-web stdout | 2020-03-31 08:52:04,463 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:04,463 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:04,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-registry stdout | 2020-03-31 08:52:04,463 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:04,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:04,470 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:04,470 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:04,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:04,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:04,476 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:04,476 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:04,477 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:04,477 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:04,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:04,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:04,492 [518] [DEBUG] [app] Starting request: urn:request:9a9212d2-63ac-4649-9991-2ff67e31637c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:04,492 [518] [DEBUG] [app] Starting request: urn:request:9a9212d2-63ac-4649-9991-2ff67e31637c (/_internal_ping) 2020-03-31 08:52:04,493 [518] [DEBUG] [app] Ending request: urn:request:9a9212d2-63ac-4649-9991-2ff67e31637c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:04,493 [518] [DEBUG] [app] Ending request: urn:request:9a9212d2-63ac-4649-9991-2ff67e31637c (/_internal_ping) 2020-03-31 08:52:04,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:52:04,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:52:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:04,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:04,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:52:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:52:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:04,505 [539] [DEBUG] [app] Starting request: urn:request:61ba41f6-ad4e-4d7c-9ba0-83c0f4a5c959 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:04,505 [539] [DEBUG] [app] Starting request: urn:request:61ba41f6-ad4e-4d7c-9ba0-83c0f4a5c959 (/c1/_internal_ping) 2020-03-31 08:52:04,506 [539] [DEBUG] [app] Ending request: urn:request:61ba41f6-ad4e-4d7c-9ba0-83c0f4a5c959 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:04,506 [539] [DEBUG] [app] Ending request: urn:request:61ba41f6-ad4e-4d7c-9ba0-83c0f4a5c959 (/c1/_internal_ping) 2020-03-31 08:52:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:04,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:04,508 [518] [DEBUG] [app] Ending request: urn:request:b16f4499-2a2d-4e6a-a08a-4e2cff018c1f (/health/instance) 2020-03-31 08:52:04,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:52:04,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:52:04,508 [518] [DEBUG] [app] Ending request: urn:request:b16f4499-2a2d-4e6a-a08a-4e2cff018c1f (/health/instance) 2020-03-31 08:52:04,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:52:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:52:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:16 UTC)" (scheduled at 2020-03-31 08:52:06.924497+00:00) 2020-03-31 08:52:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:52:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:16 UTC)" (scheduled at 2020-03-31 08:52:06.924497+00:00) 2020-03-31 08:52:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:16.924497+00:00 (in 9.999103 seconds) storagereplication stdout | 2020-03-31 08:52:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:16.924497+00:00 (in 9.999103 seconds) 2020-03-31 08:52:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 6, 926184), True, datetime.datetime(2020, 3, 31, 8, 52, 6, 926184), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:52:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 6, 926184), True, datetime.datetime(2020, 3, 31, 8, 52, 6, 926184), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:52:06,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:52:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:16 UTC)" executed successfully 2020-03-31 08:52:08,274 [518] [DEBUG] [app] Starting request: urn:request:05ac232f-611e-4615-ba94-839510a62f72 (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:08,274 [518] [DEBUG] [app] Starting request: urn:request:05ac232f-611e-4615-ba94-839510a62f72 (/health/instance) 2020-03-31 08:52:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:08,283 [552] [DEBUG] [app] Starting request: urn:request:be7c4646-5df7-46ab-8598-d79cff087002 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:08,283 [552] [DEBUG] [app] Starting request: urn:request:be7c4646-5df7-46ab-8598-d79cff087002 (/v1/_internal_ping) 2020-03-31 08:52:08,287 [552] [DEBUG] [app] Ending request: urn:request:be7c4646-5df7-46ab-8598-d79cff087002 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:08,287 [552] [DEBUG] [app] Ending request: urn:request:be7c4646-5df7-46ab-8598-d79cff087002 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:52:08,288 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:08,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:08,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:52:08,288 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:08,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:52:08,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:08,294 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:08,294 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:08,299 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:08,299 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:08,300 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:08,300 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:08,303 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:08,303 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:08,317 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:08,317 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:08,322 [518] [DEBUG] [app] Starting request: urn:request:d82aa641-151b-4ad8-9fc3-157f43d869c6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:08,322 [518] [DEBUG] [app] Starting request: urn:request:d82aa641-151b-4ad8-9fc3-157f43d869c6 (/_internal_ping) 2020-03-31 08:52:08,323 [518] [DEBUG] [app] Ending request: urn:request:d82aa641-151b-4ad8-9fc3-157f43d869c6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:08,323 [518] [DEBUG] [app] Ending request: urn:request:d82aa641-151b-4ad8-9fc3-157f43d869c6 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:52:08,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:52:08,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:08,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:08,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:52:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:08,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:08,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:08,336 [539] [DEBUG] [app] Starting request: urn:request:9de5fb9d-dc07-4597-96dd-d1c2513eaafe (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:08,336 [539] [DEBUG] [app] Starting request: urn:request:9de5fb9d-dc07-4597-96dd-d1c2513eaafe (/c1/_internal_ping) 2020-03-31 08:52:08,337 [539] [DEBUG] [app] Ending request: urn:request:9de5fb9d-dc07-4597-96dd-d1c2513eaafe (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:08,337 [539] [DEBUG] [app] Ending request: urn:request:9de5fb9d-dc07-4597-96dd-d1c2513eaafe (/c1/_internal_ping) 2020-03-31 08:52:08,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:08,338 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:08,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:52:08,338 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:08,339 [518] [DEBUG] [app] Ending request: urn:request:05ac232f-611e-4615-ba94-839510a62f72 (/health/instance) 2020-03-31 08:52:08,339 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:08,339 [518] [DEBUG] [app] Ending request: urn:request:05ac232f-611e-4615-ba94-839510a62f72 (/health/instance) 2020-03-31 08:52:08,339 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:08,340 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 08:52:08,340 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 08:52:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:52:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:21.440707+00:00 (in 9.999022 seconds) 2020-03-31 08:52:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:21 UTC)" (scheduled at 2020-03-31 08:52:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:52:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:21.440707+00:00 (in 9.999022 seconds) 2020-03-31 08:52:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:21 UTC)" (scheduled at 2020-03-31 08:52:11.440707+00:00) 2020-03-31 08:52:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:52:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:52:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:52:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:52:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:52:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:21 UTC)" executed successfully 2020-03-31 08:52:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:24.323352+00:00 (in 9.998627 seconds) 2020-03-31 08:52:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:24 UTC)" (scheduled at 2020-03-31 08:52:14.323352+00:00) notificationworker stdout | 2020-03-31 08:52:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:24.323352+00:00 (in 9.998627 seconds) 2020-03-31 08:52:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:24 UTC)" (scheduled at 2020-03-31 08:52:14.323352+00:00) 2020-03-31 08:52:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:52:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 14, 325761), True, datetime.datetime(2020, 3, 31, 8, 52, 14, 325761), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:52:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 14, 325761), True, datetime.datetime(2020, 3, 31, 8, 52, 14, 325761), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:52:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:52:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:24 UTC)" executed successfully 2020-03-31 08:52:14,446 [526] [DEBUG] [app] Starting request: urn:request:0db6631e-6a3e-4fd4-93cb-d12a949a946c (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:14,446 [526] [DEBUG] [app] Starting request: urn:request:0db6631e-6a3e-4fd4-93cb-d12a949a946c (/health/instance) 2020-03-31 08:52:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:14,453 [556] [DEBUG] [app] Starting request: urn:request:8660581e-0c5a-4b8c-8bb2-8bc1e4f27f00 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:14,453 [556] [DEBUG] [app] Starting request: urn:request:8660581e-0c5a-4b8c-8bb2-8bc1e4f27f00 (/v1/_internal_ping) 2020-03-31 08:52:14,454 [556] [DEBUG] [app] Ending request: urn:request:8660581e-0c5a-4b8c-8bb2-8bc1e4f27f00 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:14,454 [556] [DEBUG] [app] Ending request: urn:request:8660581e-0c5a-4b8c-8bb2-8bc1e4f27f00 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:14,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:52:14,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:52:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:52:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:14,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:14,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:14,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:14,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:14,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:14,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:14,473 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:14,473 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:14,493 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:14,493 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:14,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:14,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:14,498 [526] [DEBUG] [app] Starting request: urn:request:d1170913-6e2c-42e3-b10c-244b41f4b40e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:14,498 [526] [DEBUG] [app] Starting request: urn:request:d1170913-6e2c-42e3-b10c-244b41f4b40e (/_internal_ping) 2020-03-31 08:52:14,499 [526] [DEBUG] [app] Ending request: urn:request:d1170913-6e2c-42e3-b10c-244b41f4b40e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:14,499 [526] [DEBUG] [app] Ending request: urn:request:d1170913-6e2c-42e3-b10c-244b41f4b40e (/_internal_ping) 2020-03-31 08:52:14,499 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:52:14,499 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:14,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:52:14,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:14,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:14,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:14,506 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:14,506 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:14,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:52:14,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:52:14,510 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:14,510 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:14,511 [537] [DEBUG] [app] Starting request: urn:request:e3d12473-2a05-4847-8135-b2f282c65db8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:14,511 [537] [DEBUG] [app] Starting request: urn:request:e3d12473-2a05-4847-8135-b2f282c65db8 (/c1/_internal_ping) 2020-03-31 08:52:14,512 [537] [DEBUG] [app] Ending request: urn:request:e3d12473-2a05-4847-8135-b2f282c65db8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:14,512 [537] [DEBUG] [app] Ending request: urn:request:e3d12473-2a05-4847-8135-b2f282c65db8 (/c1/_internal_ping) 2020-03-31 08:52:14,512 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:14,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:14,512 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:14,514 [526] [DEBUG] [app] Ending request: urn:request:0db6631e-6a3e-4fd4-93cb-d12a949a946c (/health/instance) 2020-03-31 08:52:14,515 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:52:14,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:14,515 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 08:52:14,514 [526] [DEBUG] [app] Ending request: urn:request:0db6631e-6a3e-4fd4-93cb-d12a949a946c (/health/instance) 2020-03-31 08:52:14,515 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:14,515 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 08:52:15,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:52:15,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:45.064514+00:00 (in 29.995889 seconds) 2020-03-31 08:52:15,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:45 UTC)" (scheduled at 2020-03-31 08:52:15.064514+00:00) gcworker stdout | 2020-03-31 08:52:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:45.064514+00:00 (in 29.995889 seconds) 2020-03-31 08:52:15,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:45 UTC)" (scheduled at 2020-03-31 08:52:15.064514+00:00) 2020-03-31 08:52:15,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435135, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:52:15,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435135, 1209600, 500, 1, 0]) 2020-03-31 08:52:15,093 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:52:15,093 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:52:15,093 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:15,093 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:52:15,093 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:15,093 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:45 UTC)" executed successfully 2020-03-31 08:52:16,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:52:16,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:16,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:26 UTC)" (scheduled at 2020-03-31 08:52:16.924497+00:00) 2020-03-31 08:52:16,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:52:16,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:26 UTC)" (scheduled at 2020-03-31 08:52:16.924497+00:00) 2020-03-31 08:52:16,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:16,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:26.924497+00:00 (in 9.996659 seconds) storagereplication stdout | 2020-03-31 08:52:16,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:26.924497+00:00 (in 9.996659 seconds) 2020-03-31 08:52:16,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 16, 928613), True, datetime.datetime(2020, 3, 31, 8, 52, 16, 928613), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:52:16,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 16, 928613), True, datetime.datetime(2020, 3, 31, 8, 52, 16, 928613), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:52:16,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:52:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:26 UTC)" executed successfully 2020-03-31 08:52:18,278 [525] [DEBUG] [app] Starting request: urn:request:ab48ae8e-fab9-49db-ab2d-b9a513dacc3b (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:18,278 [525] [DEBUG] [app] Starting request: urn:request:ab48ae8e-fab9-49db-ab2d-b9a513dacc3b (/health/instance) 2020-03-31 08:52:18,285 [525] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 08:52:18,285 [525] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 08:52:18,295 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:18,295 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:18,297 [538] [DEBUG] [app] Starting request: urn:request:f2ae7aa0-7cc0-4201-a07e-1a7d6ce0e33f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:18,297 [538] [DEBUG] [app] Starting request: urn:request:f2ae7aa0-7cc0-4201-a07e-1a7d6ce0e33f (/v1/_internal_ping) 2020-03-31 08:52:18,299 [538] [DEBUG] [app] Ending request: urn:request:f2ae7aa0-7cc0-4201-a07e-1a7d6ce0e33f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:18,299 [538] [DEBUG] [app] Ending request: urn:request:f2ae7aa0-7cc0-4201-a07e-1a7d6ce0e33f (/v1/_internal_ping) 2020-03-31 08:52:18,299 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:52:18,299 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:18,299 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:52:18,299 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:18,302 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:52:18,302 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:18,302 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:18,302 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:18,304 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:18,304 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:18,305 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:18,305 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:18,314 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:18,314 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:18,328 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:18,328 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:18,331 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:18,331 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:18,333 [523] [DEBUG] [app] Starting request: urn:request:c1a585c4-f996-48f5-abfd-c3152f0bdd3e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:18,333 [523] [DEBUG] [app] Starting request: urn:request:c1a585c4-f996-48f5-abfd-c3152f0bdd3e (/_internal_ping) 2020-03-31 08:52:18,334 [523] [DEBUG] [app] Ending request: urn:request:c1a585c4-f996-48f5-abfd-c3152f0bdd3e (/_internal_ping) 2020-03-31 08:52:18,335 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:18,335 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 08:52:18,334 [523] [DEBUG] [app] Ending request: urn:request:c1a585c4-f996-48f5-abfd-c3152f0bdd3e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:18,335 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:18,335 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 08:52:18,338 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:18,338 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:18,339 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:18,339 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:18,342 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:18,342 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:18,343 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:52:18,343 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:52:18,346 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 52, 18, 344406), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 52, 18, 344453), u'quay', None, datetime.datetime(2020, 3, 30, 8, 52, 18, 344481), datetime.datetime(2020, 3, 24, 8, 52, 18, 344498), None]) gunicorn-web stdout | 2020-03-31 08:52:18,346 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 8, 52, 18, 344406), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 8, 52, 18, 344453), u'quay', None, datetime.datetime(2020, 3, 30, 8, 52, 18, 344481), datetime.datetime(2020, 3, 24, 8, 52, 18, 344498), None]) 2020-03-31 08:52:18,359 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:18,359 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:18,361 [535] [DEBUG] [app] Starting request: urn:request:8e1f3d4a-bfb0-4343-a2b0-72ce800ff356 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:18,361 [535] [DEBUG] [app] Starting request: urn:request:8e1f3d4a-bfb0-4343-a2b0-72ce800ff356 (/c1/_internal_ping) 2020-03-31 08:52:18,362 [535] [DEBUG] [app] Ending request: urn:request:8e1f3d4a-bfb0-4343-a2b0-72ce800ff356 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:18,362 [535] [DEBUG] [app] Ending request: urn:request:8e1f3d4a-bfb0-4343-a2b0-72ce800ff356 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:18,362 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:18,362 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:18,362 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:52:18,362 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:18,365 [525] [DEBUG] [app] Ending request: urn:request:ab48ae8e-fab9-49db-ab2d-b9a513dacc3b (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:18,365 [525] [DEBUG] [app] Ending request: urn:request:ab48ae8e-fab9-49db-ab2d-b9a513dacc3b (/health/instance) 2020-03-31 08:52:18,365 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:18,365 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:18,366 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.095 49 0.095) gunicorn-web stdout | 2020-03-31 08:52:18,366 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.095 49 0.095) 2020-03-31 08:52:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:52:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:31.440707+00:00 (in 9.998917 seconds) 2020-03-31 08:52:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:31 UTC)" (scheduled at 2020-03-31 08:52:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:52:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:31.440707+00:00 (in 9.998917 seconds) 2020-03-31 08:52:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:31 UTC)" (scheduled at 2020-03-31 08:52:21.440707+00:00) 2020-03-31 08:52:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:52:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:52:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:52:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:52:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:52:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:31 UTC)" executed successfully 2020-03-31 08:52:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:52:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:22 UTC)" (scheduled at 2020-03-31 08:52:22.113221+00:00) securityworker stdout | 2020-03-31 08:52:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:22 UTC)" (scheduled at 2020-03-31 08:52:22.113221+00:00) 2020-03-31 08:52:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:52:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:52.113221+00:00 (in 29.998652 seconds) securityworker stdout | 2020-03-31 08:52:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:52:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:52.113221+00:00 (in 29.998652 seconds) 2020-03-31 08:52:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:52:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:52:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:52:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:52:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:52:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:52:52 UTC)" executed successfully 2020-03-31 08:52:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:31.498397+00:00 (in 7.173985 seconds) 2020-03-31 08:52:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:34 UTC)" (scheduled at 2020-03-31 08:52:24.323352+00:00) 2020-03-31 08:52:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:52:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:31.498397+00:00 (in 7.173985 seconds) 2020-03-31 08:52:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:34 UTC)" (scheduled at 2020-03-31 08:52:24.323352+00:00) 2020-03-31 08:52:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 24, 325096), True, datetime.datetime(2020, 3, 31, 8, 52, 24, 325096), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:52:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 24, 325096), True, datetime.datetime(2020, 3, 31, 8, 52, 24, 325096), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:52:24,349 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:52:24,349 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:24,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:24,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:24,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:24,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:34 UTC)" executed successfully 2020-03-31 08:52:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:52:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:33.330112+00:00 (in 8.918515 seconds) 2020-03-31 08:52:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:24 UTC)" (scheduled at 2020-03-31 08:52:24.410927+00:00) 2020-03-31 08:52:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:52:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:33.330112+00:00 (in 8.918515 seconds) 2020-03-31 08:52:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:24 UTC)" (scheduled at 2020-03-31 08:52:24.410927+00:00) 2020-03-31 08:52:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:24 UTC)" executed successfully 2020-03-31 08:52:24,446 [518] [DEBUG] [app] Starting request: urn:request:86141faf-b143-421c-8f95-25293b06c5db (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:24,446 [518] [DEBUG] [app] Starting request: urn:request:86141faf-b143-421c-8f95-25293b06c5db (/health/instance) 2020-03-31 08:52:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:24,456 [538] [DEBUG] [app] Starting request: urn:request:10d9cf23-be04-4158-a5d6-c3ec41817b0c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:24,456 [538] [DEBUG] [app] Starting request: urn:request:10d9cf23-be04-4158-a5d6-c3ec41817b0c (/v1/_internal_ping) 2020-03-31 08:52:24,457 [538] [DEBUG] [app] Ending request: urn:request:10d9cf23-be04-4158-a5d6-c3ec41817b0c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:24,457 [538] [DEBUG] [app] Ending request: urn:request:10d9cf23-be04-4158-a5d6-c3ec41817b0c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:24,458 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:52:24,458 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:52:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:24,461 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:52:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:52:24,461 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:24,465 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:24,465 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:24,466 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:24,466 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:24,492 [526] [DEBUG] [app] Starting request: urn:request:1b55b7d7-c99b-4eff-9678-2b11c2694d4b (/_internal_ping) 2020-03-31 08:52:24,493 [526] [DEBUG] [app] Ending request: urn:request:1b55b7d7-c99b-4eff-9678-2b11c2694d4b (/_internal_ping) 2020-03-31 08:52:24,494 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:24,492 [526] [DEBUG] [app] Starting request: urn:request:1b55b7d7-c99b-4eff-9678-2b11c2694d4b (/_internal_ping) 2020-03-31 08:52:24,493 [526] [DEBUG] [app] Ending request: urn:request:1b55b7d7-c99b-4eff-9678-2b11c2694d4b (/_internal_ping) 2020-03-31 08:52:24,494 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:52:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:52:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:52:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:24,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:52:24,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:24,509 [535] [DEBUG] [app] Starting request: urn:request:e2b81bee-d0b6-4c77-8ecd-a3f9e06ee088 (/c1/_internal_ping) 2020-03-31 08:52:24,510 [535] [DEBUG] [app] Ending request: urn:request:e2b81bee-d0b6-4c77-8ecd-a3f9e06ee088 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 08:52:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:24,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:24,511 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:52:24,509 [535] [DEBUG] [app] Starting request: urn:request:e2b81bee-d0b6-4c77-8ecd-a3f9e06ee088 (/c1/_internal_ping) 2020-03-31 08:52:24,512 [518] [DEBUG] [app] Ending request: urn:request:86141faf-b143-421c-8f95-25293b06c5db (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:52:24,510 [535] [DEBUG] [app] Ending request: urn:request:e2b81bee-d0b6-4c77-8ecd-a3f9e06ee088 (/c1/_internal_ping) 2020-03-31 08:52:24,512 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:24,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 08:52:24,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:52:24,511 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:24,512 [518] [DEBUG] [app] Ending request: urn:request:86141faf-b143-421c-8f95-25293b06c5db (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:24,512 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:24,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 08:52:26,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:52:26,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:26,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:36.924497+00:00 (in 9.996680 seconds) 2020-03-31 08:52:26,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:36 UTC)" (scheduled at 2020-03-31 08:52:26.924497+00:00) 2020-03-31 08:52:26,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:52:26,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:36.924497+00:00 (in 9.996680 seconds) 2020-03-31 08:52:26,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:36 UTC)" (scheduled at 2020-03-31 08:52:26.924497+00:00) 2020-03-31 08:52:26,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 26, 928917), True, datetime.datetime(2020, 3, 31, 8, 52, 26, 928917), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:52:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 26, 928917), True, datetime.datetime(2020, 3, 31, 8, 52, 26, 928917), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:52:26,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:52:26,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:36 UTC)" executed successfully 2020-03-31 08:52:28,272 [527] [DEBUG] [app] Starting request: urn:request:dc777d14-803e-4227-9656-228cc463f3f7 (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:28,272 [527] [DEBUG] [app] Starting request: urn:request:dc777d14-803e-4227-9656-228cc463f3f7 (/health/instance) 2020-03-31 08:52:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:28,280 [550] [DEBUG] [app] Starting request: urn:request:eb6499c8-0352-4ea0-9ca3-e1d36123e251 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:28,280 [550] [DEBUG] [app] Starting request: urn:request:eb6499c8-0352-4ea0-9ca3-e1d36123e251 (/v1/_internal_ping) 2020-03-31 08:52:28,281 [550] [DEBUG] [app] Ending request: urn:request:eb6499c8-0352-4ea0-9ca3-e1d36123e251 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:28,281 [550] [DEBUG] [app] Ending request: urn:request:eb6499c8-0352-4ea0-9ca3-e1d36123e251 (/v1/_internal_ping) 2020-03-31 08:52:28,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:52:28,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 08:52:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:28,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:28,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:28,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:28,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:28,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:28,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:28,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:28,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:28,315 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:28,315 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:28,320 [527] [DEBUG] [app] Starting request: urn:request:dc7d9412-8c6d-426c-b638-9ef24c7ea2b7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:28,320 [527] [DEBUG] [app] Starting request: urn:request:dc7d9412-8c6d-426c-b638-9ef24c7ea2b7 (/_internal_ping) 2020-03-31 08:52:28,321 [527] [DEBUG] [app] Ending request: urn:request:dc7d9412-8c6d-426c-b638-9ef24c7ea2b7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:28,321 [527] [DEBUG] [app] Ending request: urn:request:dc7d9412-8c6d-426c-b638-9ef24c7ea2b7 (/_internal_ping) 2020-03-31 08:52:28,321 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:28,321 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:28,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:28,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:28,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:28,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:28,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:52:28,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:52:28,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:28,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:28,337 [543] [DEBUG] [app] Starting request: urn:request:6038e27e-0ef8-4a7c-a426-03607e89a298 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:28,337 [543] [DEBUG] [app] Starting request: urn:request:6038e27e-0ef8-4a7c-a426-03607e89a298 (/c1/_internal_ping) 2020-03-31 08:52:28,340 [543] [DEBUG] [app] Ending request: urn:request:6038e27e-0ef8-4a7c-a426-03607e89a298 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:28,340 [543] [DEBUG] [app] Ending request: urn:request:6038e27e-0ef8-4a7c-a426-03607e89a298 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) 2020-03-31 08:52:28,341 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:28,341 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:28,341 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:52:28,341 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:28,343 [527] [DEBUG] [app] Ending request: urn:request:dc777d14-803e-4227-9656-228cc463f3f7 (/health/instance) 2020-03-31 08:52:28,343 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:28,343 [527] [DEBUG] [app] Ending request: urn:request:dc777d14-803e-4227-9656-228cc463f3f7 (/health/instance) 2020-03-31 08:52:28,343 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 08:52:28,343 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:52:28,343 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:52:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:52:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:00 UTC)" (scheduled at 2020-03-31 08:52:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:52:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:00 UTC)" (scheduled at 2020-03-31 08:52:30.337699+00:00) 2020-03-31 08:52:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 52, 30, 339253), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:52:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 52, 30, 339253), False, 50, 1, 0]) 2020-03-31 08:52:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:00.337699+00:00 (in 29.999182 seconds) buildlogsarchiver stdout | 2020-03-31 08:52:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:00.337699+00:00 (in 29.999182 seconds) 2020-03-31 08:52:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:52:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:52:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:52:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:00 UTC)" executed successfully 2020-03-31 08:52:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:52:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:41.440707+00:00 (in 9.997464 seconds) 2020-03-31 08:52:31,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:41 UTC)" (scheduled at 2020-03-31 08:52:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:52:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:41.440707+00:00 (in 9.997464 seconds) 2020-03-31 08:52:31,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:41 UTC)" (scheduled at 2020-03-31 08:52:31.440707+00:00) 2020-03-31 08:52:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:52:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:52:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:52:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:52:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:52:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:41 UTC)" executed successfully 2020-03-31 08:52:31,498 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:31,498 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:34.323352+00:00 (in 2.824267 seconds) 2020-03-31 08:52:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:31 UTC)" (scheduled at 2020-03-31 08:52:31.498397+00:00) notificationworker stdout | 2020-03-31 08:52:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:34.323352+00:00 (in 2.824267 seconds) 2020-03-31 08:52:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:31 UTC)" (scheduled at 2020-03-31 08:52:31.498397+00:00) 2020-03-31 08:52:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), u'notification/%']) notificationworker stdout | 2020-03-31 08:52:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), u'notification/%']) 2020-03-31 08:52:31,529 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), True, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), 0, u'notification/%']) notificationworker stdout | 2020-03-31 08:52:31,529 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), True, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), 0, u'notification/%']) 2020-03-31 08:52:31,552 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), True, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), u'notification/%']) notificationworker stdout | 2020-03-31 08:52:31,552 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), True, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 8, 52, 31, 499969), u'notification/%']) 2020-03-31 08:52:31,556 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:52:31,556 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:31,560 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:31,560 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:57:31 UTC)" executed successfully 2020-03-31 08:52:31,560 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 08:52:31,560 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:31,562 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 08:52:31,562 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:31,562 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 08:52:31,562 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 08:52:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:52:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:03 UTC)" (scheduled at 2020-03-31 08:52:33.330112+00:00) 2020-03-31 08:52:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:52:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:03 UTC)" (scheduled at 2020-03-31 08:52:33.330112+00:00) 2020-03-31 08:52:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 33, 331564), True, datetime.datetime(2020, 3, 31, 8, 52, 33, 331564), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:52:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 33, 331564), True, datetime.datetime(2020, 3, 31, 8, 52, 33, 331564), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:52:33,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:03.330112+00:00 (in 29.999235 seconds) security_notification_worker stdout | 2020-03-31 08:52:33,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:03.330112+00:00 (in 29.999235 seconds) 2020-03-31 08:52:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:52:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:52:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:03 UTC)" executed successfully 2020-03-31 08:52:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:52:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:43.539999+00:00 (in 10.172512 seconds) 2020-03-31 08:52:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:33 UTC)" (scheduled at 2020-03-31 08:52:33.366791+00:00) 2020-03-31 08:52:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:52:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:43.539999+00:00 (in 10.172512 seconds) 2020-03-31 08:52:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:33 UTC)" (scheduled at 2020-03-31 08:52:33.366791+00:00) 2020-03-31 08:52:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 33, 368423), True, datetime.datetime(2020, 3, 31, 8, 52, 33, 368423), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:52:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 33, 368423), True, datetime.datetime(2020, 3, 31, 8, 52, 33, 368423), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:52:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:52:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:52:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:33 UTC)" executed successfully 2020-03-31 08:52:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:44.323352+00:00 (in 9.999186 seconds) 2020-03-31 08:52:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:44 UTC)" (scheduled at 2020-03-31 08:52:34.323352+00:00) 2020-03-31 08:52:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:52:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:44.323352+00:00 (in 9.999186 seconds) 2020-03-31 08:52:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:44 UTC)" (scheduled at 2020-03-31 08:52:34.323352+00:00) 2020-03-31 08:52:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 34, 325036), True, datetime.datetime(2020, 3, 31, 8, 52, 34, 325036), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:52:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 34, 325036), True, datetime.datetime(2020, 3, 31, 8, 52, 34, 325036), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:52:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:52:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:44 UTC)" executed successfully 2020-03-31 08:52:34,446 [527] [DEBUG] [app] Starting request: urn:request:e88034f4-a212-4463-ba61-891ae748ba2e (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:34,446 [527] [DEBUG] [app] Starting request: urn:request:e88034f4-a212-4463-ba61-891ae748ba2e (/health/instance) 2020-03-31 08:52:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:34,454 [556] [DEBUG] [app] Starting request: urn:request:fa83b61c-3a12-41d6-89f6-53243587ece1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:34,454 [556] [DEBUG] [app] Starting request: urn:request:fa83b61c-3a12-41d6-89f6-53243587ece1 (/v1/_internal_ping) 2020-03-31 08:52:34,455 [556] [DEBUG] [app] Ending request: urn:request:fa83b61c-3a12-41d6-89f6-53243587ece1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:34,455 [556] [DEBUG] [app] Ending request: urn:request:fa83b61c-3a12-41d6-89f6-53243587ece1 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:34,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:52:34,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:52:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:34,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:52:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:52:34,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:34,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:34,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:34,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:34,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:34,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:34,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:34,482 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:34,482 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:34,484 [527] [DEBUG] [app] Starting request: urn:request:005f0299-b141-4d54-8611-2cdadc0ffcdb (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:34,484 [527] [DEBUG] [app] Starting request: urn:request:005f0299-b141-4d54-8611-2cdadc0ffcdb (/_internal_ping) 2020-03-31 08:52:34,485 [527] [DEBUG] [app] Ending request: urn:request:005f0299-b141-4d54-8611-2cdadc0ffcdb (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:34,485 [527] [DEBUG] [app] Ending request: urn:request:005f0299-b141-4d54-8611-2cdadc0ffcdb (/_internal_ping) 2020-03-31 08:52:34,485 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:34,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:34,485 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:34,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:52:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:34,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:52:34,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:52:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:52:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:34,499 [543] [DEBUG] [app] Starting request: urn:request:7a26a3a1-90f0-49b6-ab84-95b5892bead6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:34,499 [543] [DEBUG] [app] Starting request: urn:request:7a26a3a1-90f0-49b6-ab84-95b5892bead6 (/c1/_internal_ping) 2020-03-31 08:52:34,500 [543] [DEBUG] [app] Ending request: urn:request:7a26a3a1-90f0-49b6-ab84-95b5892bead6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:34,500 [543] [DEBUG] [app] Ending request: urn:request:7a26a3a1-90f0-49b6-ab84-95b5892bead6 (/c1/_internal_ping) 2020-03-31 08:52:34,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:34,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:34,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:34,502 [527] [DEBUG] [app] Ending request: urn:request:e88034f4-a212-4463-ba61-891ae748ba2e (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:52:34,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:34,503 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:34,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 08:52:34,502 [527] [DEBUG] [app] Ending request: urn:request:e88034f4-a212-4463-ba61-891ae748ba2e (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:34,503 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:34,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 08:52:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:52:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:38.010934+00:00 (in 1.085489 seconds) 2020-03-31 08:52:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:46 UTC)" (scheduled at 2020-03-31 08:52:36.924497+00:00) 2020-03-31 08:52:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:52:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:38.010934+00:00 (in 1.085489 seconds) 2020-03-31 08:52:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:46 UTC)" (scheduled at 2020-03-31 08:52:36.924497+00:00) 2020-03-31 08:52:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 36, 926151), True, datetime.datetime(2020, 3, 31, 8, 52, 36, 926151), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:52:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 36, 926151), True, datetime.datetime(2020, 3, 31, 8, 52, 36, 926151), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:52:36,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:52:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:46 UTC)" executed successfully 2020-03-31 08:52:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:52:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:46.924497+00:00 (in 8.912770 seconds) 2020-03-31 08:52:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:38 UTC)" (scheduled at 2020-03-31 08:52:38.010934+00:00) 2020-03-31 08:52:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:46.924497+00:00 (in 8.912770 seconds) 2020-03-31 08:52:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:38 UTC)" (scheduled at 2020-03-31 08:52:38.010934+00:00) 2020-03-31 08:52:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:38 UTC)" executed successfully 2020-03-31 08:52:38,278 [527] [DEBUG] [app] Starting request: urn:request:7427ef4a-5fe0-445a-a47e-34dfb754cc88 (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:38,278 [527] [DEBUG] [app] Starting request: urn:request:7427ef4a-5fe0-445a-a47e-34dfb754cc88 (/health/instance) 2020-03-31 08:52:38,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:38,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:38,282 [550] [DEBUG] [app] Starting request: urn:request:a97a92ba-485d-4fd6-861c-d1d69d5db493 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:38,282 [550] [DEBUG] [app] Starting request: urn:request:a97a92ba-485d-4fd6-861c-d1d69d5db493 (/v1/_internal_ping) 2020-03-31 08:52:38,283 [550] [DEBUG] [app] Ending request: urn:request:a97a92ba-485d-4fd6-861c-d1d69d5db493 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:38,283 [550] [DEBUG] [app] Ending request: urn:request:a97a92ba-485d-4fd6-861c-d1d69d5db493 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:38,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:38,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:52:38,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:38,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:38,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:52:38,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:38,287 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:38,287 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:38,288 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:38,288 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:38,289 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:38,289 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:38,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:38,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:38,317 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:38,317 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:38,323 [518] [DEBUG] [app] Starting request: urn:request:e4594ad0-1149-4ec5-90aa-be28d1c8689c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:38,323 [518] [DEBUG] [app] Starting request: urn:request:e4594ad0-1149-4ec5-90aa-be28d1c8689c (/_internal_ping) 2020-03-31 08:52:38,324 [518] [DEBUG] [app] Ending request: urn:request:e4594ad0-1149-4ec5-90aa-be28d1c8689c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:38,324 [518] [DEBUG] [app] Ending request: urn:request:e4594ad0-1149-4ec5-90aa-be28d1c8689c (/_internal_ping) 2020-03-31 08:52:38,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:52:38,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:52:38,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:38,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:38,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:38,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:38,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:52:38,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:52:38,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:38,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:38,338 [535] [DEBUG] [app] Starting request: urn:request:f32b2493-c401-4fb9-861e-ce6208d3aa88 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:38,338 [535] [DEBUG] [app] Starting request: urn:request:f32b2493-c401-4fb9-861e-ce6208d3aa88 (/c1/_internal_ping) 2020-03-31 08:52:38,338 [535] [DEBUG] [app] Ending request: urn:request:f32b2493-c401-4fb9-861e-ce6208d3aa88 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:38,338 [535] [DEBUG] [app] Ending request: urn:request:f32b2493-c401-4fb9-861e-ce6208d3aa88 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:38,339 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:38,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:52:38,339 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:38,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:38,341 [527] [DEBUG] [app] Ending request: urn:request:7427ef4a-5fe0-445a-a47e-34dfb754cc88 (/health/instance) 2020-03-31 08:52:38,341 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:38,341 [527] [DEBUG] [app] Ending request: urn:request:7427ef4a-5fe0-445a-a47e-34dfb754cc88 (/health/instance) 2020-03-31 08:52:38,341 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:38,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 08:52:38,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 08:52:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:52:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:51.440707+00:00 (in 9.998902 seconds) 2020-03-31 08:52:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:51 UTC)" (scheduled at 2020-03-31 08:52:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:52:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:51.440707+00:00 (in 9.998902 seconds) 2020-03-31 08:52:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:51 UTC)" (scheduled at 2020-03-31 08:52:41.440707+00:00) 2020-03-31 08:52:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:52:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:52:41,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:52:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:52:41,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:41,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:52:41,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:41,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:51 UTC)" executed successfully 2020-03-31 08:52:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:52:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:33.366791+00:00 (in 49.825865 seconds) 2020-03-31 08:52:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:43 UTC)" (scheduled at 2020-03-31 08:52:43.539999+00:00) 2020-03-31 08:52:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:52:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:33.366791+00:00 (in 49.825865 seconds) 2020-03-31 08:52:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:43 UTC)" (scheduled at 2020-03-31 08:52:43.539999+00:00) 2020-03-31 08:52:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:43 UTC)" executed successfully 2020-03-31 08:52:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:49.498966+00:00 (in 5.174841 seconds) 2020-03-31 08:52:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:54 UTC)" (scheduled at 2020-03-31 08:52:44.323352+00:00) 2020-03-31 08:52:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:52:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:49.498966+00:00 (in 5.174841 seconds) 2020-03-31 08:52:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:54 UTC)" (scheduled at 2020-03-31 08:52:44.323352+00:00) 2020-03-31 08:52:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 44, 325138), True, datetime.datetime(2020, 3, 31, 8, 52, 44, 325138), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:52:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 44, 325138), True, datetime.datetime(2020, 3, 31, 8, 52, 44, 325138), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:52:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:52:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:54 UTC)" executed successfully 2020-03-31 08:52:44,446 [518] [DEBUG] [app] Starting request: urn:request:02bf2685-1e76-42a3-8587-878db0423f3c (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:44,446 [518] [DEBUG] [app] Starting request: urn:request:02bf2685-1e76-42a3-8587-878db0423f3c (/health/instance) 2020-03-31 08:52:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:44,454 [553] [DEBUG] [app] Starting request: urn:request:6220f370-5e91-479a-b990-f198f34bd1b9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:44,454 [553] [DEBUG] [app] Starting request: urn:request:6220f370-5e91-479a-b990-f198f34bd1b9 (/v1/_internal_ping) 2020-03-31 08:52:44,455 [553] [DEBUG] [app] Ending request: urn:request:6220f370-5e91-479a-b990-f198f34bd1b9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:44,455 [553] [DEBUG] [app] Ending request: urn:request:6220f370-5e91-479a-b990-f198f34bd1b9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:44,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:52:44,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:52:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:52:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:44,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:44,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:44,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:44,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:44,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:44,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:44,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:44,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:44,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:44,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:44,484 [518] [DEBUG] [app] Starting request: urn:request:435cfbcb-1167-4e56-8a70-b6885a1dcc38 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:44,484 [518] [DEBUG] [app] Starting request: urn:request:435cfbcb-1167-4e56-8a70-b6885a1dcc38 (/_internal_ping) 2020-03-31 08:52:44,484 [518] [DEBUG] [app] Ending request: urn:request:435cfbcb-1167-4e56-8a70-b6885a1dcc38 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:44,484 [518] [DEBUG] [app] Ending request: urn:request:435cfbcb-1167-4e56-8a70-b6885a1dcc38 (/_internal_ping) 2020-03-31 08:52:44,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:44,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:44,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:44,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:52:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:44,497 [535] [DEBUG] [app] Starting request: urn:request:9e3ca3d2-15da-4bdb-ad1a-7b8372e62d83 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:44,497 [535] [DEBUG] [app] Starting request: urn:request:9e3ca3d2-15da-4bdb-ad1a-7b8372e62d83 (/c1/_internal_ping) 2020-03-31 08:52:44,498 [535] [DEBUG] [app] Ending request: urn:request:9e3ca3d2-15da-4bdb-ad1a-7b8372e62d83 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:44,498 [535] [DEBUG] [app] Ending request: urn:request:9e3ca3d2-15da-4bdb-ad1a-7b8372e62d83 (/c1/_internal_ping) 2020-03-31 08:52:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:44,499 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:52:44,499 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:44,500 [518] [DEBUG] [app] Ending request: urn:request:02bf2685-1e76-42a3-8587-878db0423f3c (/health/instance) 2020-03-31 08:52:44,500 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:44,500 [518] [DEBUG] [app] Ending request: urn:request:02bf2685-1e76-42a3-8587-878db0423f3c (/health/instance) 2020-03-31 08:52:44,500 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:44,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 08:52:44,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 08:52:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:52:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:15.064514+00:00 (in 29.998725 seconds) 2020-03-31 08:52:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:15 UTC)" (scheduled at 2020-03-31 08:52:45.064514+00:00) gcworker stdout | 2020-03-31 08:52:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:15.064514+00:00 (in 29.998725 seconds) 2020-03-31 08:52:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:15 UTC)" (scheduled at 2020-03-31 08:52:45.064514+00:00) 2020-03-31 08:52:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435165, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:52:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435165, 1209600, 500, 1, 0]) 2020-03-31 08:52:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:52:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:52:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:52:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:15 UTC)" executed successfully 2020-03-31 08:52:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:56.924497+00:00 (in 9.999030 seconds) 2020-03-31 08:52:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:56 UTC)" (scheduled at 2020-03-31 08:52:46.924497+00:00) 2020-03-31 08:52:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:52:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:56.924497+00:00 (in 9.999030 seconds) 2020-03-31 08:52:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:56 UTC)" (scheduled at 2020-03-31 08:52:46.924497+00:00) 2020-03-31 08:52:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 46, 926173), True, datetime.datetime(2020, 3, 31, 8, 52, 46, 926173), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:52:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 46, 926173), True, datetime.datetime(2020, 3, 31, 8, 52, 46, 926173), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:52:46,949 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:52:46,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:46,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:46,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:52:56 UTC)" executed successfully 2020-03-31 08:52:48,273 [518] [DEBUG] [app] Starting request: urn:request:12c1eada-0155-415c-aadd-86822dcb999b (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:48,273 [518] [DEBUG] [app] Starting request: urn:request:12c1eada-0155-415c-aadd-86822dcb999b (/health/instance) 2020-03-31 08:52:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:48,280 [538] [DEBUG] [app] Starting request: urn:request:281dd863-8e59-429b-8a3a-b64009c61fb7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:48,280 [538] [DEBUG] [app] Starting request: urn:request:281dd863-8e59-429b-8a3a-b64009c61fb7 (/v1/_internal_ping) 2020-03-31 08:52:48,281 [538] [DEBUG] [app] Ending request: urn:request:281dd863-8e59-429b-8a3a-b64009c61fb7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:48,281 [538] [DEBUG] [app] Ending request: urn:request:281dd863-8e59-429b-8a3a-b64009c61fb7 (/v1/_internal_ping) 2020-03-31 08:52:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:52:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:48,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:48,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:48,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 08:52:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:52:48,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:48,300 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:48,300 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:48,321 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:48,321 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:48,326 [527] [DEBUG] [app] Starting request: urn:request:fe2b1b5c-36a2-4c48-85a8-a42f012f5c73 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:48,326 [527] [DEBUG] [app] Starting request: urn:request:fe2b1b5c-36a2-4c48-85a8-a42f012f5c73 (/_internal_ping) 2020-03-31 08:52:48,327 [527] [DEBUG] [app] Ending request: urn:request:fe2b1b5c-36a2-4c48-85a8-a42f012f5c73 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:48,327 [527] [DEBUG] [app] Ending request: urn:request:fe2b1b5c-36a2-4c48-85a8-a42f012f5c73 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:52:48,328 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:52:48,328 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:48,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:48,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:48,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:52:48,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:52:48,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:48,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:48,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:52:48,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:52:48,341 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:48,341 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:48,345 [532] [DEBUG] [app] Starting request: urn:request:874379ca-22e4-41cd-8097-e2d903bf9e24 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:48,345 [532] [DEBUG] [app] Starting request: urn:request:874379ca-22e4-41cd-8097-e2d903bf9e24 (/c1/_internal_ping) 2020-03-31 08:52:48,349 [532] [DEBUG] [app] Ending request: urn:request:874379ca-22e4-41cd-8097-e2d903bf9e24 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:48,349 [532] [DEBUG] [app] Ending request: urn:request:874379ca-22e4-41cd-8097-e2d903bf9e24 (/c1/_internal_ping) 2020-03-31 08:52:48,350 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) 2020-03-31 08:52:48,349 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:48,350 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) gunicorn-verbs stdout | 2020-03-31 08:52:48,349 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:48,352 [518] [DEBUG] [app] Ending request: urn:request:12c1eada-0155-415c-aadd-86822dcb999b (/health/instance) 2020-03-31 08:52:48,352 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:48,353 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) gunicorn-web stdout | 2020-03-31 08:52:48,352 [518] [DEBUG] [app] Ending request: urn:request:12c1eada-0155-415c-aadd-86822dcb999b (/health/instance) 2020-03-31 08:52:48,352 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:48,353 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) 2020-03-31 08:52:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:54.323352+00:00 (in 4.823556 seconds) 2020-03-31 08:52:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:49 UTC)" (scheduled at 2020-03-31 08:52:49.498966+00:00) 2020-03-31 08:52:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:54.323352+00:00 (in 4.823556 seconds) 2020-03-31 08:52:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:49 UTC)" (scheduled at 2020-03-31 08:52:49.498966+00:00) 2020-03-31 08:52:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:49 UTC)" executed successfully 2020-03-31 08:52:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:52:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:59.560326+00:00 (in 8.448909 seconds) 2020-03-31 08:52:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:51 UTC)" (scheduled at 2020-03-31 08:52:51.110610+00:00) 2020-03-31 08:52:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:52:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:52:59.560326+00:00 (in 8.448909 seconds) 2020-03-31 08:52:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:51 UTC)" (scheduled at 2020-03-31 08:52:51.110610+00:00) 2020-03-31 08:52:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 51, 112308), True, datetime.datetime(2020, 3, 31, 8, 52, 51, 112308), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:52:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 51, 112308), True, datetime.datetime(2020, 3, 31, 8, 52, 51, 112308), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:52:51,310 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:52:51,310 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:51,310 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:51,311 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:52:51,310 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:51,311 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:51 UTC)" executed successfully 2020-03-31 08:52:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:52:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:01 UTC)" (scheduled at 2020-03-31 08:52:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:52:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:01 UTC)" (scheduled at 2020-03-31 08:52:51.440707+00:00) 2020-03-31 08:52:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:01.440707+00:00 (in 9.998957 seconds) repositoryactioncounter stdout | 2020-03-31 08:52:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:01.440707+00:00 (in 9.998957 seconds) 2020-03-31 08:52:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:52:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:52:51,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:52:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:52:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:52:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:01 UTC)" executed successfully 2020-03-31 08:52:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:52:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:22.113221+00:00 (in 29.997210 seconds) 2020-03-31 08:52:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:22 UTC)" (scheduled at 2020-03-31 08:52:52.113221+00:00) securityworker stdout | 2020-03-31 08:52:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:22.113221+00:00 (in 29.997210 seconds) 2020-03-31 08:52:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:22 UTC)" (scheduled at 2020-03-31 08:52:52.113221+00:00) 2020-03-31 08:52:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:52:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:52:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:52:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:52:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:52:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:52:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:52:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:22 UTC)" executed successfully 2020-03-31 08:52:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:52:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:04.323352+00:00 (in 9.999213 seconds) 2020-03-31 08:52:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:04 UTC)" (scheduled at 2020-03-31 08:52:54.323352+00:00) 2020-03-31 08:52:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:52:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:04.323352+00:00 (in 9.999213 seconds) 2020-03-31 08:52:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:04 UTC)" (scheduled at 2020-03-31 08:52:54.323352+00:00) 2020-03-31 08:52:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 54, 324897), True, datetime.datetime(2020, 3, 31, 8, 52, 54, 324897), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:52:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 54, 324897), True, datetime.datetime(2020, 3, 31, 8, 52, 54, 324897), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:52:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:52:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:52:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:04 UTC)" executed successfully 2020-03-31 08:52:54,446 [527] [DEBUG] [app] Starting request: urn:request:f9b9be31-e603-4674-9d39-158560f08f89 (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:54,446 [527] [DEBUG] [app] Starting request: urn:request:f9b9be31-e603-4674-9d39-158560f08f89 (/health/instance) 2020-03-31 08:52:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:54,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:54,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:54,457 [547] [DEBUG] [app] Starting request: urn:request:ca45bf79-c41f-443f-9667-a933fb50be51 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:54,457 [547] [DEBUG] [app] Starting request: urn:request:ca45bf79-c41f-443f-9667-a933fb50be51 (/v1/_internal_ping) 2020-03-31 08:52:54,461 [547] [DEBUG] [app] Ending request: urn:request:ca45bf79-c41f-443f-9667-a933fb50be51 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:54,461 [547] [DEBUG] [app] Ending request: urn:request:ca45bf79-c41f-443f-9667-a933fb50be51 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:52:54,462 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:52:54,462 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:54,462 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 08:52:54,462 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:54,463 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:52:54,463 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:54,468 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:54,468 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:54,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:54,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:54,473 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:54,473 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:54,474 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:54,474 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:54,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:54,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:54,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:54,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:54,490 [527] [DEBUG] [app] Starting request: urn:request:6f88d35b-b344-448b-8bfe-d360fb10c7ba (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:54,490 [527] [DEBUG] [app] Starting request: urn:request:6f88d35b-b344-448b-8bfe-d360fb10c7ba (/_internal_ping) 2020-03-31 08:52:54,490 [527] [DEBUG] [app] Ending request: urn:request:6f88d35b-b344-448b-8bfe-d360fb10c7ba (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:54,490 [527] [DEBUG] [app] Ending request: urn:request:6f88d35b-b344-448b-8bfe-d360fb10c7ba (/_internal_ping) 2020-03-31 08:52:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:52:54,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:54,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:54,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:52:54,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:52:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:52:54,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:54,503 [532] [DEBUG] [app] Starting request: urn:request:782c95fb-5516-411f-943c-8055e97c7881 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:54,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 08:52:54,503 [532] [DEBUG] [app] Starting request: urn:request:782c95fb-5516-411f-943c-8055e97c7881 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:54,504 [532] [DEBUG] [app] Ending request: urn:request:782c95fb-5516-411f-943c-8055e97c7881 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:52:54,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:54,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:52:54,504 [532] [DEBUG] [app] Ending request: urn:request:782c95fb-5516-411f-943c-8055e97c7881 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:54,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:54,506 [527] [DEBUG] [app] Ending request: urn:request:f9b9be31-e603-4674-9d39-158560f08f89 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:52:54,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:54,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:54,506 [527] [DEBUG] [app] Ending request: urn:request:f9b9be31-e603-4674-9d39-158560f08f89 (/health/instance) 2020-03-31 08:52:54,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 08:52:54,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:54,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 08:52:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:52:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:06 UTC)" (scheduled at 2020-03-31 08:52:56.924497+00:00) 2020-03-31 08:52:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:56,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:06.924497+00:00 (in 9.999156 seconds) storagereplication stdout | 2020-03-31 08:52:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:06 UTC)" (scheduled at 2020-03-31 08:52:56.924497+00:00) 2020-03-31 08:52:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:52:56,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:06.924497+00:00 (in 9.999156 seconds) 2020-03-31 08:52:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 56, 926111), True, datetime.datetime(2020, 3, 31, 8, 52, 56, 926111), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:52:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 52, 56, 926111), True, datetime.datetime(2020, 3, 31, 8, 52, 56, 926111), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:52:56,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:52:56,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:52:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:52:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:52:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:06 UTC)" executed successfully 2020-03-31 08:52:58,273 [518] [DEBUG] [app] Starting request: urn:request:8cf4cb45-0906-4c1d-9cb4-b36e80c61ee6 (/health/instance) gunicorn-web stdout | 2020-03-31 08:52:58,273 [518] [DEBUG] [app] Starting request: urn:request:8cf4cb45-0906-4c1d-9cb4-b36e80c61ee6 (/health/instance) 2020-03-31 08:52:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:58,284 [549] [DEBUG] [app] Starting request: urn:request:44fc1cae-d501-44bc-b769-8dbb0d1610cc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:58,284 [549] [DEBUG] [app] Starting request: urn:request:44fc1cae-d501-44bc-b769-8dbb0d1610cc (/v1/_internal_ping) 2020-03-31 08:52:58,287 [549] [DEBUG] [app] Ending request: urn:request:44fc1cae-d501-44bc-b769-8dbb0d1610cc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:52:58,287 [549] [DEBUG] [app] Ending request: urn:request:44fc1cae-d501-44bc-b769-8dbb0d1610cc (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:52:58,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:52:58,288 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:58,288 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:58,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:52:58,288 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:58,289 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:52:58,294 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:52:58,294 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:52:58,299 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:52:58,299 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:52:58,300 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:52:58,300 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:52:58,301 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:52:58,301 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:52:58,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:52:58,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:52:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:58,321 [526] [DEBUG] [app] Starting request: urn:request:b256b8e0-53c5-4f97-8da5-91e8b936807b (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:58,321 [526] [DEBUG] [app] Starting request: urn:request:b256b8e0-53c5-4f97-8da5-91e8b936807b (/_internal_ping) 2020-03-31 08:52:58,322 [526] [DEBUG] [app] Ending request: urn:request:b256b8e0-53c5-4f97-8da5-91e8b936807b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:58,322 [526] [DEBUG] [app] Ending request: urn:request:b256b8e0-53c5-4f97-8da5-91e8b936807b (/_internal_ping) 2020-03-31 08:52:58,322 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:52:58,322 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:52:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:52:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:52:58,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:58,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:58,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:52:58,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:52:58,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:52:58,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:52:58,335 [532] [DEBUG] [app] Starting request: urn:request:1f5eec83-6b6b-4cec-88e2-81e7f833be7a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:52:58,335 [532] [DEBUG] [app] Starting request: urn:request:1f5eec83-6b6b-4cec-88e2-81e7f833be7a (/c1/_internal_ping) 2020-03-31 08:52:58,336 [532] [DEBUG] [app] Ending request: urn:request:1f5eec83-6b6b-4cec-88e2-81e7f833be7a (/c1/_internal_ping) 2020-03-31 08:52:58,337 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:58,337 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:52:58,336 [532] [DEBUG] [app] Ending request: urn:request:1f5eec83-6b6b-4cec-88e2-81e7f833be7a (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:52:58,337 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:52:58,338 [518] [DEBUG] [app] Ending request: urn:request:8cf4cb45-0906-4c1d-9cb4-b36e80c61ee6 (/health/instance) 2020-03-31 08:52:58,339 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:52:58,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-verbs stdout | 2020-03-31 08:52:58,337 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:52:58,338 [518] [DEBUG] [app] Ending request: urn:request:8cf4cb45-0906-4c1d-9cb4-b36e80c61ee6 (/health/instance) 2020-03-31 08:52:58,339 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:52:58,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:52:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:52:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 08:52:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:52:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:52:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:51.110610+00:00 (in 51.549168 seconds) 2020-03-31 08:52:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:59 UTC)" (scheduled at 2020-03-31 08:52:59.560326+00:00) 2020-03-31 08:52:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 08:52:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:51.110610+00:00 (in 51.549168 seconds) 2020-03-31 08:52:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:59 UTC)" (scheduled at 2020-03-31 08:52:59.560326+00:00) 2020-03-31 08:52:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:52:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:52:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:53:59 UTC)" executed successfully 2020-03-31 08:53:00,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:53:00,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:30.337699+00:00 (in 29.996629 seconds) 2020-03-31 08:53:00,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:30 UTC)" (scheduled at 2020-03-31 08:53:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:53:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:30.337699+00:00 (in 29.996629 seconds) 2020-03-31 08:53:00,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:30 UTC)" (scheduled at 2020-03-31 08:53:00.337699+00:00) 2020-03-31 08:53:00,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 53, 0, 341807), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:53:00,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 53, 0, 341807), False, 50, 1, 0]) 2020-03-31 08:53:00,359 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:53:00,359 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:53:00,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:53:00,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:30 UTC)" executed successfully 2020-03-31 08:53:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:53:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:11.440707+00:00 (in 9.997831 seconds) 2020-03-31 08:53:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:11 UTC)" (scheduled at 2020-03-31 08:53:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:53:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:11.440707+00:00 (in 9.997831 seconds) 2020-03-31 08:53:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:11 UTC)" (scheduled at 2020-03-31 08:53:01.440707+00:00) 2020-03-31 08:53:01,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:53:01,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:53:01,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:53:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:53:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:53:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:11 UTC)" executed successfully 2020-03-31 08:53:03,334 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:53:03,334 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:03,335 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:24.410927+00:00 (in 21.075925 seconds) 2020-03-31 08:53:03,335 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:33 UTC)" (scheduled at 2020-03-31 08:53:03.330112+00:00) 2020-03-31 08:53:03,335 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:53:03,335 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:24.410927+00:00 (in 21.075925 seconds) 2020-03-31 08:53:03,335 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:33 UTC)" (scheduled at 2020-03-31 08:53:03.330112+00:00) 2020-03-31 08:53:03,335 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:03,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 3, 335984), True, datetime.datetime(2020, 3, 31, 8, 53, 3, 335984), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:53:03,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 3, 335984), True, datetime.datetime(2020, 3, 31, 8, 53, 3, 335984), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:53:03,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:53:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:53:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:33 UTC)" executed successfully 2020-03-31 08:53:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:53:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:14.323352+00:00 (in 9.998774 seconds) 2020-03-31 08:53:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:14 UTC)" (scheduled at 2020-03-31 08:53:04.323352+00:00) 2020-03-31 08:53:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:53:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:14.323352+00:00 (in 9.998774 seconds) 2020-03-31 08:53:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:14 UTC)" (scheduled at 2020-03-31 08:53:04.323352+00:00) 2020-03-31 08:53:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 4, 325781), True, datetime.datetime(2020, 3, 31, 8, 53, 4, 325781), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:53:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 4, 325781), True, datetime.datetime(2020, 3, 31, 8, 53, 4, 325781), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:53:04,350 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:53:04,350 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:04,350 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:04,350 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:53:04,350 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:04,350 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:14 UTC)" executed successfully 2020-03-31 08:53:04,446 [526] [DEBUG] [app] Starting request: urn:request:60533029-3c26-485f-8cae-27721475f96a (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:04,446 [526] [DEBUG] [app] Starting request: urn:request:60533029-3c26-485f-8cae-27721475f96a (/health/instance) 2020-03-31 08:53:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:04,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:04,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:04,454 [549] [DEBUG] [app] Starting request: urn:request:1998d872-64ef-40d7-81f3-9ee54c8c57f6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:04,454 [549] [DEBUG] [app] Starting request: urn:request:1998d872-64ef-40d7-81f3-9ee54c8c57f6 (/v1/_internal_ping) 2020-03-31 08:53:04,455 [549] [DEBUG] [app] Ending request: urn:request:1998d872-64ef-40d7-81f3-9ee54c8c57f6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:04,455 [549] [DEBUG] [app] Ending request: urn:request:1998d872-64ef-40d7-81f3-9ee54c8c57f6 (/v1/_internal_ping) 2020-03-31 08:53:04,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:04,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:04,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:04,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:04,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:04,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:53:04,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:53:04,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:04,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:04,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:04,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:04,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:04,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:04,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:04,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:04,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:04,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:04,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:04,487 [526] [DEBUG] [app] Starting request: urn:request:023c836c-f39a-4131-b8a6-ad3810ca9e96 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:04,487 [526] [DEBUG] [app] Starting request: urn:request:023c836c-f39a-4131-b8a6-ad3810ca9e96 (/_internal_ping) 2020-03-31 08:53:04,488 [526] [DEBUG] [app] Ending request: urn:request:023c836c-f39a-4131-b8a6-ad3810ca9e96 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:04,488 [526] [DEBUG] [app] Ending request: urn:request:023c836c-f39a-4131-b8a6-ad3810ca9e96 (/_internal_ping) 2020-03-31 08:53:04,488 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:53:04,488 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:04,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:53:04,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:04,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:04,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:04,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:04,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:04,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:04,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:53:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:53:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:04,501 [532] [DEBUG] [app] Starting request: urn:request:eeed49a0-0f81-48fc-b4cf-44351be2e202 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:04,501 [532] [DEBUG] [app] Starting request: urn:request:eeed49a0-0f81-48fc-b4cf-44351be2e202 (/c1/_internal_ping) 2020-03-31 08:53:04,502 [532] [DEBUG] [app] Ending request: urn:request:eeed49a0-0f81-48fc-b4cf-44351be2e202 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:04,502 [532] [DEBUG] [app] Ending request: urn:request:eeed49a0-0f81-48fc-b4cf-44351be2e202 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:04,502 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:04,502 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:04,502 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:53:04,502 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:04,504 [526] [DEBUG] [app] Ending request: urn:request:60533029-3c26-485f-8cae-27721475f96a (/health/instance) 2020-03-31 08:53:04,504 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:04,504 [526] [DEBUG] [app] Ending request: urn:request:60533029-3c26-485f-8cae-27721475f96a (/health/instance) 2020-03-31 08:53:04,504 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:04,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:53:04,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:53:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:53:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:16.924497+00:00 (in 9.998371 seconds) 2020-03-31 08:53:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:16 UTC)" (scheduled at 2020-03-31 08:53:06.924497+00:00) 2020-03-31 08:53:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:53:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:16.924497+00:00 (in 9.998371 seconds) 2020-03-31 08:53:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:16 UTC)" (scheduled at 2020-03-31 08:53:06.924497+00:00) 2020-03-31 08:53:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 6, 926809), True, datetime.datetime(2020, 3, 31, 8, 53, 6, 926809), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:53:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 6, 926809), True, datetime.datetime(2020, 3, 31, 8, 53, 6, 926809), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:53:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:53:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:53:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:16 UTC)" executed successfully 2020-03-31 08:53:08,273 [518] [DEBUG] [app] Starting request: urn:request:11d151e7-4d6b-4980-b16f-b6fa585bfb6c (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:08,273 [518] [DEBUG] [app] Starting request: urn:request:11d151e7-4d6b-4980-b16f-b6fa585bfb6c (/health/instance) 2020-03-31 08:53:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:08,280 [551] [DEBUG] [app] Starting request: urn:request:dddc3889-49a8-4706-b9c3-33ad34b07df7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:08,280 [551] [DEBUG] [app] Starting request: urn:request:dddc3889-49a8-4706-b9c3-33ad34b07df7 (/v1/_internal_ping) 2020-03-31 08:53:08,281 [551] [DEBUG] [app] Ending request: urn:request:dddc3889-49a8-4706-b9c3-33ad34b07df7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:08,281 [551] [DEBUG] [app] Ending request: urn:request:dddc3889-49a8-4706-b9c3-33ad34b07df7 (/v1/_internal_ping) 2020-03-31 08:53:08,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:08,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:08,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:53:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:53:08,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:08,285 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:08,285 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:08,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:08,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:08,314 [526] [DEBUG] [app] Starting request: urn:request:75bd3c41-361f-46df-948f-eaaac0bd95bb (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:08,314 [526] [DEBUG] [app] Starting request: urn:request:75bd3c41-361f-46df-948f-eaaac0bd95bb (/_internal_ping) 2020-03-31 08:53:08,315 [526] [DEBUG] [app] Ending request: urn:request:75bd3c41-361f-46df-948f-eaaac0bd95bb (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:08,315 [526] [DEBUG] [app] Ending request: urn:request:75bd3c41-361f-46df-948f-eaaac0bd95bb (/_internal_ping) 2020-03-31 08:53:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:53:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:53:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:53:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:08,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:08,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:08,330 [539] [DEBUG] [app] Starting request: urn:request:9566053b-3701-4a79-af2b-780b1d0597cd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:08,330 [539] [DEBUG] [app] Starting request: urn:request:9566053b-3701-4a79-af2b-780b1d0597cd (/c1/_internal_ping) 2020-03-31 08:53:08,331 [539] [DEBUG] [app] Ending request: urn:request:9566053b-3701-4a79-af2b-780b1d0597cd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:08,331 [539] [DEBUG] [app] Ending request: urn:request:9566053b-3701-4a79-af2b-780b1d0597cd (/c1/_internal_ping) 2020-03-31 08:53:08,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:08,332 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:08,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:53:08,332 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:08,334 [518] [DEBUG] [app] Ending request: urn:request:11d151e7-4d6b-4980-b16f-b6fa585bfb6c (/health/instance) 2020-03-31 08:53:08,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:08,334 [518] [DEBUG] [app] Ending request: urn:request:11d151e7-4d6b-4980-b16f-b6fa585bfb6c (/health/instance) 2020-03-31 08:53:08,334 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 08:53:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:53:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:53:11,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:53:11,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:21.440707+00:00 (in 9.995229 seconds) 2020-03-31 08:53:11,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:21 UTC)" (scheduled at 2020-03-31 08:53:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:53:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:21.440707+00:00 (in 9.995229 seconds) 2020-03-31 08:53:11,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:21 UTC)" (scheduled at 2020-03-31 08:53:11.440707+00:00) 2020-03-31 08:53:11,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:53:11,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:53:11,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:53:11,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:53:11,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:11,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:53:11,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:11,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:21 UTC)" executed successfully 2020-03-31 08:53:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:53:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:24 UTC)" (scheduled at 2020-03-31 08:53:14.323352+00:00) 2020-03-31 08:53:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:53:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:24 UTC)" (scheduled at 2020-03-31 08:53:14.323352+00:00) 2020-03-31 08:53:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 14, 324987), True, datetime.datetime(2020, 3, 31, 8, 53, 14, 324987), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:53:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 14, 324987), True, datetime.datetime(2020, 3, 31, 8, 53, 14, 324987), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:53:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:24.323352+00:00 (in 9.999063 seconds) notificationworker stdout | 2020-03-31 08:53:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:24.323352+00:00 (in 9.999063 seconds) 2020-03-31 08:53:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:53:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:53:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:24 UTC)" executed successfully 2020-03-31 08:53:14,446 [527] [DEBUG] [app] Starting request: urn:request:b9901c91-13e3-42d7-851f-26dd57ef6540 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:14,446 [527] [DEBUG] [app] Starting request: urn:request:b9901c91-13e3-42d7-851f-26dd57ef6540 (/health/instance) 2020-03-31 08:53:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:14,454 [555] [DEBUG] [app] Starting request: urn:request:f359a76b-70b0-4dd6-b216-1a7ea95da721 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:14,454 [555] [DEBUG] [app] Starting request: urn:request:f359a76b-70b0-4dd6-b216-1a7ea95da721 (/v1/_internal_ping) 2020-03-31 08:53:14,455 [555] [DEBUG] [app] Ending request: urn:request:f359a76b-70b0-4dd6-b216-1a7ea95da721 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:14,455 [555] [DEBUG] [app] Ending request: urn:request:f359a76b-70b0-4dd6-b216-1a7ea95da721 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:53:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:14,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:14,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:14,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:14,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:14,488 [527] [DEBUG] [app] Starting request: urn:request:78a25499-e2e4-4695-af14-2c7764eea173 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:14,488 [527] [DEBUG] [app] Starting request: urn:request:78a25499-e2e4-4695-af14-2c7764eea173 (/_internal_ping) 2020-03-31 08:53:14,489 [527] [DEBUG] [app] Ending request: urn:request:78a25499-e2e4-4695-af14-2c7764eea173 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:14,489 [527] [DEBUG] [app] Ending request: urn:request:78a25499-e2e4-4695-af14-2c7764eea173 (/_internal_ping) 2020-03-31 08:53:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:53:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:53:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:53:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:14,501 [537] [DEBUG] [app] Starting request: urn:request:5d988575-083a-44d9-97fd-597b2cf6d021 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:14,501 [537] [DEBUG] [app] Starting request: urn:request:5d988575-083a-44d9-97fd-597b2cf6d021 (/c1/_internal_ping) 2020-03-31 08:53:14,502 [537] [DEBUG] [app] Ending request: urn:request:5d988575-083a-44d9-97fd-597b2cf6d021 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:14,502 [537] [DEBUG] [app] Ending request: urn:request:5d988575-083a-44d9-97fd-597b2cf6d021 (/c1/_internal_ping) 2020-03-31 08:53:14,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:14,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:14,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:14,504 [527] [DEBUG] [app] Ending request: urn:request:b9901c91-13e3-42d7-851f-26dd57ef6540 (/health/instance) 2020-03-31 08:53:14,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:53:14,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:14,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:53:14,504 [527] [DEBUG] [app] Ending request: urn:request:b9901c91-13e3-42d7-851f-26dd57ef6540 (/health/instance) 2020-03-31 08:53:14,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:14,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:53:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:53:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:45.064514+00:00 (in 29.998965 seconds) 2020-03-31 08:53:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:45 UTC)" (scheduled at 2020-03-31 08:53:15.064514+00:00) gcworker stdout | 2020-03-31 08:53:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:45.064514+00:00 (in 29.998965 seconds) 2020-03-31 08:53:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:45 UTC)" (scheduled at 2020-03-31 08:53:15.064514+00:00) 2020-03-31 08:53:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435195, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:53:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435195, 1209600, 500, 1, 0]) 2020-03-31 08:53:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:53:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:53:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:53:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:45 UTC)" executed successfully 2020-03-31 08:53:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:26.924497+00:00 (in 9.999242 seconds) 2020-03-31 08:53:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:26 UTC)" (scheduled at 2020-03-31 08:53:16.924497+00:00) 2020-03-31 08:53:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:53:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:53:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:26.924497+00:00 (in 9.999242 seconds) 2020-03-31 08:53:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:26 UTC)" (scheduled at 2020-03-31 08:53:16.924497+00:00) 2020-03-31 08:53:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 16, 926091), True, datetime.datetime(2020, 3, 31, 8, 53, 16, 926091), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:53:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 16, 926091), True, datetime.datetime(2020, 3, 31, 8, 53, 16, 926091), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:53:16,950 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:53:16,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:16,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:16,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:53:16,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:16,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:26 UTC)" executed successfully 2020-03-31 08:53:18,273 [527] [DEBUG] [app] Starting request: urn:request:a2e4eefa-299b-4d76-885b-c15207f8ff9d (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:18,273 [527] [DEBUG] [app] Starting request: urn:request:a2e4eefa-299b-4d76-885b-c15207f8ff9d (/health/instance) 2020-03-31 08:53:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:18,278 [553] [DEBUG] [app] Starting request: urn:request:ef9c694c-affa-4f34-9cbc-a21cd94f394a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:18,278 [553] [DEBUG] [app] Starting request: urn:request:ef9c694c-affa-4f34-9cbc-a21cd94f394a (/v1/_internal_ping) 2020-03-31 08:53:18,279 [553] [DEBUG] [app] Ending request: urn:request:ef9c694c-affa-4f34-9cbc-a21cd94f394a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:18,279 [553] [DEBUG] [app] Ending request: urn:request:ef9c694c-affa-4f34-9cbc-a21cd94f394a (/v1/_internal_ping) 2020-03-31 08:53:18,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:53:18,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:18,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:18,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:18,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:18,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:53:18,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:53:18,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:18,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:18,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:18,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:18,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:18,299 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:18,299 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:18,316 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:18,316 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:18,322 [527] [DEBUG] [app] Starting request: urn:request:f0ff895c-866e-41f6-94dc-284d2aa60583 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:18,322 [527] [DEBUG] [app] Starting request: urn:request:f0ff895c-866e-41f6-94dc-284d2aa60583 (/_internal_ping) 2020-03-31 08:53:18,323 [527] [DEBUG] [app] Ending request: urn:request:f0ff895c-866e-41f6-94dc-284d2aa60583 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:18,323 [527] [DEBUG] [app] Ending request: urn:request:f0ff895c-866e-41f6-94dc-284d2aa60583 (/_internal_ping) 2020-03-31 08:53:18,323 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:53:18,323 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:18,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:53:18,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:18,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:18,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:18,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:53:18,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:53:18,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:18,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:53:18,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:18,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:18,334 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:18,334 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:18,335 [543] [DEBUG] [app] Starting request: urn:request:f74377ec-6be0-4493-873e-7a32f10dd593 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:18,335 [543] [DEBUG] [app] Starting request: urn:request:f74377ec-6be0-4493-873e-7a32f10dd593 (/c1/_internal_ping) 2020-03-31 08:53:18,336 [543] [DEBUG] [app] Ending request: urn:request:f74377ec-6be0-4493-873e-7a32f10dd593 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:18,336 [543] [DEBUG] [app] Ending request: urn:request:f74377ec-6be0-4493-873e-7a32f10dd593 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:18,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:18,337 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:18,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:53:18,337 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:18,338 [527] [DEBUG] [app] Ending request: urn:request:a2e4eefa-299b-4d76-885b-c15207f8ff9d (/health/instance) 2020-03-31 08:53:18,338 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:18,338 [527] [DEBUG] [app] Ending request: urn:request:a2e4eefa-299b-4d76-885b-c15207f8ff9d (/health/instance) 2020-03-31 08:53:18,338 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:18,339 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 08:53:18,339 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 08:53:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:53:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:31 UTC)" (scheduled at 2020-03-31 08:53:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:53:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:31 UTC)" (scheduled at 2020-03-31 08:53:21.440707+00:00) 2020-03-31 08:53:21,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:31.440707+00:00 (in 9.998964 seconds) repositoryactioncounter stdout | 2020-03-31 08:53:21,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:31.440707+00:00 (in 9.998964 seconds) 2020-03-31 08:53:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:53:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:53:21,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:53:21,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:53:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:53:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:31 UTC)" executed successfully 2020-03-31 08:53:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:53:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:52.113221+00:00 (in 29.998809 seconds) 2020-03-31 08:53:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:52 UTC)" (scheduled at 2020-03-31 08:53:22.113221+00:00) securityworker stdout | 2020-03-31 08:53:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:52.113221+00:00 (in 29.998809 seconds) 2020-03-31 08:53:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:52 UTC)" (scheduled at 2020-03-31 08:53:22.113221+00:00) 2020-03-31 08:53:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:53:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:53:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:53:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:53:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:53:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:53:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:53:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:52 UTC)" executed successfully 2020-03-31 08:53:24,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:53:24,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:34.323352+00:00 (in 9.994945 seconds) 2020-03-31 08:53:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:34 UTC)" (scheduled at 2020-03-31 08:53:24.323352+00:00) 2020-03-31 08:53:24,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:53:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:34.323352+00:00 (in 9.994945 seconds) 2020-03-31 08:53:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:34 UTC)" (scheduled at 2020-03-31 08:53:24.323352+00:00) 2020-03-31 08:53:24,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 24, 329413), True, datetime.datetime(2020, 3, 31, 8, 53, 24, 329413), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:53:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 24, 329413), True, datetime.datetime(2020, 3, 31, 8, 53, 24, 329413), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:53:24,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:53:24,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:24,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:53:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:24,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:34 UTC)" executed successfully 2020-03-31 08:53:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:53:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:24 UTC)" (scheduled at 2020-03-31 08:53:24.410927+00:00) 2020-03-31 08:53:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:53:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:24 UTC)" (scheduled at 2020-03-31 08:53:24.410927+00:00) 2020-03-31 08:53:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:24 UTC)" executed successfully 2020-03-31 08:53:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:33.330112+00:00 (in 8.918339 seconds) security_notification_worker stdout | 2020-03-31 08:53:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:33.330112+00:00 (in 8.918339 seconds) 2020-03-31 08:53:24,446 [527] [DEBUG] [app] Starting request: urn:request:3d8dddb4-01cd-436a-b99d-60a0d50b1146 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:24,446 [527] [DEBUG] [app] Starting request: urn:request:3d8dddb4-01cd-436a-b99d-60a0d50b1146 (/health/instance) 2020-03-31 08:53:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:24,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:24,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:24,453 [549] [DEBUG] [app] Starting request: urn:request:785bf27a-cdf5-41de-8913-06e50701d74c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:24,453 [549] [DEBUG] [app] Starting request: urn:request:785bf27a-cdf5-41de-8913-06e50701d74c (/v1/_internal_ping) 2020-03-31 08:53:24,455 [549] [DEBUG] [app] Ending request: urn:request:785bf27a-cdf5-41de-8913-06e50701d74c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:24,455 [549] [DEBUG] [app] Ending request: urn:request:785bf27a-cdf5-41de-8913-06e50701d74c (/v1/_internal_ping) 2020-03-31 08:53:24,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:24,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:24,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:24,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:24,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:53:24,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:24,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:24,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:24,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:24,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:24,461 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:24,461 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:24,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:24,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:24,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:24,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:24,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:24,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:24,492 [520] [DEBUG] [app] Starting request: urn:request:c328c383-6a75-405b-9309-db03d48cd4a9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:24,492 [520] [DEBUG] [app] Starting request: urn:request:c328c383-6a75-405b-9309-db03d48cd4a9 (/_internal_ping) 2020-03-31 08:53:24,495 [520] [DEBUG] [app] Ending request: urn:request:c328c383-6a75-405b-9309-db03d48cd4a9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:24,495 [520] [DEBUG] [app] Ending request: urn:request:c328c383-6a75-405b-9309-db03d48cd4a9 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 159 0.010) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 159 0.010) 2020-03-31 08:53:24,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:24,497 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:24,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:24,497 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:24,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:24,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:24,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:24,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:53:24,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:24,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:24,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:24,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:24,510 [537] [DEBUG] [app] Starting request: urn:request:48a63fcf-9ecf-4907-8244-e91fcd252935 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:24,510 [537] [DEBUG] [app] Starting request: urn:request:48a63fcf-9ecf-4907-8244-e91fcd252935 (/c1/_internal_ping) 2020-03-31 08:53:24,511 [537] [DEBUG] [app] Ending request: urn:request:48a63fcf-9ecf-4907-8244-e91fcd252935 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:24,511 [537] [DEBUG] [app] Ending request: urn:request:48a63fcf-9ecf-4907-8244-e91fcd252935 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:24,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:24,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:24,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:53:24,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:24,513 [527] [DEBUG] [app] Ending request: urn:request:3d8dddb4-01cd-436a-b99d-60a0d50b1146 (/health/instance) 2020-03-31 08:53:24,513 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:24,513 [527] [DEBUG] [app] Ending request: urn:request:3d8dddb4-01cd-436a-b99d-60a0d50b1146 (/health/instance) 2020-03-31 08:53:24,513 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:24,514 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 08:53:24,514 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 08:53:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:53:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:36.924497+00:00 (in 9.999181 seconds) 2020-03-31 08:53:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:36 UTC)" (scheduled at 2020-03-31 08:53:26.924497+00:00) 2020-03-31 08:53:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:53:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:36.924497+00:00 (in 9.999181 seconds) 2020-03-31 08:53:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:36 UTC)" (scheduled at 2020-03-31 08:53:26.924497+00:00) 2020-03-31 08:53:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 26, 926076), True, datetime.datetime(2020, 3, 31, 8, 53, 26, 926076), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:53:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 26, 926076), True, datetime.datetime(2020, 3, 31, 8, 53, 26, 926076), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:53:26,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:53:26,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:53:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:36 UTC)" executed successfully 2020-03-31 08:53:28,273 [518] [DEBUG] [app] Starting request: urn:request:b3a8a0cf-5b79-49ea-b66f-53ad58579ebd (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:28,273 [518] [DEBUG] [app] Starting request: urn:request:b3a8a0cf-5b79-49ea-b66f-53ad58579ebd (/health/instance) 2020-03-31 08:53:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:28,284 [544] [DEBUG] [app] Starting request: urn:request:0e14a0a5-435f-4d2e-9bdb-dcb8d1c56dfa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:28,284 [544] [DEBUG] [app] Starting request: urn:request:0e14a0a5-435f-4d2e-9bdb-dcb8d1c56dfa (/v1/_internal_ping) 2020-03-31 08:53:28,288 [544] [DEBUG] [app] Ending request: urn:request:0e14a0a5-435f-4d2e-9bdb-dcb8d1c56dfa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:28,288 [544] [DEBUG] [app] Ending request: urn:request:0e14a0a5-435f-4d2e-9bdb-dcb8d1c56dfa (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:53:28,289 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:28,289 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:53:28,289 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:28,290 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:28,289 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:53:28,290 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:28,295 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:28,295 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:28,301 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:28,301 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:28,302 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:28,302 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:28,304 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:28,304 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:28,318 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:28,318 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:28,323 [518] [DEBUG] [app] Starting request: urn:request:a9175069-262b-43d6-9f13-e88678379b62 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:28,323 [518] [DEBUG] [app] Starting request: urn:request:a9175069-262b-43d6-9f13-e88678379b62 (/_internal_ping) 2020-03-31 08:53:28,324 [518] [DEBUG] [app] Ending request: urn:request:a9175069-262b-43d6-9f13-e88678379b62 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:28,324 [518] [DEBUG] [app] Ending request: urn:request:a9175069-262b-43d6-9f13-e88678379b62 (/_internal_ping) 2020-03-31 08:53:28,325 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:53:28,325 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:53:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:28,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:28,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:53:28,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:28,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:28,340 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:28,340 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:28,341 [537] [DEBUG] [app] Starting request: urn:request:d99882b5-d215-48e4-a571-7be5fa1eb0e2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:28,341 [537] [DEBUG] [app] Starting request: urn:request:d99882b5-d215-48e4-a571-7be5fa1eb0e2 (/c1/_internal_ping) 2020-03-31 08:53:28,342 [537] [DEBUG] [app] Ending request: urn:request:d99882b5-d215-48e4-a571-7be5fa1eb0e2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:28,342 [537] [DEBUG] [app] Ending request: urn:request:d99882b5-d215-48e4-a571-7be5fa1eb0e2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:28,342 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:28,342 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:28,342 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:53:28,342 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:28,344 [518] [DEBUG] [app] Ending request: urn:request:b3a8a0cf-5b79-49ea-b66f-53ad58579ebd (/health/instance) 2020-03-31 08:53:28,344 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:28,344 [518] [DEBUG] [app] Ending request: urn:request:b3a8a0cf-5b79-49ea-b66f-53ad58579ebd (/health/instance) 2020-03-31 08:53:28,344 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:28,345 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 08:53:28,345 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 08:53:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:53:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:00.337699+00:00 (in 29.998994 seconds) 2020-03-31 08:53:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:00 UTC)" (scheduled at 2020-03-31 08:53:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:53:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:00.337699+00:00 (in 29.998994 seconds) 2020-03-31 08:53:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:00 UTC)" (scheduled at 2020-03-31 08:53:30.337699+00:00) 2020-03-31 08:53:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 53, 30, 339522), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:53:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 53, 30, 339522), False, 50, 1, 0]) 2020-03-31 08:53:30,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:53:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:53:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:53:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:00 UTC)" executed successfully 2020-03-31 08:53:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:53:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:41.440707+00:00 (in 9.996564 seconds) 2020-03-31 08:53:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:41 UTC)" (scheduled at 2020-03-31 08:53:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:53:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:41.440707+00:00 (in 9.996564 seconds) 2020-03-31 08:53:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:41 UTC)" (scheduled at 2020-03-31 08:53:31.440707+00:00) 2020-03-31 08:53:31,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:53:31,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:53:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:53:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:53:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:53:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:41 UTC)" executed successfully 2020-03-31 08:53:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:53:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:03.330112+00:00 (in 29.998520 seconds) 2020-03-31 08:53:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:03 UTC)" (scheduled at 2020-03-31 08:53:33.330112+00:00) 2020-03-31 08:53:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:53:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:03.330112+00:00 (in 29.998520 seconds) 2020-03-31 08:53:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:03 UTC)" (scheduled at 2020-03-31 08:53:33.330112+00:00) 2020-03-31 08:53:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 33, 332590), True, datetime.datetime(2020, 3, 31, 8, 53, 33, 332590), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:53:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 33, 332590), True, datetime.datetime(2020, 3, 31, 8, 53, 33, 332590), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:53:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:53:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:43.539999+00:00 (in 10.172394 seconds) 2020-03-31 08:53:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:33 UTC)" (scheduled at 2020-03-31 08:53:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 08:53:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:43.539999+00:00 (in 10.172394 seconds) 2020-03-31 08:53:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:33 UTC)" (scheduled at 2020-03-31 08:53:33.366791+00:00) 2020-03-31 08:53:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:53:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 33, 368566), True, datetime.datetime(2020, 3, 31, 8, 53, 33, 368566), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:53:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 33, 368566), True, datetime.datetime(2020, 3, 31, 8, 53, 33, 368566), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:53:33,557 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:33,557 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 08:53:33,557 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:33,557 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:33,557 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:53:33,557 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:33 UTC)" executed successfully 2020-03-31 08:53:33,559 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:53:33,559 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:33,559 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:33,559 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:53:33,559 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:33,559 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:03 UTC)" executed successfully 2020-03-31 08:53:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:53:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:44.323352+00:00 (in 9.998690 seconds) 2020-03-31 08:53:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:44 UTC)" (scheduled at 2020-03-31 08:53:34.323352+00:00) 2020-03-31 08:53:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:53:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:44.323352+00:00 (in 9.998690 seconds) 2020-03-31 08:53:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:44 UTC)" (scheduled at 2020-03-31 08:53:34.323352+00:00) 2020-03-31 08:53:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 34, 325535), True, datetime.datetime(2020, 3, 31, 8, 53, 34, 325535), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:53:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 34, 325535), True, datetime.datetime(2020, 3, 31, 8, 53, 34, 325535), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:53:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:53:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:53:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:44 UTC)" executed successfully 2020-03-31 08:53:34,447 [518] [DEBUG] [app] Starting request: urn:request:086d004a-af05-4826-b408-69600c54c1c3 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:34,447 [518] [DEBUG] [app] Starting request: urn:request:086d004a-af05-4826-b408-69600c54c1c3 (/health/instance) 2020-03-31 08:53:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:34,458 [548] [DEBUG] [app] Starting request: urn:request:b1f0d30e-2353-426f-9906-f77d13fa52c7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:34,458 [548] [DEBUG] [app] Starting request: urn:request:b1f0d30e-2353-426f-9906-f77d13fa52c7 (/v1/_internal_ping) 2020-03-31 08:53:34,462 [548] [DEBUG] [app] Ending request: urn:request:b1f0d30e-2353-426f-9906-f77d13fa52c7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:34,462 [548] [DEBUG] [app] Ending request: urn:request:b1f0d30e-2353-426f-9906-f77d13fa52c7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:53:34,463 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:53:34,463 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:34,463 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 08:53:34,463 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:34,464 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:34,464 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:34,469 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:34,469 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:34,473 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:34,473 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:34,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:34,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:34,474 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:34,474 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:34,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:34,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:34,493 [518] [DEBUG] [app] Starting request: urn:request:b7b27679-d786-427e-b834-f1e17290a204 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:34,493 [518] [DEBUG] [app] Starting request: urn:request:b7b27679-d786-427e-b834-f1e17290a204 (/_internal_ping) 2020-03-31 08:53:34,494 [518] [DEBUG] [app] Ending request: urn:request:b7b27679-d786-427e-b834-f1e17290a204 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:34,494 [518] [DEBUG] [app] Ending request: urn:request:b7b27679-d786-427e-b834-f1e17290a204 (/_internal_ping) 2020-03-31 08:53:34,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:53:34,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:53:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:34,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:34,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:53:34,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:34,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:34,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:34,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:34,509 [535] [DEBUG] [app] Starting request: urn:request:75c47732-bff0-47a9-8cbc-1df0e2dac0fb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:34,509 [535] [DEBUG] [app] Starting request: urn:request:75c47732-bff0-47a9-8cbc-1df0e2dac0fb (/c1/_internal_ping) 2020-03-31 08:53:34,509 [535] [DEBUG] [app] Ending request: urn:request:75c47732-bff0-47a9-8cbc-1df0e2dac0fb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:34,509 [535] [DEBUG] [app] Ending request: urn:request:75c47732-bff0-47a9-8cbc-1df0e2dac0fb (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:34,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:34,510 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:34,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:53:34,510 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:34,511 [518] [DEBUG] [app] Ending request: urn:request:086d004a-af05-4826-b408-69600c54c1c3 (/health/instance) 2020-03-31 08:53:34,511 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:34,511 [518] [DEBUG] [app] Ending request: urn:request:086d004a-af05-4826-b408-69600c54c1c3 (/health/instance) 2020-03-31 08:53:34,511 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:34,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 08:53:34,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 08:53:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:53:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:38.010934+00:00 (in 1.085669 seconds) 2020-03-31 08:53:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:46 UTC)" (scheduled at 2020-03-31 08:53:36.924497+00:00) 2020-03-31 08:53:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:53:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:38.010934+00:00 (in 1.085669 seconds) 2020-03-31 08:53:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:46 UTC)" (scheduled at 2020-03-31 08:53:36.924497+00:00) 2020-03-31 08:53:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 36, 926087), True, datetime.datetime(2020, 3, 31, 8, 53, 36, 926087), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:53:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 36, 926087), True, datetime.datetime(2020, 3, 31, 8, 53, 36, 926087), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:53:36,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:53:36,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:36,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:53:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:36,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:46 UTC)" executed successfully 2020-03-31 08:53:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:53:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:46.924497+00:00 (in 8.912560 seconds) 2020-03-31 08:53:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:38 UTC)" (scheduled at 2020-03-31 08:53:38.010934+00:00) 2020-03-31 08:53:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:53:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:46.924497+00:00 (in 8.912560 seconds) 2020-03-31 08:53:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:38 UTC)" (scheduled at 2020-03-31 08:53:38.010934+00:00) 2020-03-31 08:53:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:38 UTC)" executed successfully 2020-03-31 08:53:38,273 [518] [DEBUG] [app] Starting request: urn:request:dab7870e-1ce6-4231-acec-a291bc7cff74 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:38,273 [518] [DEBUG] [app] Starting request: urn:request:dab7870e-1ce6-4231-acec-a291bc7cff74 (/health/instance) 2020-03-31 08:53:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:38,277 [552] [DEBUG] [app] Starting request: urn:request:4cdabdb4-ac1b-4f9c-9c6b-1fb39473d42e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:38,277 [552] [DEBUG] [app] Starting request: urn:request:4cdabdb4-ac1b-4f9c-9c6b-1fb39473d42e (/v1/_internal_ping) 2020-03-31 08:53:38,279 [552] [DEBUG] [app] Ending request: urn:request:4cdabdb4-ac1b-4f9c-9c6b-1fb39473d42e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:38,279 [552] [DEBUG] [app] Ending request: urn:request:4cdabdb4-ac1b-4f9c-9c6b-1fb39473d42e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:53:38,280 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:38,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:53:38,280 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:38,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:38,283 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:38,283 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:38,284 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:38,284 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:38,285 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:38,285 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:38,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:38,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:38,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:38,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:38,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:38,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:38,441 [518] [DEBUG] [app] Starting request: urn:request:646c7938-83a3-4db9-bc71-2fac421f5f1d (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:38,441 [518] [DEBUG] [app] Starting request: urn:request:646c7938-83a3-4db9-bc71-2fac421f5f1d (/_internal_ping) 2020-03-31 08:53:38,442 [518] [DEBUG] [app] Ending request: urn:request:646c7938-83a3-4db9-bc71-2fac421f5f1d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.135 159 0.135) gunicorn-web stdout | 2020-03-31 08:53:38,442 [518] [DEBUG] [app] Ending request: urn:request:646c7938-83a3-4db9-bc71-2fac421f5f1d (/_internal_ping) 2020-03-31 08:53:38,443 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:38,444 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.135 159 0.135) gunicorn-web stdout | 2020-03-31 08:53:38,443 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:38,444 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:38,448 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:38,448 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:38,448 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:38,448 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:38,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:38,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:53:38,453 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:38,453 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:38,457 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:38,457 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:38,459 [535] [DEBUG] [app] Starting request: urn:request:473a3ba5-8da1-4ffb-8893-0e50e4c18613 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:38,459 [535] [DEBUG] [app] Starting request: urn:request:473a3ba5-8da1-4ffb-8893-0e50e4c18613 (/c1/_internal_ping) 2020-03-31 08:53:38,460 [535] [DEBUG] [app] Ending request: urn:request:473a3ba5-8da1-4ffb-8893-0e50e4c18613 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:38,460 [535] [DEBUG] [app] Ending request: urn:request:473a3ba5-8da1-4ffb-8893-0e50e4c18613 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:38,461 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:38,460 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:38,461 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:53:38,460 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:38,462 [518] [DEBUG] [app] Ending request: urn:request:dab7870e-1ce6-4231-acec-a291bc7cff74 (/health/instance) 2020-03-31 08:53:38,463 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:38,462 [518] [DEBUG] [app] Ending request: urn:request:dab7870e-1ce6-4231-acec-a291bc7cff74 (/health/instance) 2020-03-31 08:53:38,463 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:38,463 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.192 49 0.192) gunicorn-web stdout | 2020-03-31 08:53:38,463 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.192 49 0.192) 2020-03-31 08:53:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:53:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:51.440707+00:00 (in 9.998628 seconds) 2020-03-31 08:53:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:51 UTC)" (scheduled at 2020-03-31 08:53:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:53:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:51.440707+00:00 (in 9.998628 seconds) 2020-03-31 08:53:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:51 UTC)" (scheduled at 2020-03-31 08:53:41.440707+00:00) 2020-03-31 08:53:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:53:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:53:41,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:53:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:53:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:53:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:51 UTC)" executed successfully 2020-03-31 08:53:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:53:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:33.366791+00:00 (in 49.825727 seconds) 2020-03-31 08:53:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:43 UTC)" (scheduled at 2020-03-31 08:53:43.539999+00:00) 2020-03-31 08:53:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:53:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:33.366791+00:00 (in 49.825727 seconds) 2020-03-31 08:53:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:43 UTC)" (scheduled at 2020-03-31 08:53:43.539999+00:00) 2020-03-31 08:53:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:43 UTC)" executed successfully 2020-03-31 08:53:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:53:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:54 UTC)" (scheduled at 2020-03-31 08:53:44.323352+00:00) 2020-03-31 08:53:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:49.498966+00:00 (in 5.174688 seconds) 2020-03-31 08:53:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:53:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:54 UTC)" (scheduled at 2020-03-31 08:53:44.323352+00:00) 2020-03-31 08:53:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:49.498966+00:00 (in 5.174688 seconds) 2020-03-31 08:53:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 44, 325360), True, datetime.datetime(2020, 3, 31, 8, 53, 44, 325360), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:53:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 44, 325360), True, datetime.datetime(2020, 3, 31, 8, 53, 44, 325360), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:53:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:53:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:53:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:54 UTC)" executed successfully 2020-03-31 08:53:44,447 [518] [DEBUG] [app] Starting request: urn:request:da0dfbf9-89dd-4b2d-9be8-cc067a0bb167 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:44,447 [518] [DEBUG] [app] Starting request: urn:request:da0dfbf9-89dd-4b2d-9be8-cc067a0bb167 (/health/instance) 2020-03-31 08:53:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:44,453 [553] [DEBUG] [app] Starting request: urn:request:eb80bd06-d575-4f9b-a503-9bbbb33ac9a4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:44,453 [553] [DEBUG] [app] Starting request: urn:request:eb80bd06-d575-4f9b-a503-9bbbb33ac9a4 (/v1/_internal_ping) 2020-03-31 08:53:44,455 [553] [DEBUG] [app] Ending request: urn:request:eb80bd06-d575-4f9b-a503-9bbbb33ac9a4 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:44,455 [553] [DEBUG] [app] Ending request: urn:request:eb80bd06-d575-4f9b-a503-9bbbb33ac9a4 (/v1/_internal_ping) 2020-03-31 08:53:44,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:44,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:44,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:44,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:44,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:44,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:44,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:44,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:44,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:44,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:44,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:44,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:44,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:44,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:44,488 [518] [DEBUG] [app] Starting request: urn:request:9685c318-e53b-48ee-a35b-848ddecdbc8c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:44,488 [518] [DEBUG] [app] Starting request: urn:request:9685c318-e53b-48ee-a35b-848ddecdbc8c (/_internal_ping) 2020-03-31 08:53:44,489 [518] [DEBUG] [app] Ending request: urn:request:9685c318-e53b-48ee-a35b-848ddecdbc8c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:44,489 [518] [DEBUG] [app] Ending request: urn:request:9685c318-e53b-48ee-a35b-848ddecdbc8c (/_internal_ping) 2020-03-31 08:53:44,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:53:44,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:53:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:53:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:44,503 [543] [DEBUG] [app] Starting request: urn:request:a1d8d98f-7983-4759-b556-83e9b9f1ab49 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:44,503 [543] [DEBUG] [app] Starting request: urn:request:a1d8d98f-7983-4759-b556-83e9b9f1ab49 (/c1/_internal_ping) 2020-03-31 08:53:44,504 [543] [DEBUG] [app] Ending request: urn:request:a1d8d98f-7983-4759-b556-83e9b9f1ab49 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:44,504 [543] [DEBUG] [app] Ending request: urn:request:a1d8d98f-7983-4759-b556-83e9b9f1ab49 (/c1/_internal_ping) 2020-03-31 08:53:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:44,505 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:44,506 [518] [DEBUG] [app] Ending request: urn:request:da0dfbf9-89dd-4b2d-9be8-cc067a0bb167 (/health/instance) 2020-03-31 08:53:44,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:53:44,505 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 08:53:44,506 [518] [DEBUG] [app] Ending request: urn:request:da0dfbf9-89dd-4b2d-9be8-cc067a0bb167 (/health/instance) 2020-03-31 08:53:44,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 08:53:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:53:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:15.064514+00:00 (in 29.998996 seconds) 2020-03-31 08:53:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:15 UTC)" (scheduled at 2020-03-31 08:53:45.064514+00:00) gcworker stdout | 2020-03-31 08:53:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:15.064514+00:00 (in 29.998996 seconds) 2020-03-31 08:53:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:15 UTC)" (scheduled at 2020-03-31 08:53:45.064514+00:00) 2020-03-31 08:53:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435225, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:53:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435225, 1209600, 500, 1, 0]) 2020-03-31 08:53:45,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:53:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:53:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:53:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:15 UTC)" executed successfully 2020-03-31 08:53:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:53:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:56.924497+00:00 (in 9.998965 seconds) 2020-03-31 08:53:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:56 UTC)" (scheduled at 2020-03-31 08:53:46.924497+00:00) 2020-03-31 08:53:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:53:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:56.924497+00:00 (in 9.998965 seconds) 2020-03-31 08:53:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:56 UTC)" (scheduled at 2020-03-31 08:53:46.924497+00:00) 2020-03-31 08:53:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 46, 926454), True, datetime.datetime(2020, 3, 31, 8, 53, 46, 926454), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:53:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 46, 926454), True, datetime.datetime(2020, 3, 31, 8, 53, 46, 926454), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:53:46,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:53:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:53:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:53:56 UTC)" executed successfully 2020-03-31 08:53:48,273 [518] [DEBUG] [app] Starting request: urn:request:350e9c8d-67a9-48db-8301-595963062a53 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:48,273 [518] [DEBUG] [app] Starting request: urn:request:350e9c8d-67a9-48db-8301-595963062a53 (/health/instance) 2020-03-31 08:53:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:48,278 [547] [DEBUG] [app] Starting request: urn:request:23bb0a58-d43b-4e72-b4b4-594a3f47200a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:48,278 [547] [DEBUG] [app] Starting request: urn:request:23bb0a58-d43b-4e72-b4b4-594a3f47200a (/v1/_internal_ping) 2020-03-31 08:53:48,280 [547] [DEBUG] [app] Ending request: urn:request:23bb0a58-d43b-4e72-b4b4-594a3f47200a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:48,280 [547] [DEBUG] [app] Ending request: urn:request:23bb0a58-d43b-4e72-b4b4-594a3f47200a (/v1/_internal_ping) 2020-03-31 08:53:48,281 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:48,281 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:53:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:48,284 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:48,284 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:48,285 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:48,285 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:48,286 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:48,286 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:48,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:48,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:48,313 [518] [DEBUG] [app] Starting request: urn:request:adc59bad-e3a2-4a78-9496-c58718129316 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:48,313 [518] [DEBUG] [app] Starting request: urn:request:adc59bad-e3a2-4a78-9496-c58718129316 (/_internal_ping) 2020-03-31 08:53:48,314 [518] [DEBUG] [app] Ending request: urn:request:adc59bad-e3a2-4a78-9496-c58718129316 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:48,314 [518] [DEBUG] [app] Ending request: urn:request:adc59bad-e3a2-4a78-9496-c58718129316 (/_internal_ping) 2020-03-31 08:53:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:53:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:53:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:53:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:48,327 [543] [DEBUG] [app] Starting request: urn:request:849362bd-96d9-4f9d-97f7-951dd2219801 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:48,327 [543] [DEBUG] [app] Starting request: urn:request:849362bd-96d9-4f9d-97f7-951dd2219801 (/c1/_internal_ping) 2020-03-31 08:53:48,328 [543] [DEBUG] [app] Ending request: urn:request:849362bd-96d9-4f9d-97f7-951dd2219801 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:48,328 [543] [DEBUG] [app] Ending request: urn:request:849362bd-96d9-4f9d-97f7-951dd2219801 (/c1/_internal_ping) 2020-03-31 08:53:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:48,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:53:48,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:48,330 [518] [DEBUG] [app] Ending request: urn:request:350e9c8d-67a9-48db-8301-595963062a53 (/health/instance) 2020-03-31 08:53:48,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:48,330 [518] [DEBUG] [app] Ending request: urn:request:350e9c8d-67a9-48db-8301-595963062a53 (/health/instance) 2020-03-31 08:53:48,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:53:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:53:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:53:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:54.323352+00:00 (in 4.822819 seconds) 2020-03-31 08:53:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:49 UTC)" (scheduled at 2020-03-31 08:53:49.498966+00:00) 2020-03-31 08:53:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:53:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:54.323352+00:00 (in 4.822819 seconds) 2020-03-31 08:53:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:49 UTC)" (scheduled at 2020-03-31 08:53:49.498966+00:00) 2020-03-31 08:53:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:49 UTC)" executed successfully 2020-03-31 08:53:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:53:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:59.560326+00:00 (in 8.448488 seconds) 2020-03-31 08:53:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:51 UTC)" (scheduled at 2020-03-31 08:53:51.110610+00:00) namespacegcworker stdout | 2020-03-31 08:53:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:53:59.560326+00:00 (in 8.448488 seconds) 2020-03-31 08:53:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:51 UTC)" (scheduled at 2020-03-31 08:53:51.110610+00:00) 2020-03-31 08:53:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:53:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 51, 112943), True, datetime.datetime(2020, 3, 31, 8, 53, 51, 112943), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:53:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 51, 112943), True, datetime.datetime(2020, 3, 31, 8, 53, 51, 112943), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:53:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:53:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:53:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:51 UTC)" executed successfully 2020-03-31 08:53:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:53:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:01.440707+00:00 (in 9.998981 seconds) 2020-03-31 08:53:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:01 UTC)" (scheduled at 2020-03-31 08:53:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:53:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:01.440707+00:00 (in 9.998981 seconds) 2020-03-31 08:53:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:01 UTC)" (scheduled at 2020-03-31 08:53:51.440707+00:00) 2020-03-31 08:53:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:53:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:53:51,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:53:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:53:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:53:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:01 UTC)" executed successfully 2020-03-31 08:53:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:53:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:52 UTC)" (scheduled at 2020-03-31 08:53:52.113221+00:00) securityworker stdout | 2020-03-31 08:53:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:53:52 UTC)" (scheduled at 2020-03-31 08:53:52.113221+00:00) 2020-03-31 08:53:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:53:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:22.113221+00:00 (in 29.999009 seconds) securityworker stdout | 2020-03-31 08:53:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:53:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:22.113221+00:00 (in 29.999009 seconds) 2020-03-31 08:53:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:53:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:53:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:53:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:53:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:53:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:22 UTC)" executed successfully 2020-03-31 08:53:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:53:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:04.323352+00:00 (in 9.999143 seconds) 2020-03-31 08:53:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:04 UTC)" (scheduled at 2020-03-31 08:53:54.323352+00:00) 2020-03-31 08:53:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:53:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:04.323352+00:00 (in 9.999143 seconds) 2020-03-31 08:53:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:04 UTC)" (scheduled at 2020-03-31 08:53:54.323352+00:00) 2020-03-31 08:53:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 54, 325134), True, datetime.datetime(2020, 3, 31, 8, 53, 54, 325134), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:53:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 54, 325134), True, datetime.datetime(2020, 3, 31, 8, 53, 54, 325134), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:53:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:53:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:53:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:04 UTC)" executed successfully 2020-03-31 08:53:54,446 [518] [DEBUG] [app] Starting request: urn:request:5e2bda0d-6ac7-4f1a-bbba-f4a15db96908 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:54,446 [518] [DEBUG] [app] Starting request: urn:request:5e2bda0d-6ac7-4f1a-bbba-f4a15db96908 (/health/instance) 2020-03-31 08:53:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:54,453 [549] [DEBUG] [app] Starting request: urn:request:632801c5-c9da-4648-bb72-28e0e47013ce (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:54,453 [549] [DEBUG] [app] Starting request: urn:request:632801c5-c9da-4648-bb72-28e0e47013ce (/v1/_internal_ping) 2020-03-31 08:53:54,454 [549] [DEBUG] [app] Ending request: urn:request:632801c5-c9da-4648-bb72-28e0e47013ce (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:54,454 [549] [DEBUG] [app] Ending request: urn:request:632801c5-c9da-4648-bb72-28e0e47013ce (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:54,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:53:54,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:53:54,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:53:54,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:54,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:54,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:53:54,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:54,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:54,485 [527] [DEBUG] [app] Starting request: urn:request:0d3b411a-1964-4284-848e-22d40ab986bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:54,485 [527] [DEBUG] [app] Starting request: urn:request:0d3b411a-1964-4284-848e-22d40ab986bc (/_internal_ping) 2020-03-31 08:53:54,486 [527] [DEBUG] [app] Ending request: urn:request:0d3b411a-1964-4284-848e-22d40ab986bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:54,486 [527] [DEBUG] [app] Ending request: urn:request:0d3b411a-1964-4284-848e-22d40ab986bc (/_internal_ping) 2020-03-31 08:53:54,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:53:54,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:53:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:53:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:53:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:54,501 [543] [DEBUG] [app] Starting request: urn:request:22e0356b-c413-46ab-bbb0-898e8710a787 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:54,501 [543] [DEBUG] [app] Starting request: urn:request:22e0356b-c413-46ab-bbb0-898e8710a787 (/c1/_internal_ping) 2020-03-31 08:53:54,502 [543] [DEBUG] [app] Ending request: urn:request:22e0356b-c413-46ab-bbb0-898e8710a787 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:54,502 [543] [DEBUG] [app] Ending request: urn:request:22e0356b-c413-46ab-bbb0-898e8710a787 (/c1/_internal_ping) 2020-03-31 08:53:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:54,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:53:54,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:54,504 [518] [DEBUG] [app] Ending request: urn:request:5e2bda0d-6ac7-4f1a-bbba-f4a15db96908 (/health/instance) 2020-03-31 08:53:54,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:54,504 [518] [DEBUG] [app] Ending request: urn:request:5e2bda0d-6ac7-4f1a-bbba-f4a15db96908 (/health/instance) 2020-03-31 08:53:54,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:53:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:53:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:06.924497+00:00 (in 9.998837 seconds) 2020-03-31 08:53:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:06 UTC)" (scheduled at 2020-03-31 08:53:56.924497+00:00) 2020-03-31 08:53:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:53:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:53:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:06.924497+00:00 (in 9.998837 seconds) 2020-03-31 08:53:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:06 UTC)" (scheduled at 2020-03-31 08:53:56.924497+00:00) 2020-03-31 08:53:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:53:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 56, 926390), True, datetime.datetime(2020, 3, 31, 8, 53, 56, 926390), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:53:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 53, 56, 926390), True, datetime.datetime(2020, 3, 31, 8, 53, 56, 926390), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:53:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:53:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:53:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:53:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:06 UTC)" executed successfully 2020-03-31 08:53:58,273 [527] [DEBUG] [app] Starting request: urn:request:9d978c53-63f2-45a9-9bf9-6a16d21ebff3 (/health/instance) gunicorn-web stdout | 2020-03-31 08:53:58,273 [527] [DEBUG] [app] Starting request: urn:request:9d978c53-63f2-45a9-9bf9-6a16d21ebff3 (/health/instance) 2020-03-31 08:53:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:58,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:58,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:58,280 [556] [DEBUG] [app] Starting request: urn:request:97c9da06-8bd6-4e60-840d-7ba71ffc119c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:58,280 [556] [DEBUG] [app] Starting request: urn:request:97c9da06-8bd6-4e60-840d-7ba71ffc119c (/v1/_internal_ping) 2020-03-31 08:53:58,281 [556] [DEBUG] [app] Ending request: urn:request:97c9da06-8bd6-4e60-840d-7ba71ffc119c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:53:58,281 [556] [DEBUG] [app] Ending request: urn:request:97c9da06-8bd6-4e60-840d-7ba71ffc119c (/v1/_internal_ping) 2020-03-31 08:53:58,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:53:58,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:53:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:53:58,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:58,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 08:53:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:53:58,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:53:58,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:53:58,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:53:58,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:53:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:53:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:53:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:53:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:53:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:58,313 [527] [DEBUG] [app] Starting request: urn:request:6b84319c-98cb-4663-aeb7-9ff936ea6747 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:58,313 [527] [DEBUG] [app] Starting request: urn:request:6b84319c-98cb-4663-aeb7-9ff936ea6747 (/_internal_ping) 2020-03-31 08:53:58,314 [527] [DEBUG] [app] Ending request: urn:request:6b84319c-98cb-4663-aeb7-9ff936ea6747 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:53:58,314 [527] [DEBUG] [app] Ending request: urn:request:6b84319c-98cb-4663-aeb7-9ff936ea6747 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:53:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:58,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:53:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:53:58,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:53:58,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:53:58,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:53:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:53:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:53:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:53:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:53:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:53:58,327 [543] [DEBUG] [app] Starting request: urn:request:055696e9-96a5-402c-9ddc-27cc79363026 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:58,327 [543] [DEBUG] [app] Starting request: urn:request:055696e9-96a5-402c-9ddc-27cc79363026 (/c1/_internal_ping) 2020-03-31 08:53:58,328 [543] [DEBUG] [app] Ending request: urn:request:055696e9-96a5-402c-9ddc-27cc79363026 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:53:58,328 [543] [DEBUG] [app] Ending request: urn:request:055696e9-96a5-402c-9ddc-27cc79363026 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:58,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:53:58,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:53:58,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:53:58,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:53:58,330 [527] [DEBUG] [app] Ending request: urn:request:9d978c53-63f2-45a9-9bf9-6a16d21ebff3 (/health/instance) 2020-03-31 08:53:58,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:53:58,330 [527] [DEBUG] [app] Ending request: urn:request:9d978c53-63f2-45a9-9bf9-6a16d21ebff3 (/health/instance) 2020-03-31 08:53:58,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:53:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:53:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:53:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:53:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:53:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:53:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:53:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:51.110610+00:00 (in 51.549014 seconds) 2020-03-31 08:53:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:59 UTC)" (scheduled at 2020-03-31 08:53:59.560326+00:00) 2020-03-31 08:53:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:53:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:51.110610+00:00 (in 51.549014 seconds) 2020-03-31 08:53:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:59 UTC)" (scheduled at 2020-03-31 08:53:59.560326+00:00) 2020-03-31 08:53:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:53:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:54:59 UTC)" executed successfully 2020-03-31 08:54:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:54:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:30.337699+00:00 (in 29.999208 seconds) 2020-03-31 08:54:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:30 UTC)" (scheduled at 2020-03-31 08:54:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:54:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:30.337699+00:00 (in 29.999208 seconds) 2020-03-31 08:54:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:30 UTC)" (scheduled at 2020-03-31 08:54:00.337699+00:00) 2020-03-31 08:54:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 54, 0, 339461), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:54:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 54, 0, 339461), False, 50, 1, 0]) 2020-03-31 08:54:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:54:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:54:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:54:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:30 UTC)" executed successfully 2020-03-31 08:54:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:54:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:11.440707+00:00 (in 9.999061 seconds) 2020-03-31 08:54:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:11 UTC)" (scheduled at 2020-03-31 08:54:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:54:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:11.440707+00:00 (in 9.999061 seconds) 2020-03-31 08:54:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:11 UTC)" (scheduled at 2020-03-31 08:54:01.440707+00:00) 2020-03-31 08:54:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:54:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:54:01,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:54:01,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:54:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:54:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:11 UTC)" executed successfully 2020-03-31 08:54:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:54:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:23.410371+00:00 (in 20.079154 seconds) 2020-03-31 08:54:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:33 UTC)" (scheduled at 2020-03-31 08:54:03.330112+00:00) 2020-03-31 08:54:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:54:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:23.410371+00:00 (in 20.079154 seconds) 2020-03-31 08:54:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:33 UTC)" (scheduled at 2020-03-31 08:54:03.330112+00:00) 2020-03-31 08:54:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 3, 332352), True, datetime.datetime(2020, 3, 31, 8, 54, 3, 332352), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:54:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 3, 332352), True, datetime.datetime(2020, 3, 31, 8, 54, 3, 332352), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:54:03,353 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:54:03,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:54:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:33 UTC)" executed successfully 2020-03-31 08:54:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:54:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:14.323352+00:00 (in 9.998931 seconds) 2020-03-31 08:54:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:14 UTC)" (scheduled at 2020-03-31 08:54:04.323352+00:00) 2020-03-31 08:54:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:54:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:14.323352+00:00 (in 9.998931 seconds) 2020-03-31 08:54:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:14 UTC)" (scheduled at 2020-03-31 08:54:04.323352+00:00) 2020-03-31 08:54:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 4, 325425), True, datetime.datetime(2020, 3, 31, 8, 54, 4, 325425), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:54:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 4, 325425), True, datetime.datetime(2020, 3, 31, 8, 54, 4, 325425), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:54:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:54:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:54:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:14 UTC)" executed successfully 2020-03-31 08:54:04,446 [518] [DEBUG] [app] Starting request: urn:request:bfbc7d88-1078-4bfa-a404-6dc2db3c8bb4 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:04,446 [518] [DEBUG] [app] Starting request: urn:request:bfbc7d88-1078-4bfa-a404-6dc2db3c8bb4 (/health/instance) 2020-03-31 08:54:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:04,453 [553] [DEBUG] [app] Starting request: urn:request:725cc4ed-0453-4ca7-af2c-17544318b4b7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:04,453 [553] [DEBUG] [app] Starting request: urn:request:725cc4ed-0453-4ca7-af2c-17544318b4b7 (/v1/_internal_ping) 2020-03-31 08:54:04,454 [553] [DEBUG] [app] Ending request: urn:request:725cc4ed-0453-4ca7-af2c-17544318b4b7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:04,454 [553] [DEBUG] [app] Ending request: urn:request:725cc4ed-0453-4ca7-af2c-17544318b4b7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:54:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:04,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:04,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:04,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:04,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:04,491 [518] [DEBUG] [app] Starting request: urn:request:a082a0e5-189e-4edf-a6ff-9782f40ecb6a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:04,491 [518] [DEBUG] [app] Starting request: urn:request:a082a0e5-189e-4edf-a6ff-9782f40ecb6a (/_internal_ping) 2020-03-31 08:54:04,492 [518] [DEBUG] [app] Ending request: urn:request:a082a0e5-189e-4edf-a6ff-9782f40ecb6a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:04,492 [518] [DEBUG] [app] Ending request: urn:request:a082a0e5-189e-4edf-a6ff-9782f40ecb6a (/_internal_ping) 2020-03-31 08:54:04,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:54:04,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:54:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:54:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:04,505 [535] [DEBUG] [app] Starting request: urn:request:08a6d78a-51b6-4c6e-a7a4-01a06dd90d85 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:04,505 [535] [DEBUG] [app] Starting request: urn:request:08a6d78a-51b6-4c6e-a7a4-01a06dd90d85 (/c1/_internal_ping) 2020-03-31 08:54:04,506 [535] [DEBUG] [app] Ending request: urn:request:08a6d78a-51b6-4c6e-a7a4-01a06dd90d85 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:04,506 [535] [DEBUG] [app] Ending request: urn:request:08a6d78a-51b6-4c6e-a7a4-01a06dd90d85 (/c1/_internal_ping) 2020-03-31 08:54:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:04,507 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:04,508 [518] [DEBUG] [app] Ending request: urn:request:bfbc7d88-1078-4bfa-a404-6dc2db3c8bb4 (/health/instance) 2020-03-31 08:54:04,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:54:04,507 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:54:04,508 [518] [DEBUG] [app] Ending request: urn:request:bfbc7d88-1078-4bfa-a404-6dc2db3c8bb4 (/health/instance) 2020-03-31 08:54:04,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:54:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:54:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:54:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:16.924497+00:00 (in 9.999207 seconds) 2020-03-31 08:54:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:16 UTC)" (scheduled at 2020-03-31 08:54:06.924497+00:00) 2020-03-31 08:54:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:54:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:16.924497+00:00 (in 9.999207 seconds) 2020-03-31 08:54:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:16 UTC)" (scheduled at 2020-03-31 08:54:06.924497+00:00) 2020-03-31 08:54:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 6, 925923), True, datetime.datetime(2020, 3, 31, 8, 54, 6, 925923), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:54:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 6, 925923), True, datetime.datetime(2020, 3, 31, 8, 54, 6, 925923), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:54:06,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:54:06,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:54:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:16 UTC)" executed successfully 2020-03-31 08:54:08,273 [526] [DEBUG] [app] Starting request: urn:request:ea6c6288-c058-49df-9d89-2ca39e920a04 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:08,273 [526] [DEBUG] [app] Starting request: urn:request:ea6c6288-c058-49df-9d89-2ca39e920a04 (/health/instance) 2020-03-31 08:54:08,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:08,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:08,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:08,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:08,280 [538] [DEBUG] [app] Starting request: urn:request:81d09c72-caa9-4098-bb44-170bb06d2db4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:08,280 [538] [DEBUG] [app] Starting request: urn:request:81d09c72-caa9-4098-bb44-170bb06d2db4 (/v1/_internal_ping) 2020-03-31 08:54:08,281 [538] [DEBUG] [app] Ending request: urn:request:81d09c72-caa9-4098-bb44-170bb06d2db4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:08,281 [538] [DEBUG] [app] Ending request: urn:request:81d09c72-caa9-4098-bb44-170bb06d2db4 (/v1/_internal_ping) 2020-03-31 08:54:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:54:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:08,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 08:54:08,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:08,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:08,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:08,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:08,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:08,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:08,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:08,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:08,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:08,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:08,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:08,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:08,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:08,314 [526] [DEBUG] [app] Starting request: urn:request:9676be8b-23a2-4f72-9761-d849d1ada5aa (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:08,314 [526] [DEBUG] [app] Starting request: urn:request:9676be8b-23a2-4f72-9761-d849d1ada5aa (/_internal_ping) 2020-03-31 08:54:08,315 [526] [DEBUG] [app] Ending request: urn:request:9676be8b-23a2-4f72-9761-d849d1ada5aa (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:08,315 [526] [DEBUG] [app] Ending request: urn:request:9676be8b-23a2-4f72-9761-d849d1ada5aa (/_internal_ping) 2020-03-31 08:54:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:54:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:08,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:54:08,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:08,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:08,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:08,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:54:08,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:54:08,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:08,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:54:08,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:08,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:08,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:08,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:08,328 [543] [DEBUG] [app] Starting request: urn:request:e71387de-74e3-490a-adc4-3800d72aba1f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:08,328 [543] [DEBUG] [app] Starting request: urn:request:e71387de-74e3-490a-adc4-3800d72aba1f (/c1/_internal_ping) 2020-03-31 08:54:08,329 [543] [DEBUG] [app] Ending request: urn:request:e71387de-74e3-490a-adc4-3800d72aba1f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:08,329 [543] [DEBUG] [app] Ending request: urn:request:e71387de-74e3-490a-adc4-3800d72aba1f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:08,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:08,330 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:54:08,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:08,330 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:08,331 [526] [DEBUG] [app] Ending request: urn:request:ea6c6288-c058-49df-9d89-2ca39e920a04 (/health/instance) 2020-03-31 08:54:08,331 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:08,331 [526] [DEBUG] [app] Ending request: urn:request:ea6c6288-c058-49df-9d89-2ca39e920a04 (/health/instance) 2020-03-31 08:54:08,331 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:08,332 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:54:08,332 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 08:54:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:54:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:21.440707+00:00 (in 9.998969 seconds) 2020-03-31 08:54:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:21 UTC)" (scheduled at 2020-03-31 08:54:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:54:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:21.440707+00:00 (in 9.998969 seconds) 2020-03-31 08:54:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:21 UTC)" (scheduled at 2020-03-31 08:54:11.440707+00:00) 2020-03-31 08:54:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:54:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:54:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:54:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:54:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:54:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:21 UTC)" executed successfully 2020-03-31 08:54:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:54:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:24 UTC)" (scheduled at 2020-03-31 08:54:14.323352+00:00) 2020-03-31 08:54:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:24.323352+00:00 (in 9.998908 seconds) 2020-03-31 08:54:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:54:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:24 UTC)" (scheduled at 2020-03-31 08:54:14.323352+00:00) 2020-03-31 08:54:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:24.323352+00:00 (in 9.998908 seconds) 2020-03-31 08:54:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 14, 325603), True, datetime.datetime(2020, 3, 31, 8, 54, 14, 325603), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:54:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 14, 325603), True, datetime.datetime(2020, 3, 31, 8, 54, 14, 325603), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:54:14,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:54:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:54:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:24 UTC)" executed successfully 2020-03-31 08:54:14,446 [527] [DEBUG] [app] Starting request: urn:request:ac7a7485-806a-4cf9-a2e8-e093d05fb010 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:14,446 [527] [DEBUG] [app] Starting request: urn:request:ac7a7485-806a-4cf9-a2e8-e093d05fb010 (/health/instance) 2020-03-31 08:54:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:14,453 [553] [DEBUG] [app] Starting request: urn:request:2e1aeaf2-8aa4-4f2c-92e7-425292f299fd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:14,453 [553] [DEBUG] [app] Starting request: urn:request:2e1aeaf2-8aa4-4f2c-92e7-425292f299fd (/v1/_internal_ping) 2020-03-31 08:54:14,454 [553] [DEBUG] [app] Ending request: urn:request:2e1aeaf2-8aa4-4f2c-92e7-425292f299fd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:14,454 [553] [DEBUG] [app] Ending request: urn:request:2e1aeaf2-8aa4-4f2c-92e7-425292f299fd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:54:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:14,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:14,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:14,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:14,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:14,485 [527] [DEBUG] [app] Starting request: urn:request:b6c51232-fd8b-4d3c-a18f-9aded5d83750 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:14,485 [527] [DEBUG] [app] Starting request: urn:request:b6c51232-fd8b-4d3c-a18f-9aded5d83750 (/_internal_ping) 2020-03-31 08:54:14,486 [527] [DEBUG] [app] Ending request: urn:request:b6c51232-fd8b-4d3c-a18f-9aded5d83750 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:14,486 [527] [DEBUG] [app] Ending request: urn:request:b6c51232-fd8b-4d3c-a18f-9aded5d83750 (/_internal_ping) 2020-03-31 08:54:14,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:14,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:14,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:14,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:14,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:14,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:14,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:14,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:14,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:14,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:54:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:14,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:14,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:14,500 [537] [DEBUG] [app] Starting request: urn:request:00ed3dd8-7869-4232-a6f8-e7876db94ee0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:14,500 [537] [DEBUG] [app] Starting request: urn:request:00ed3dd8-7869-4232-a6f8-e7876db94ee0 (/c1/_internal_ping) 2020-03-31 08:54:14,501 [537] [DEBUG] [app] Ending request: urn:request:00ed3dd8-7869-4232-a6f8-e7876db94ee0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:14,501 [537] [DEBUG] [app] Ending request: urn:request:00ed3dd8-7869-4232-a6f8-e7876db94ee0 (/c1/_internal_ping) 2020-03-31 08:54:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:14,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:54:14,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:14,503 [527] [DEBUG] [app] Ending request: urn:request:ac7a7485-806a-4cf9-a2e8-e093d05fb010 (/health/instance) 2020-03-31 08:54:14,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:14,503 [527] [DEBUG] [app] Ending request: urn:request:ac7a7485-806a-4cf9-a2e8-e093d05fb010 (/health/instance) 2020-03-31 08:54:14,503 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:14,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 08:54:14,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 08:54:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:54:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:45.064514+00:00 (in 29.999009 seconds) 2020-03-31 08:54:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:45 UTC)" (scheduled at 2020-03-31 08:54:15.064514+00:00) gcworker stdout | 2020-03-31 08:54:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:45.064514+00:00 (in 29.999009 seconds) 2020-03-31 08:54:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:45 UTC)" (scheduled at 2020-03-31 08:54:15.064514+00:00) 2020-03-31 08:54:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435255, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:54:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435255, 1209600, 500, 1, 0]) 2020-03-31 08:54:15,093 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:54:15,093 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:54:15,093 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:15,093 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:54:15,093 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:15,093 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:45 UTC)" executed successfully 2020-03-31 08:54:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:26.924497+00:00 (in 9.999236 seconds) 2020-03-31 08:54:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:26 UTC)" (scheduled at 2020-03-31 08:54:16.924497+00:00) 2020-03-31 08:54:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:54:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:54:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:26.924497+00:00 (in 9.999236 seconds) 2020-03-31 08:54:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:26 UTC)" (scheduled at 2020-03-31 08:54:16.924497+00:00) 2020-03-31 08:54:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 16, 925853), True, datetime.datetime(2020, 3, 31, 8, 54, 16, 925853), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:54:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 16, 925853), True, datetime.datetime(2020, 3, 31, 8, 54, 16, 925853), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:54:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:54:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:54:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:26 UTC)" executed successfully 2020-03-31 08:54:18,273 [518] [DEBUG] [app] Starting request: urn:request:60897438-3fcd-4577-b365-9fc49abf4e13 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:18,273 [518] [DEBUG] [app] Starting request: urn:request:60897438-3fcd-4577-b365-9fc49abf4e13 (/health/instance) 2020-03-31 08:54:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:18,282 [550] [DEBUG] [app] Starting request: urn:request:e7abdc81-ee3a-4a8c-adfc-c8658e8017ea (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:18,282 [550] [DEBUG] [app] Starting request: urn:request:e7abdc81-ee3a-4a8c-adfc-c8658e8017ea (/v1/_internal_ping) 2020-03-31 08:54:18,283 [550] [DEBUG] [app] Ending request: urn:request:e7abdc81-ee3a-4a8c-adfc-c8658e8017ea (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:18,283 [550] [DEBUG] [app] Ending request: urn:request:e7abdc81-ee3a-4a8c-adfc-c8658e8017ea (/v1/_internal_ping) 2020-03-31 08:54:18,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:54:18,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:18,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 08:54:18,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:18,287 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:18,287 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:18,289 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:18,289 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:18,290 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:18,290 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:18,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:18,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:18,318 [518] [DEBUG] [app] Starting request: urn:request:ea2beba7-b894-450a-897d-0bf9ba4775fd (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:18,318 [518] [DEBUG] [app] Starting request: urn:request:ea2beba7-b894-450a-897d-0bf9ba4775fd (/_internal_ping) 2020-03-31 08:54:18,318 [518] [DEBUG] [app] Ending request: urn:request:ea2beba7-b894-450a-897d-0bf9ba4775fd (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:18,318 [518] [DEBUG] [app] Ending request: urn:request:ea2beba7-b894-450a-897d-0bf9ba4775fd (/_internal_ping) 2020-03-31 08:54:18,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:18,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:54:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:54:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:18,332 [539] [DEBUG] [app] Starting request: urn:request:2d97e0e4-72b5-44ea-9149-3ed89a0a7716 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:18,332 [539] [DEBUG] [app] Starting request: urn:request:2d97e0e4-72b5-44ea-9149-3ed89a0a7716 (/c1/_internal_ping) 2020-03-31 08:54:18,333 [539] [DEBUG] [app] Ending request: urn:request:2d97e0e4-72b5-44ea-9149-3ed89a0a7716 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:18,333 [539] [DEBUG] [app] Ending request: urn:request:2d97e0e4-72b5-44ea-9149-3ed89a0a7716 (/c1/_internal_ping) 2020-03-31 08:54:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:18,334 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:18,335 [518] [DEBUG] [app] Ending request: urn:request:60897438-3fcd-4577-b365-9fc49abf4e13 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:54:18,334 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:18,335 [518] [DEBUG] [app] Ending request: urn:request:60897438-3fcd-4577-b365-9fc49abf4e13 (/health/instance) 2020-03-31 08:54:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 08:54:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 08:54:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:54:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:31.440707+00:00 (in 9.998812 seconds) 2020-03-31 08:54:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:31 UTC)" (scheduled at 2020-03-31 08:54:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:54:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:31.440707+00:00 (in 9.998812 seconds) 2020-03-31 08:54:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:31 UTC)" (scheduled at 2020-03-31 08:54:21.440707+00:00) 2020-03-31 08:54:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:54:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:54:21,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:54:21,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:54:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:54:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:31 UTC)" executed successfully 2020-03-31 08:54:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:54:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:52.113221+00:00 (in 29.997382 seconds) 2020-03-31 08:54:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:52 UTC)" (scheduled at 2020-03-31 08:54:22.113221+00:00) securityworker stdout | 2020-03-31 08:54:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:52.113221+00:00 (in 29.997382 seconds) 2020-03-31 08:54:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:52 UTC)" (scheduled at 2020-03-31 08:54:22.113221+00:00) 2020-03-31 08:54:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:54:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:54:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:54:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:54:22,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:54:22,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:54:22,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:22,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:54:22,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:22,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:54:52 UTC)" executed successfully 2020-03-31 08:54:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:54:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:24.410927+00:00 (in 0.999456 seconds) 2020-03-31 08:54:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:23 UTC)" (scheduled at 2020-03-31 08:54:23.410371+00:00) security_notification_worker stdout | 2020-03-31 08:54:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:24.410927+00:00 (in 0.999456 seconds) 2020-03-31 08:54:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:23 UTC)" (scheduled at 2020-03-31 08:54:23.410371+00:00) 2020-03-31 08:54:23,412 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 08:54:23,412 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), u'security_notification/%']) 2020-03-31 08:54:23,426 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), True, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 08:54:23,426 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), True, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), 0, u'security_notification/%']) 2020-03-31 08:54:23,430 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), True, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 08:54:23,430 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), True, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 8, 54, 23, 412135), u'security_notification/%']) 2020-03-31 08:54:23,434 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 08:54:23,434 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:23,435 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:54:23,435 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:23 UTC)" executed successfully 2020-03-31 08:54:23,437 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 08:54:23,437 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:23,439 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 08:54:23,439 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:23,439 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 08:54:23,439 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 08:54:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:54:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:34.323352+00:00 (in 9.998565 seconds) notificationworker stdout | 2020-03-31 08:54:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:34.323352+00:00 (in 9.998565 seconds) 2020-03-31 08:54:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:34 UTC)" (scheduled at 2020-03-31 08:54:24.323352+00:00) 2020-03-31 08:54:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:54:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:34 UTC)" (scheduled at 2020-03-31 08:54:24.323352+00:00) 2020-03-31 08:54:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 24, 325840), True, datetime.datetime(2020, 3, 31, 8, 54, 24, 325840), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:54:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 24, 325840), True, datetime.datetime(2020, 3, 31, 8, 54, 24, 325840), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:54:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:24,341 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:54:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:54:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:34 UTC)" executed successfully 2020-03-31 08:54:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:54:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:33.330112+00:00 (in 8.918301 seconds) 2020-03-31 08:54:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:24 UTC)" (scheduled at 2020-03-31 08:54:24.410927+00:00) 2020-03-31 08:54:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 08:54:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:33.330112+00:00 (in 8.918301 seconds) 2020-03-31 08:54:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:24 UTC)" (scheduled at 2020-03-31 08:54:24.410927+00:00) 2020-03-31 08:54:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:54:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:24 UTC)" executed successfully 2020-03-31 08:54:24,446 [525] [DEBUG] [app] Starting request: urn:request:06c1a4ad-4a76-4dfe-81ef-28c74edcf0b5 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:24,446 [525] [DEBUG] [app] Starting request: urn:request:06c1a4ad-4a76-4dfe-81ef-28c74edcf0b5 (/health/instance) 2020-03-31 08:54:24,448 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:24,448 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:24,452 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:24,452 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:24,453 [555] [DEBUG] [app] Starting request: urn:request:26927eae-22f3-4f95-85fa-8125614fa8ef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:24,453 [555] [DEBUG] [app] Starting request: urn:request:26927eae-22f3-4f95-85fa-8125614fa8ef (/v1/_internal_ping) 2020-03-31 08:54:24,455 [555] [DEBUG] [app] Ending request: urn:request:26927eae-22f3-4f95-85fa-8125614fa8ef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:24,455 [555] [DEBUG] [app] Ending request: urn:request:26927eae-22f3-4f95-85fa-8125614fa8ef (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:24,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:24,455 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:54:24,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:24,456 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:24,455 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:24,456 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:24,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:24,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:24,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:24,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:24,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:24,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:24,467 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:24,467 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:24,481 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:24,481 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:24,485 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:24,485 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:24,486 [525] [DEBUG] [app] Starting request: urn:request:b68c98dc-710e-42bd-a648-a96f3427d4b8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:24,486 [525] [DEBUG] [app] Starting request: urn:request:b68c98dc-710e-42bd-a648-a96f3427d4b8 (/_internal_ping) 2020-03-31 08:54:24,487 [525] [DEBUG] [app] Ending request: urn:request:b68c98dc-710e-42bd-a648-a96f3427d4b8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:24,487 [525] [DEBUG] [app] Ending request: urn:request:b68c98dc-710e-42bd-a648-a96f3427d4b8 (/_internal_ping) 2020-03-31 08:54:24,488 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:24,488 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:24,488 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:24,488 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:24,491 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:24,491 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:24,492 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:24,492 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:24,494 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:24,494 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:24,495 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:54:24,495 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:54:24,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:24,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:24,499 [539] [DEBUG] [app] Starting request: urn:request:be7bb097-4f40-45cf-99a2-b04e6b6df81b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:24,499 [539] [DEBUG] [app] Starting request: urn:request:be7bb097-4f40-45cf-99a2-b04e6b6df81b (/c1/_internal_ping) 2020-03-31 08:54:24,500 [539] [DEBUG] [app] Ending request: urn:request:be7bb097-4f40-45cf-99a2-b04e6b6df81b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:24,500 [539] [DEBUG] [app] Ending request: urn:request:be7bb097-4f40-45cf-99a2-b04e6b6df81b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:24,500 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:24,501 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:54:24,500 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:24,501 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:24,502 [525] [DEBUG] [app] Ending request: urn:request:06c1a4ad-4a76-4dfe-81ef-28c74edcf0b5 (/health/instance) 2020-03-31 08:54:24,502 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:24,502 [525] [DEBUG] [app] Ending request: urn:request:06c1a4ad-4a76-4dfe-81ef-28c74edcf0b5 (/health/instance) 2020-03-31 08:54:24,502 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:24,502 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 08:54:24,502 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 08:54:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:36.924497+00:00 (in 9.999154 seconds) storagereplication stdout | 2020-03-31 08:54:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:36.924497+00:00 (in 9.999154 seconds) 2020-03-31 08:54:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:36 UTC)" (scheduled at 2020-03-31 08:54:26.924497+00:00) 2020-03-31 08:54:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:54:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:36 UTC)" (scheduled at 2020-03-31 08:54:26.924497+00:00) 2020-03-31 08:54:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 26, 927053), True, datetime.datetime(2020, 3, 31, 8, 54, 26, 927053), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:54:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 26, 927053), True, datetime.datetime(2020, 3, 31, 8, 54, 26, 927053), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:54:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:54:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:54:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:36 UTC)" executed successfully 2020-03-31 08:54:28,273 [518] [DEBUG] [app] Starting request: urn:request:5ff95a3e-ff14-4576-882f-01be815c191e (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:28,273 [518] [DEBUG] [app] Starting request: urn:request:5ff95a3e-ff14-4576-882f-01be815c191e (/health/instance) 2020-03-31 08:54:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:28,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:28,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:28,283 [550] [DEBUG] [app] Starting request: urn:request:a1861a93-b195-43b6-bafd-3d065cb164d1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:28,283 [550] [DEBUG] [app] Starting request: urn:request:a1861a93-b195-43b6-bafd-3d065cb164d1 (/v1/_internal_ping) 2020-03-31 08:54:28,284 [550] [DEBUG] [app] Ending request: urn:request:a1861a93-b195-43b6-bafd-3d065cb164d1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:28,284 [550] [DEBUG] [app] Ending request: urn:request:a1861a93-b195-43b6-bafd-3d065cb164d1 (/v1/_internal_ping) 2020-03-31 08:54:28,285 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:54:28,285 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:54:28,286 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:28,286 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:28,287 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:28,287 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:28,290 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:28,290 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:28,291 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:28,291 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:28,292 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:28,292 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:28,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:28,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:28,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:28,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:28,319 [527] [DEBUG] [app] Starting request: urn:request:fa236c04-0893-4ae6-91dd-2dd26cc22f69 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:28,319 [527] [DEBUG] [app] Starting request: urn:request:fa236c04-0893-4ae6-91dd-2dd26cc22f69 (/_internal_ping) 2020-03-31 08:54:28,320 [527] [DEBUG] [app] Ending request: urn:request:fa236c04-0893-4ae6-91dd-2dd26cc22f69 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:28,320 [527] [DEBUG] [app] Ending request: urn:request:fa236c04-0893-4ae6-91dd-2dd26cc22f69 (/_internal_ping) 2020-03-31 08:54:28,321 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:54:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:28,321 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:54:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:54:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:54:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:28,340 [541] [DEBUG] [app] Starting request: urn:request:9f14d719-53d3-44a3-a0a1-d4d1fdbbc2bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:28,340 [541] [DEBUG] [app] Starting request: urn:request:9f14d719-53d3-44a3-a0a1-d4d1fdbbc2bd (/c1/_internal_ping) 2020-03-31 08:54:28,345 [541] [DEBUG] [app] Ending request: urn:request:9f14d719-53d3-44a3-a0a1-d4d1fdbbc2bd (/c1/_internal_ping) 2020-03-31 08:54:28,346 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 162 0.010) 2020-03-31 08:54:28,348 [518] [DEBUG] [app] Ending request: urn:request:5ff95a3e-ff14-4576-882f-01be815c191e (/health/instance) 2020-03-31 08:54:28,347 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:28,348 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:54:28,345 [541] [DEBUG] [app] Ending request: urn:request:9f14d719-53d3-44a3-a0a1-d4d1fdbbc2bd (/c1/_internal_ping) 2020-03-31 08:54:28,349 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) gunicorn-web stdout | 2020-03-31 08:54:28,346 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.010 162 0.010) gunicorn-web stdout | 2020-03-31 08:54:28,348 [518] [DEBUG] [app] Ending request: urn:request:5ff95a3e-ff14-4576-882f-01be815c191e (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:54:28,347 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:28,348 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:28,349 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) 2020-03-31 08:54:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:54:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:00.337699+00:00 (in 29.998988 seconds) 2020-03-31 08:54:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:00 UTC)" (scheduled at 2020-03-31 08:54:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:54:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:00.337699+00:00 (in 29.998988 seconds) 2020-03-31 08:54:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:00 UTC)" (scheduled at 2020-03-31 08:54:30.337699+00:00) 2020-03-31 08:54:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 54, 30, 339621), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:54:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 54, 30, 339621), False, 50, 1, 0]) 2020-03-31 08:54:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:54:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:54:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:54:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:00 UTC)" executed successfully 2020-03-31 08:54:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:54:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:41.440707+00:00 (in 9.998924 seconds) 2020-03-31 08:54:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:41 UTC)" (scheduled at 2020-03-31 08:54:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:54:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:41.440707+00:00 (in 9.998924 seconds) 2020-03-31 08:54:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:41 UTC)" (scheduled at 2020-03-31 08:54:31.440707+00:00) 2020-03-31 08:54:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:54:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:54:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:54:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:54:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:54:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:41 UTC)" executed successfully 2020-03-31 08:54:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:54:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:03.330112+00:00 (in 29.999057 seconds) 2020-03-31 08:54:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:03 UTC)" (scheduled at 2020-03-31 08:54:33.330112+00:00) 2020-03-31 08:54:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:54:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:03.330112+00:00 (in 29.999057 seconds) 2020-03-31 08:54:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:03 UTC)" (scheduled at 2020-03-31 08:54:33.330112+00:00) 2020-03-31 08:54:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 33, 332064), True, datetime.datetime(2020, 3, 31, 8, 54, 33, 332064), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:54:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 33, 332064), True, datetime.datetime(2020, 3, 31, 8, 54, 33, 332064), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:54:33,347 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:54:33,347 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:33,347 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:33,347 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:54:33,347 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:33,347 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:03 UTC)" executed successfully 2020-03-31 08:54:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:54:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:43.539999+00:00 (in 10.172410 seconds) 2020-03-31 08:54:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:33 UTC)" (scheduled at 2020-03-31 08:54:33.366791+00:00) 2020-03-31 08:54:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:54:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:43.539999+00:00 (in 10.172410 seconds) 2020-03-31 08:54:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:33 UTC)" (scheduled at 2020-03-31 08:54:33.366791+00:00) 2020-03-31 08:54:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 33, 368578), True, datetime.datetime(2020, 3, 31, 8, 54, 33, 368578), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:54:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 33, 368578), True, datetime.datetime(2020, 3, 31, 8, 54, 33, 368578), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:54:33,382 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:54:33,382 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:33,382 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:54:33,382 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:33 UTC)" executed successfully 2020-03-31 08:54:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:54:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:44.323352+00:00 (in 9.998848 seconds) 2020-03-31 08:54:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:44 UTC)" (scheduled at 2020-03-31 08:54:34.323352+00:00) 2020-03-31 08:54:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:54:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:44.323352+00:00 (in 9.998848 seconds) 2020-03-31 08:54:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:44 UTC)" (scheduled at 2020-03-31 08:54:34.323352+00:00) 2020-03-31 08:54:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 34, 325145), True, datetime.datetime(2020, 3, 31, 8, 54, 34, 325145), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:54:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 34, 325145), True, datetime.datetime(2020, 3, 31, 8, 54, 34, 325145), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:54:34,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:54:34,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:54:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:44 UTC)" executed successfully 2020-03-31 08:54:34,446 [527] [DEBUG] [app] Starting request: urn:request:74820c15-9316-48e2-af45-18ed9645de1d (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:34,446 [527] [DEBUG] [app] Starting request: urn:request:74820c15-9316-48e2-af45-18ed9645de1d (/health/instance) 2020-03-31 08:54:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:34,454 [550] [DEBUG] [app] Starting request: urn:request:fa2f95e0-03e9-404d-bb88-98db114bd8c6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:34,454 [550] [DEBUG] [app] Starting request: urn:request:fa2f95e0-03e9-404d-bb88-98db114bd8c6 (/v1/_internal_ping) 2020-03-31 08:54:34,455 [550] [DEBUG] [app] Ending request: urn:request:fa2f95e0-03e9-404d-bb88-98db114bd8c6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:34,455 [550] [DEBUG] [app] Ending request: urn:request:fa2f95e0-03e9-404d-bb88-98db114bd8c6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:34,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:54:34,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:34,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:54:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:54:34,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:34,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:34,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:34,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:34,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:34,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:34,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:34,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:34,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:34,485 [527] [DEBUG] [app] Starting request: urn:request:3ef98589-f675-4688-991f-92fee1de1942 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:34,485 [527] [DEBUG] [app] Starting request: urn:request:3ef98589-f675-4688-991f-92fee1de1942 (/_internal_ping) 2020-03-31 08:54:34,486 [527] [DEBUG] [app] Ending request: urn:request:3ef98589-f675-4688-991f-92fee1de1942 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:34,486 [527] [DEBUG] [app] Ending request: urn:request:3ef98589-f675-4688-991f-92fee1de1942 (/_internal_ping) 2020-03-31 08:54:34,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 08:54:34,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 08:54:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:54:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:54:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:54:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:34,500 [541] [DEBUG] [app] Starting request: urn:request:95f2c6a7-1c65-48bd-b01e-778602630f9f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:34,500 [541] [DEBUG] [app] Starting request: urn:request:95f2c6a7-1c65-48bd-b01e-778602630f9f (/c1/_internal_ping) 2020-03-31 08:54:34,501 [541] [DEBUG] [app] Ending request: urn:request:95f2c6a7-1c65-48bd-b01e-778602630f9f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:34,501 [541] [DEBUG] [app] Ending request: urn:request:95f2c6a7-1c65-48bd-b01e-778602630f9f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:34,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:54:34,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:34,504 [527] [DEBUG] [app] Ending request: urn:request:74820c15-9316-48e2-af45-18ed9645de1d (/health/instance) 2020-03-31 08:54:34,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:34,504 [527] [DEBUG] [app] Ending request: urn:request:74820c15-9316-48e2-af45-18ed9645de1d (/health/instance) 2020-03-31 08:54:34,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:34,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:54:34,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:54:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:54:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:38.010934+00:00 (in 1.084779 seconds) storagereplication stdout | 2020-03-31 08:54:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:38.010934+00:00 (in 1.084779 seconds) 2020-03-31 08:54:36,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:46 UTC)" (scheduled at 2020-03-31 08:54:36.924497+00:00) 2020-03-31 08:54:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:54:36,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:46 UTC)" (scheduled at 2020-03-31 08:54:36.924497+00:00) 2020-03-31 08:54:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 36, 928103), True, datetime.datetime(2020, 3, 31, 8, 54, 36, 928103), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:54:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 36, 928103), True, datetime.datetime(2020, 3, 31, 8, 54, 36, 928103), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:54:36,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:54:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:54:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:46 UTC)" executed successfully 2020-03-31 08:54:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:54:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:46.924497+00:00 (in 8.912840 seconds) 2020-03-31 08:54:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:38 UTC)" (scheduled at 2020-03-31 08:54:38.010934+00:00) 2020-03-31 08:54:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:54:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:46.924497+00:00 (in 8.912840 seconds) 2020-03-31 08:54:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:38 UTC)" (scheduled at 2020-03-31 08:54:38.010934+00:00) 2020-03-31 08:54:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:38 UTC)" executed successfully 2020-03-31 08:54:38,273 [518] [DEBUG] [app] Starting request: urn:request:d2ad096c-4ecf-47cd-a90e-3d83ad777652 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:38,273 [518] [DEBUG] [app] Starting request: urn:request:d2ad096c-4ecf-47cd-a90e-3d83ad777652 (/health/instance) 2020-03-31 08:54:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:38,280 [538] [DEBUG] [app] Starting request: urn:request:3e4b91b9-dff1-4534-bbbb-08ed52fae8db (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:38,280 [538] [DEBUG] [app] Starting request: urn:request:3e4b91b9-dff1-4534-bbbb-08ed52fae8db (/v1/_internal_ping) 2020-03-31 08:54:38,281 [538] [DEBUG] [app] Ending request: urn:request:3e4b91b9-dff1-4534-bbbb-08ed52fae8db (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:38,281 [538] [DEBUG] [app] Ending request: urn:request:3e4b91b9-dff1-4534-bbbb-08ed52fae8db (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:38,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:54:38,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:38,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:38,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:38,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:38,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:38,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:38,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:38,319 [518] [DEBUG] [app] Starting request: urn:request:0691fd64-df2f-4dc2-80d7-95abf47eb4a4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:38,319 [518] [DEBUG] [app] Starting request: urn:request:0691fd64-df2f-4dc2-80d7-95abf47eb4a4 (/_internal_ping) 2020-03-31 08:54:38,319 [518] [DEBUG] [app] Ending request: urn:request:0691fd64-df2f-4dc2-80d7-95abf47eb4a4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:38,319 [518] [DEBUG] [app] Ending request: urn:request:0691fd64-df2f-4dc2-80d7-95abf47eb4a4 (/_internal_ping) 2020-03-31 08:54:38,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:38,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:54:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:54:38,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:38,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:38,332 [541] [DEBUG] [app] Starting request: urn:request:11bc85f5-fa33-4160-a1ac-21fa6ac2fd85 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:38,332 [541] [DEBUG] [app] Starting request: urn:request:11bc85f5-fa33-4160-a1ac-21fa6ac2fd85 (/c1/_internal_ping) 2020-03-31 08:54:38,333 [541] [DEBUG] [app] Ending request: urn:request:11bc85f5-fa33-4160-a1ac-21fa6ac2fd85 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:38,333 [541] [DEBUG] [app] Ending request: urn:request:11bc85f5-fa33-4160-a1ac-21fa6ac2fd85 (/c1/_internal_ping) 2020-03-31 08:54:38,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:38,333 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:38,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:38,335 [518] [DEBUG] [app] Ending request: urn:request:d2ad096c-4ecf-47cd-a90e-3d83ad777652 (/health/instance) 2020-03-31 08:54:38,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:54:38,333 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:38,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:54:38,335 [518] [DEBUG] [app] Ending request: urn:request:d2ad096c-4ecf-47cd-a90e-3d83ad777652 (/health/instance) 2020-03-31 08:54:38,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:38,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:54:39,155 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2020-03-31 08:54:39,155 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:39,156 [165] [INFO] [apscheduler.executors.default] Running job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 08:54:39 UTC)" (scheduled at 2020-03-31 08:54:39.155593+00:00) servicekey stdout | 2020-03-31 08:54:39,156 [165] [INFO] [apscheduler.executors.default] Running job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 08:54:39 UTC)" (scheduled at 2020-03-31 08:54:39.155593+00:00) 2020-03-31 08:54:39,157 [165] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:39.155593+00:00 (in 3299.999563 seconds) 2020-03-31 08:54:39,157 [165] [DEBUG] [__main__] Starting automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 to new expiration 2020-03-31 10:54:39.156731 servicekey stdout | 2020-03-31 08:54:39,157 [165] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:39.155593+00:00 (in 3299.999563 seconds) 2020-03-31 08:54:39,157 [165] [DEBUG] [__main__] Starting automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 to new expiration 2020-03-31 10:54:39.156731 2020-03-31 08:54:39,159 [165] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', [u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', datetime.datetime(2020, 3, 24, 8, 54, 39, 157879), None, 1, 0]) servicekey stdout | 2020-03-31 08:54:39,159 [165] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', [u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', datetime.datetime(2020, 3, 24, 8, 54, 39, 157879), None, 1, 0]) 2020-03-31 08:54:39,181 [165] [DEBUG] [peewee] ('UPDATE "servicekey" SET "name" = %s, "kid" = %s, "service" = %s, "jwk" = %s, "metadata" = %s, "created_date" = %s, "expiration_date" = %s, "rotation_duration" = %s, "approval_id" = %s WHERE ("servicekey"."id" = %s)', [u'https://quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com:443', u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', u'quay', '{"e": "AQAB", "kty": "RSA", "n": "x2cjbBkjdqblm4sOgP83bzbi9iWUQh70dRViEYlT_NPxaj0tU4s1IVaiU2gADr4n0LORlbpr6aaSOh5QWeDu4RkWFgbCmDGqBfnMioSyt2wMpcmBunBUDGh8Lw9t86_XTm-KzDZ9OdcRpwaA7qKabZjH-kD8U6txl_rPkkzC5xoEhjr_l4U-pXq8dW65pB8iecP0h8mbr6xflR1TlR20QDvB7nzm59rj1ZtvbvqOyOQo5XHvT-8JBK7FSfBi7GQgSc4zZG7UBUcSnBR6l6nxKbuPtceesAm1PGy6xV2wVpIHGfn6dIHJ4uEwCRVqEumR2qZonmZzmSgROCgooVOnkw"}', '{"created_by": "CLI tool"}', datetime.datetime(2020, 3, 31, 8, 49, 54, 471685), datetime.datetime(2020, 3, 31, 10, 54, 39, 156731), None, 1, 1]) servicekey stdout | 2020-03-31 08:54:39,181 [165] [DEBUG] [peewee] ('UPDATE "servicekey" SET "name" = %s, "kid" = %s, "service" = %s, "jwk" = %s, "metadata" = %s, "created_date" = %s, "expiration_date" = %s, "rotation_duration" = %s, "approval_id" = %s WHERE ("servicekey"."id" = %s)', [u'https://quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com:443', u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', u'quay', '{"e": "AQAB", "kty": "RSA", "n": "x2cjbBkjdqblm4sOgP83bzbi9iWUQh70dRViEYlT_NPxaj0tU4s1IVaiU2gADr4n0LORlbpr6aaSOh5QWeDu4RkWFgbCmDGqBfnMioSyt2wMpcmBunBUDGh8Lw9t86_XTm-KzDZ9OdcRpwaA7qKabZjH-kD8U6txl_rPkkzC5xoEhjr_l4U-pXq8dW65pB8iecP0h8mbr6xflR1TlR20QDvB7nzm59rj1ZtvbvqOyOQo5XHvT-8JBK7FSfBi7GQgSc4zZG7UBUcSnBR6l6nxKbuPtceesAm1PGy6xV2wVpIHGfn6dIHJ4uEwCRVqEumR2qZonmZzmSgROCgooVOnkw"}', '{"created_by": "CLI tool"}', datetime.datetime(2020, 3, 31, 8, 49, 54, 471685), datetime.datetime(2020, 3, 31, 10, 54, 39, 156731), None, 1, 1]) 2020-03-31 08:54:39,187 [165] [DEBUG] [__main__] Finished automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 with new expiration 2020-03-31 10:54:39.156731 servicekey stdout | 2020-03-31 08:54:39,187 [165] [DEBUG] [__main__] Finished automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 with new expiration 2020-03-31 10:54:39.156731 2020-03-31 08:54:39,188 [165] [DEBUG] [data.database] Disconnecting from database. servicekey stdout | 2020-03-31 08:54:39,188 [165] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:39,190 [165] [INFO] [apscheduler.executors.default] Job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 09:49:39 UTC)" executed successfully servicekey stdout | 2020-03-31 08:54:39,190 [165] [INFO] [apscheduler.executors.default] Job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 09:49:39 UTC)" executed successfully 2020-03-31 08:54:39,193 [165] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 servicekey stdout | 2020-03-31 08:54:39,193 [165] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:39,194 [165] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 servicekey stdout | 2020-03-31 08:54:39,194 [165] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:39,195 [165] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics servicekey stdout | 2020-03-31 08:54:39,195 [165] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 08:54:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:54:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:51.440707+00:00 (in 9.998259 seconds) 2020-03-31 08:54:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:51 UTC)" (scheduled at 2020-03-31 08:54:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:54:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:51.440707+00:00 (in 9.998259 seconds) 2020-03-31 08:54:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:51 UTC)" (scheduled at 2020-03-31 08:54:41.440707+00:00) 2020-03-31 08:54:41,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:54:41,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:54:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:54:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:54:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:54:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:51 UTC)" executed successfully 2020-03-31 08:54:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:54:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:14.539408+00:00 (in 30.997959 seconds) 2020-03-31 08:54:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:43 UTC)" (scheduled at 2020-03-31 08:54:43.539999+00:00) 2020-03-31 08:54:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 08:54:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:14.539408+00:00 (in 30.997959 seconds) 2020-03-31 08:54:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:43 UTC)" (scheduled at 2020-03-31 08:54:43.539999+00:00) 2020-03-31 08:54:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:54:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:43 UTC)" executed successfully 2020-03-31 08:54:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:54:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:49.498966+00:00 (in 5.174671 seconds) 2020-03-31 08:54:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:54 UTC)" (scheduled at 2020-03-31 08:54:44.323352+00:00) 2020-03-31 08:54:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:54:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:49.498966+00:00 (in 5.174671 seconds) 2020-03-31 08:54:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:54 UTC)" (scheduled at 2020-03-31 08:54:44.323352+00:00) 2020-03-31 08:54:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 44, 325056), True, datetime.datetime(2020, 3, 31, 8, 54, 44, 325056), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:54:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 44, 325056), True, datetime.datetime(2020, 3, 31, 8, 54, 44, 325056), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:54:44,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:54:44,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:54:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:54 UTC)" executed successfully 2020-03-31 08:54:44,446 [518] [DEBUG] [app] Starting request: urn:request:cc5df784-2d4f-4f4a-8ed1-388cc3241495 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:44,446 [518] [DEBUG] [app] Starting request: urn:request:cc5df784-2d4f-4f4a-8ed1-388cc3241495 (/health/instance) 2020-03-31 08:54:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:44,454 [556] [DEBUG] [app] Starting request: urn:request:377b53f4-0918-423e-85ea-cfb71ce406ac (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:44,454 [556] [DEBUG] [app] Starting request: urn:request:377b53f4-0918-423e-85ea-cfb71ce406ac (/v1/_internal_ping) 2020-03-31 08:54:44,455 [556] [DEBUG] [app] Ending request: urn:request:377b53f4-0918-423e-85ea-cfb71ce406ac (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:44,455 [556] [DEBUG] [app] Ending request: urn:request:377b53f4-0918-423e-85ea-cfb71ce406ac (/v1/_internal_ping) 2020-03-31 08:54:44,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:54:44,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:54:44,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:54:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:54:44,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:44,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:44,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:44,463 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:44,463 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:44,485 [518] [DEBUG] [app] Starting request: urn:request:fa83b526-1560-47fd-bb2e-6f28ca91c7c5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:44,485 [518] [DEBUG] [app] Starting request: urn:request:fa83b526-1560-47fd-bb2e-6f28ca91c7c5 (/_internal_ping) 2020-03-31 08:54:44,486 [518] [DEBUG] [app] Ending request: urn:request:fa83b526-1560-47fd-bb2e-6f28ca91c7c5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:44,486 [518] [DEBUG] [app] Ending request: urn:request:fa83b526-1560-47fd-bb2e-6f28ca91c7c5 (/_internal_ping) 2020-03-31 08:54:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:54:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:44,499 [532] [DEBUG] [app] Starting request: urn:request:3a991ac8-8320-44a6-970a-4bcee1bedd68 (/c1/_internal_ping) 2020-03-31 08:54:44,500 [532] [DEBUG] [app] Ending request: urn:request:3a991ac8-8320-44a6-970a-4bcee1bedd68 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:54:44,499 [532] [DEBUG] [app] Starting request: urn:request:3a991ac8-8320-44a6-970a-4bcee1bedd68 (/c1/_internal_ping) 2020-03-31 08:54:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:44,500 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:54:44,500 [532] [DEBUG] [app] Ending request: urn:request:3a991ac8-8320-44a6-970a-4bcee1bedd68 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:44,502 [518] [DEBUG] [app] Ending request: urn:request:cc5df784-2d4f-4f4a-8ed1-388cc3241495 (/health/instance) 2020-03-31 08:54:44,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:44,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-verbs stdout | 2020-03-31 08:54:44,500 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:44,502 [518] [DEBUG] [app] Ending request: urn:request:cc5df784-2d4f-4f4a-8ed1-388cc3241495 (/health/instance) 2020-03-31 08:54:44,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:44,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 08:54:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:54:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:15.064514+00:00 (in 29.999170 seconds) 2020-03-31 08:54:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:15 UTC)" (scheduled at 2020-03-31 08:54:45.064514+00:00) gcworker stdout | 2020-03-31 08:54:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:15.064514+00:00 (in 29.999170 seconds) 2020-03-31 08:54:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:15 UTC)" (scheduled at 2020-03-31 08:54:45.064514+00:00) 2020-03-31 08:54:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435285, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:54:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435285, 1209600, 500, 1, 0]) 2020-03-31 08:54:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:54:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:54:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:54:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:15 UTC)" executed successfully 2020-03-31 08:54:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:54:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:56.924497+00:00 (in 9.998915 seconds) 2020-03-31 08:54:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:56 UTC)" (scheduled at 2020-03-31 08:54:46.924497+00:00) 2020-03-31 08:54:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:54:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:56.924497+00:00 (in 9.998915 seconds) 2020-03-31 08:54:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:56 UTC)" (scheduled at 2020-03-31 08:54:46.924497+00:00) 2020-03-31 08:54:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 46, 926242), True, datetime.datetime(2020, 3, 31, 8, 54, 46, 926242), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:54:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 46, 926242), True, datetime.datetime(2020, 3, 31, 8, 54, 46, 926242), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:54:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:54:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:54:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:54:56 UTC)" executed successfully 2020-03-31 08:54:48,273 [527] [DEBUG] [app] Starting request: urn:request:46ee80fb-315b-4736-908f-0c7b69238ace (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:48,273 [527] [DEBUG] [app] Starting request: urn:request:46ee80fb-315b-4736-908f-0c7b69238ace (/health/instance) 2020-03-31 08:54:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:48,280 [538] [DEBUG] [app] Starting request: urn:request:f54887de-50fd-4b44-ade1-ffdd66037ffe (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:48,280 [538] [DEBUG] [app] Starting request: urn:request:f54887de-50fd-4b44-ade1-ffdd66037ffe (/v1/_internal_ping) 2020-03-31 08:54:48,281 [538] [DEBUG] [app] Ending request: urn:request:f54887de-50fd-4b44-ade1-ffdd66037ffe (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:48,281 [538] [DEBUG] [app] Ending request: urn:request:f54887de-50fd-4b44-ade1-ffdd66037ffe (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:54:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:48,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:48,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:48,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:48,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:48,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:48,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:48,312 [527] [DEBUG] [app] Starting request: urn:request:cb064f01-876b-4e09-8ec1-d36ef002968e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:48,312 [527] [DEBUG] [app] Starting request: urn:request:cb064f01-876b-4e09-8ec1-d36ef002968e (/_internal_ping) 2020-03-31 08:54:48,313 [527] [DEBUG] [app] Ending request: urn:request:cb064f01-876b-4e09-8ec1-d36ef002968e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:48,313 [527] [DEBUG] [app] Ending request: urn:request:cb064f01-876b-4e09-8ec1-d36ef002968e (/_internal_ping) 2020-03-31 08:54:48,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:54:48,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:54:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:54:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:54:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:54:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:48,327 [532] [DEBUG] [app] Starting request: urn:request:8bd77c5f-2c18-48aa-b12a-e67ccd23e950 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:48,327 [532] [DEBUG] [app] Starting request: urn:request:8bd77c5f-2c18-48aa-b12a-e67ccd23e950 (/c1/_internal_ping) 2020-03-31 08:54:48,327 [532] [DEBUG] [app] Ending request: urn:request:8bd77c5f-2c18-48aa-b12a-e67ccd23e950 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:48,327 [532] [DEBUG] [app] Ending request: urn:request:8bd77c5f-2c18-48aa-b12a-e67ccd23e950 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:54:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:48,328 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:54:48,328 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:48,329 [527] [DEBUG] [app] Ending request: urn:request:46ee80fb-315b-4736-908f-0c7b69238ace (/health/instance) 2020-03-31 08:54:48,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:48,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 08:54:48,329 [527] [DEBUG] [app] Ending request: urn:request:46ee80fb-315b-4736-908f-0c7b69238ace (/health/instance) 2020-03-31 08:54:48,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:48,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 08:54:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:54:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:54.323352+00:00 (in 4.823439 seconds) 2020-03-31 08:54:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:49 UTC)" (scheduled at 2020-03-31 08:54:49.498966+00:00) 2020-03-31 08:54:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:54:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:54.323352+00:00 (in 4.823439 seconds) 2020-03-31 08:54:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:49 UTC)" (scheduled at 2020-03-31 08:54:49.498966+00:00) 2020-03-31 08:54:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:49 UTC)" executed successfully 2020-03-31 08:54:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:54:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:53.556282+00:00 (in 2.444947 seconds) 2020-03-31 08:54:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:51 UTC)" (scheduled at 2020-03-31 08:54:51.110610+00:00) 2020-03-31 08:54:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:54:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:53.556282+00:00 (in 2.444947 seconds) 2020-03-31 08:54:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:51 UTC)" (scheduled at 2020-03-31 08:54:51.110610+00:00) 2020-03-31 08:54:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:51,113 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 51, 112133), True, datetime.datetime(2020, 3, 31, 8, 54, 51, 112133), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:54:51,113 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 51, 112133), True, datetime.datetime(2020, 3, 31, 8, 54, 51, 112133), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:54:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:54:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:54:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:51 UTC)" executed successfully 2020-03-31 08:54:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:54:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:01 UTC)" (scheduled at 2020-03-31 08:54:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:54:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:01 UTC)" (scheduled at 2020-03-31 08:54:51.440707+00:00) 2020-03-31 08:54:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:54:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:54:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:01.440707+00:00 (in 9.998766 seconds) repositoryactioncounter stdout | 2020-03-31 08:54:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:01.440707+00:00 (in 9.998766 seconds) 2020-03-31 08:54:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:54:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:54:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:54:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:01 UTC)" executed successfully 2020-03-31 08:54:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:54:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:22 UTC)" (scheduled at 2020-03-31 08:54:52.113221+00:00) securityworker stdout | 2020-03-31 08:54:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:22 UTC)" (scheduled at 2020-03-31 08:54:52.113221+00:00) 2020-03-31 08:54:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:54:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:22.113221+00:00 (in 29.998970 seconds) securityworker stdout | 2020-03-31 08:54:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:54:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:22.113221+00:00 (in 29.998970 seconds) 2020-03-31 08:54:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:54:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:54:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:54:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:54:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:54:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:22 UTC)" executed successfully 2020-03-31 08:54:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:54:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:59.560326+00:00 (in 6.003074 seconds) 2020-03-31 08:54:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:53 UTC)" (scheduled at 2020-03-31 08:54:53.556282+00:00) namespacegcworker stdout | 2020-03-31 08:54:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:54:59.560326+00:00 (in 6.003074 seconds) 2020-03-31 08:54:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:53 UTC)" (scheduled at 2020-03-31 08:54:53.556282+00:00) 2020-03-31 08:54:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 08:54:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), u'namespacegc/%']) 2020-03-31 08:54:53,579 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), True, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 08:54:53,579 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), True, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), 0, u'namespacegc/%']) 2020-03-31 08:54:53,584 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), True, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 08:54:53,584 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), True, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 8, 54, 53, 558179), u'namespacegc/%']) 2020-03-31 08:54:53,587 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 08:54:53,587 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:53,588 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:54:53,588 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 08:59:53 UTC)" executed successfully 2020-03-31 08:54:53,590 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 08:54:53,590 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:53,591 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 08:54:53,591 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:53,592 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 08:54:53,592 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 08:54:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:54:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:04.323352+00:00 (in 9.998826 seconds) 2020-03-31 08:54:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:04 UTC)" (scheduled at 2020-03-31 08:54:54.323352+00:00) 2020-03-31 08:54:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:54:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:04.323352+00:00 (in 9.998826 seconds) 2020-03-31 08:54:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:04 UTC)" (scheduled at 2020-03-31 08:54:54.323352+00:00) 2020-03-31 08:54:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 54, 325540), True, datetime.datetime(2020, 3, 31, 8, 54, 54, 325540), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:54:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 54, 325540), True, datetime.datetime(2020, 3, 31, 8, 54, 54, 325540), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:54:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:54:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:54:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:04 UTC)" executed successfully 2020-03-31 08:54:54,447 [518] [DEBUG] [app] Starting request: urn:request:adbe5d02-28c4-4f5c-9316-dc2503617c9b (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:54,447 [518] [DEBUG] [app] Starting request: urn:request:adbe5d02-28c4-4f5c-9316-dc2503617c9b (/health/instance) 2020-03-31 08:54:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:54,455 [556] [DEBUG] [app] Starting request: urn:request:7c40f644-dc75-47e0-8488-676dad258030 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:54,455 [556] [DEBUG] [app] Starting request: urn:request:7c40f644-dc75-47e0-8488-676dad258030 (/v1/_internal_ping) 2020-03-31 08:54:54,456 [556] [DEBUG] [app] Ending request: urn:request:7c40f644-dc75-47e0-8488-676dad258030 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:54,456 [556] [DEBUG] [app] Ending request: urn:request:7c40f644-dc75-47e0-8488-676dad258030 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:54,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:54:54,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:54,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:54,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:54:54,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:54,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:54,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:54,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:54,484 [518] [DEBUG] [app] Starting request: urn:request:d0a6dabc-7a68-4f40-a184-02a7531e01e9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:54,484 [518] [DEBUG] [app] Starting request: urn:request:d0a6dabc-7a68-4f40-a184-02a7531e01e9 (/_internal_ping) 2020-03-31 08:54:54,485 [518] [DEBUG] [app] Ending request: urn:request:d0a6dabc-7a68-4f40-a184-02a7531e01e9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:54,485 [518] [DEBUG] [app] Ending request: urn:request:d0a6dabc-7a68-4f40-a184-02a7531e01e9 (/_internal_ping) 2020-03-31 08:54:54,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:54:54,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:54:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:54:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:54,497 [532] [DEBUG] [app] Starting request: urn:request:e89989af-a8d4-402d-a803-b9d2428fe378 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:54,497 [532] [DEBUG] [app] Starting request: urn:request:e89989af-a8d4-402d-a803-b9d2428fe378 (/c1/_internal_ping) 2020-03-31 08:54:54,498 [532] [DEBUG] [app] Ending request: urn:request:e89989af-a8d4-402d-a803-b9d2428fe378 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:54,498 [532] [DEBUG] [app] Ending request: urn:request:e89989af-a8d4-402d-a803-b9d2428fe378 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:54,499 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:54:54,499 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:54,500 [518] [DEBUG] [app] Ending request: urn:request:adbe5d02-28c4-4f5c-9316-dc2503617c9b (/health/instance) 2020-03-31 08:54:54,500 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:54,500 [518] [DEBUG] [app] Ending request: urn:request:adbe5d02-28c4-4f5c-9316-dc2503617c9b (/health/instance) 2020-03-31 08:54:54,500 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:54,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 08:54:54,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 08:54:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:06.924497+00:00 (in 9.999142 seconds) 2020-03-31 08:54:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:06 UTC)" (scheduled at 2020-03-31 08:54:56.924497+00:00) 2020-03-31 08:54:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:54:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 56, 926028), True, datetime.datetime(2020, 3, 31, 8, 54, 56, 926028), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:54:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:54:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:06.924497+00:00 (in 9.999142 seconds) 2020-03-31 08:54:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:06 UTC)" (scheduled at 2020-03-31 08:54:56.924497+00:00) 2020-03-31 08:54:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:54:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 54, 56, 926028), True, datetime.datetime(2020, 3, 31, 8, 54, 56, 926028), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:54:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:54:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:54:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:54:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:06 UTC)" executed successfully 2020-03-31 08:54:58,272 [518] [DEBUG] [app] Starting request: urn:request:95b37869-701f-43aa-bb4c-cd6e85e3d8c0 (/health/instance) gunicorn-web stdout | 2020-03-31 08:54:58,272 [518] [DEBUG] [app] Starting request: urn:request:95b37869-701f-43aa-bb4c-cd6e85e3d8c0 (/health/instance) 2020-03-31 08:54:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:58,277 [556] [DEBUG] [app] Starting request: urn:request:075a5401-0fa8-461c-aa63-f44cbceefcb6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:58,277 [556] [DEBUG] [app] Starting request: urn:request:075a5401-0fa8-461c-aa63-f44cbceefcb6 (/v1/_internal_ping) 2020-03-31 08:54:58,278 [556] [DEBUG] [app] Ending request: urn:request:075a5401-0fa8-461c-aa63-f44cbceefcb6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:54:58,278 [556] [DEBUG] [app] Ending request: urn:request:075a5401-0fa8-461c-aa63-f44cbceefcb6 (/v1/_internal_ping) 2020-03-31 08:54:58,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:54:58,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:54:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:58,281 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:54:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:54:58,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:54:58,281 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:54:58,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:54:58,283 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:54:58,283 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:54:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:54:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:54:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:54:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:54:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:58,314 [526] [DEBUG] [app] Starting request: urn:request:cad5cc88-5c93-4056-8d8c-4b23cac6e1ef (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:58,314 [526] [DEBUG] [app] Starting request: urn:request:cad5cc88-5c93-4056-8d8c-4b23cac6e1ef (/_internal_ping) 2020-03-31 08:54:58,315 [526] [DEBUG] [app] Ending request: urn:request:cad5cc88-5c93-4056-8d8c-4b23cac6e1ef (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:54:58,315 [526] [DEBUG] [app] Ending request: urn:request:cad5cc88-5c93-4056-8d8c-4b23cac6e1ef (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:54:58,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:54:58,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:54:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:54:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:54:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:54:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:54:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:54:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:54:58,330 [532] [DEBUG] [app] Starting request: urn:request:94b12fed-4906-4ea4-9067-77ed6d1d4eb8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:58,330 [532] [DEBUG] [app] Starting request: urn:request:94b12fed-4906-4ea4-9067-77ed6d1d4eb8 (/c1/_internal_ping) 2020-03-31 08:54:58,330 [532] [DEBUG] [app] Ending request: urn:request:94b12fed-4906-4ea4-9067-77ed6d1d4eb8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:54:58,330 [532] [DEBUG] [app] Ending request: urn:request:94b12fed-4906-4ea4-9067-77ed6d1d4eb8 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:54:58,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:54:58,331 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:54:58,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:54:58,331 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:54:58,332 [518] [DEBUG] [app] Ending request: urn:request:95b37869-701f-43aa-bb4c-cd6e85e3d8c0 (/health/instance) 2020-03-31 08:54:58,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:54:58,332 [518] [DEBUG] [app] Ending request: urn:request:95b37869-701f-43aa-bb4c-cd6e85e3d8c0 (/health/instance) 2020-03-31 08:54:58,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:54:58,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 08:54:58,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:54:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:54:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 08:54:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:54:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:54:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:51.110610+00:00 (in 51.549211 seconds) 2020-03-31 08:54:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:59 UTC)" (scheduled at 2020-03-31 08:54:59.560326+00:00) 2020-03-31 08:54:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:54:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:51.110610+00:00 (in 51.549211 seconds) 2020-03-31 08:54:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:59 UTC)" (scheduled at 2020-03-31 08:54:59.560326+00:00) 2020-03-31 08:54:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:54:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:55:59 UTC)" executed successfully 2020-03-31 08:55:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:55:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:30.337699+00:00 (in 29.999136 seconds) 2020-03-31 08:55:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:30 UTC)" (scheduled at 2020-03-31 08:55:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:55:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:30.337699+00:00 (in 29.999136 seconds) 2020-03-31 08:55:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:30 UTC)" (scheduled at 2020-03-31 08:55:00.337699+00:00) 2020-03-31 08:55:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 55, 0, 339363), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:55:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 55, 0, 339363), False, 50, 1, 0]) 2020-03-31 08:55:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:55:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:55:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:55:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:30 UTC)" executed successfully 2020-03-31 08:55:01,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:55:01,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:11.440707+00:00 (in 9.996182 seconds) 2020-03-31 08:55:01,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:11 UTC)" (scheduled at 2020-03-31 08:55:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:55:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:11.440707+00:00 (in 9.996182 seconds) 2020-03-31 08:55:01,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:11 UTC)" (scheduled at 2020-03-31 08:55:01.440707+00:00) 2020-03-31 08:55:01,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:55:01,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:55:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:55:01,466 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 08:55:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:55:01,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:01,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:55:01,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:11 UTC)" executed successfully 2020-03-31 08:55:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:55:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:24.410927+00:00 (in 21.079874 seconds) 2020-03-31 08:55:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:33 UTC)" (scheduled at 2020-03-31 08:55:03.330112+00:00) 2020-03-31 08:55:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:55:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:24.410927+00:00 (in 21.079874 seconds) 2020-03-31 08:55:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:33 UTC)" (scheduled at 2020-03-31 08:55:03.330112+00:00) 2020-03-31 08:55:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 3, 332028), True, datetime.datetime(2020, 3, 31, 8, 55, 3, 332028), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:55:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 3, 332028), True, datetime.datetime(2020, 3, 31, 8, 55, 3, 332028), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:55:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:55:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:55:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:33 UTC)" executed successfully 2020-03-31 08:55:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:55:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:14.323352+00:00 (in 9.998661 seconds) 2020-03-31 08:55:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:14 UTC)" (scheduled at 2020-03-31 08:55:04.323352+00:00) 2020-03-31 08:55:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:55:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:14.323352+00:00 (in 9.998661 seconds) 2020-03-31 08:55:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:14 UTC)" (scheduled at 2020-03-31 08:55:04.323352+00:00) 2020-03-31 08:55:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 4, 325558), True, datetime.datetime(2020, 3, 31, 8, 55, 4, 325558), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:55:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 4, 325558), True, datetime.datetime(2020, 3, 31, 8, 55, 4, 325558), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:55:04,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:04,352 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:55:04,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:04,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:04,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:55:04,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:14 UTC)" executed successfully 2020-03-31 08:55:04,446 [518] [DEBUG] [app] Starting request: urn:request:f337a1d2-df51-461b-b865-e38c0ac8c802 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:04,446 [518] [DEBUG] [app] Starting request: urn:request:f337a1d2-df51-461b-b865-e38c0ac8c802 (/health/instance) 2020-03-31 08:55:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:04,457 [546] [DEBUG] [app] Starting request: urn:request:d61e36b5-d397-474d-977b-b1643163a7f1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:04,457 [546] [DEBUG] [app] Starting request: urn:request:d61e36b5-d397-474d-977b-b1643163a7f1 (/v1/_internal_ping) 2020-03-31 08:55:04,461 [546] [DEBUG] [app] Ending request: urn:request:d61e36b5-d397-474d-977b-b1643163a7f1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:04,461 [546] [DEBUG] [app] Ending request: urn:request:d61e36b5-d397-474d-977b-b1643163a7f1 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:55:04,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:55:04,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:04,462 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 08:55:04,462 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:04,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:04,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:55:04,468 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:55:04,468 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:04,473 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:04,473 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:04,473 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:04,473 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:04,477 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:04,477 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:04,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:04,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:04,497 [518] [DEBUG] [app] Starting request: urn:request:1ceefc68-e074-4df3-84db-400746397334 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:04,497 [518] [DEBUG] [app] Starting request: urn:request:1ceefc68-e074-4df3-84db-400746397334 (/_internal_ping) 2020-03-31 08:55:04,498 [518] [DEBUG] [app] Ending request: urn:request:1ceefc68-e074-4df3-84db-400746397334 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:04,498 [518] [DEBUG] [app] Ending request: urn:request:1ceefc68-e074-4df3-84db-400746397334 (/_internal_ping) 2020-03-31 08:55:04,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:55:04,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:55:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:55:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:04,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:04,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:04,514 [537] [DEBUG] [app] Starting request: urn:request:ac788c8e-d584-4f41-a0f9-8a09f8b6914a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:04,514 [537] [DEBUG] [app] Starting request: urn:request:ac788c8e-d584-4f41-a0f9-8a09f8b6914a (/c1/_internal_ping) 2020-03-31 08:55:04,515 [537] [DEBUG] [app] Ending request: urn:request:ac788c8e-d584-4f41-a0f9-8a09f8b6914a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:04,515 [537] [DEBUG] [app] Ending request: urn:request:ac788c8e-d584-4f41-a0f9-8a09f8b6914a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:04,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:04,515 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:04,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:55:04,515 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:04,517 [518] [DEBUG] [app] Ending request: urn:request:f337a1d2-df51-461b-b865-e38c0ac8c802 (/health/instance) 2020-03-31 08:55:04,517 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:04,517 [518] [DEBUG] [app] Ending request: urn:request:f337a1d2-df51-461b-b865-e38c0ac8c802 (/health/instance) 2020-03-31 08:55:04,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:04,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 08:55:04,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 08:55:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:55:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:16.924497+00:00 (in 9.999239 seconds) 2020-03-31 08:55:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:16 UTC)" (scheduled at 2020-03-31 08:55:06.924497+00:00) 2020-03-31 08:55:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:55:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:16.924497+00:00 (in 9.999239 seconds) 2020-03-31 08:55:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:16 UTC)" (scheduled at 2020-03-31 08:55:06.924497+00:00) 2020-03-31 08:55:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 6, 926283), True, datetime.datetime(2020, 3, 31, 8, 55, 6, 926283), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:55:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 6, 926283), True, datetime.datetime(2020, 3, 31, 8, 55, 6, 926283), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:55:06,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:55:06,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:55:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:16 UTC)" executed successfully 2020-03-31 08:55:08,273 [527] [DEBUG] [app] Starting request: urn:request:c8eead65-2421-4f47-b8df-07f53f090361 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:08,273 [527] [DEBUG] [app] Starting request: urn:request:c8eead65-2421-4f47-b8df-07f53f090361 (/health/instance) 2020-03-31 08:55:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:08,281 [551] [DEBUG] [app] Starting request: urn:request:a8a9273f-38ff-42aa-ba89-c1e5f50ec2ed (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:08,281 [551] [DEBUG] [app] Starting request: urn:request:a8a9273f-38ff-42aa-ba89-c1e5f50ec2ed (/v1/_internal_ping) 2020-03-31 08:55:08,282 [551] [DEBUG] [app] Ending request: urn:request:a8a9273f-38ff-42aa-ba89-c1e5f50ec2ed (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:08,282 [551] [DEBUG] [app] Ending request: urn:request:a8a9273f-38ff-42aa-ba89-c1e5f50ec2ed (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:08,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:55:08,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:08,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:55:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:55:08,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:08,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:08,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:08,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:08,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:08,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:08,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:08,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:08,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:08,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:08,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:08,315 [527] [DEBUG] [app] Starting request: urn:request:92633a35-aa3b-4592-a1ba-bb2e02948be4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:08,315 [527] [DEBUG] [app] Starting request: urn:request:92633a35-aa3b-4592-a1ba-bb2e02948be4 (/_internal_ping) 2020-03-31 08:55:08,316 [527] [DEBUG] [app] Ending request: urn:request:92633a35-aa3b-4592-a1ba-bb2e02948be4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:08,316 [527] [DEBUG] [app] Ending request: urn:request:92633a35-aa3b-4592-a1ba-bb2e02948be4 (/_internal_ping) 2020-03-31 08:55:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:08,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:08,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:08,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:08,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:08,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:08,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:55:08,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:08,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:08,331 [534] [DEBUG] [app] Starting request: urn:request:74139198-ca23-44ce-98bc-70a051ff236d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:08,331 [534] [DEBUG] [app] Starting request: urn:request:74139198-ca23-44ce-98bc-70a051ff236d (/c1/_internal_ping) 2020-03-31 08:55:08,335 [534] [DEBUG] [app] Ending request: urn:request:74139198-ca23-44ce-98bc-70a051ff236d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:08,335 [534] [DEBUG] [app] Ending request: urn:request:74139198-ca23-44ce-98bc-70a051ff236d (/c1/_internal_ping) 2020-03-31 08:55:08,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-web stdout | 2020-03-31 08:55:08,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:08,336 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-verbs stdout | 2020-03-31 08:55:08,336 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:08,338 [527] [DEBUG] [app] Ending request: urn:request:c8eead65-2421-4f47-b8df-07f53f090361 (/health/instance) 2020-03-31 08:55:08,338 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:08,338 [527] [DEBUG] [app] Ending request: urn:request:c8eead65-2421-4f47-b8df-07f53f090361 (/health/instance) 2020-03-31 08:55:08,338 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:08,339 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 08:55:08,339 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 08:55:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:55:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:21.440707+00:00 (in 9.998567 seconds) 2020-03-31 08:55:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:21 UTC)" (scheduled at 2020-03-31 08:55:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:55:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:21.440707+00:00 (in 9.998567 seconds) 2020-03-31 08:55:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:21 UTC)" (scheduled at 2020-03-31 08:55:11.440707+00:00) 2020-03-31 08:55:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:55:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:55:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:55:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:55:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:55:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:21 UTC)" executed successfully 2020-03-31 08:55:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:55:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:24.323352+00:00 (in 9.998859 seconds) 2020-03-31 08:55:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:24 UTC)" (scheduled at 2020-03-31 08:55:14.323352+00:00) 2020-03-31 08:55:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:55:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:24.323352+00:00 (in 9.998859 seconds) 2020-03-31 08:55:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:24 UTC)" (scheduled at 2020-03-31 08:55:14.323352+00:00) 2020-03-31 08:55:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 14, 325466), True, datetime.datetime(2020, 3, 31, 8, 55, 14, 325466), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:55:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 14, 325466), True, datetime.datetime(2020, 3, 31, 8, 55, 14, 325466), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:55:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:55:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:55:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:24 UTC)" executed successfully 2020-03-31 08:55:14,446 [518] [DEBUG] [app] Starting request: urn:request:406aebb2-aecc-4e59-82ab-5436fd465f4c (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:14,446 [518] [DEBUG] [app] Starting request: urn:request:406aebb2-aecc-4e59-82ab-5436fd465f4c (/health/instance) 2020-03-31 08:55:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:14,454 [538] [DEBUG] [app] Starting request: urn:request:a10fcdbd-bba4-4419-abce-e01ea28697af (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:14,454 [538] [DEBUG] [app] Starting request: urn:request:a10fcdbd-bba4-4419-abce-e01ea28697af (/v1/_internal_ping) 2020-03-31 08:55:14,455 [538] [DEBUG] [app] Ending request: urn:request:a10fcdbd-bba4-4419-abce-e01ea28697af (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:14,455 [538] [DEBUG] [app] Ending request: urn:request:a10fcdbd-bba4-4419-abce-e01ea28697af (/v1/_internal_ping) 2020-03-31 08:55:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:55:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:14,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:55:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:55:14,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:14,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:14,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:14,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:14,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:14,486 [525] [DEBUG] [app] Starting request: urn:request:62788c04-6c0b-4d0e-830f-8dfd184ddca9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:14,486 [525] [DEBUG] [app] Starting request: urn:request:62788c04-6c0b-4d0e-830f-8dfd184ddca9 (/_internal_ping) 2020-03-31 08:55:14,487 [525] [DEBUG] [app] Ending request: urn:request:62788c04-6c0b-4d0e-830f-8dfd184ddca9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:14,487 [525] [DEBUG] [app] Ending request: urn:request:62788c04-6c0b-4d0e-830f-8dfd184ddca9 (/_internal_ping) 2020-03-31 08:55:14,488 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:55:14,488 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:55:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:55:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:14,501 [537] [DEBUG] [app] Starting request: urn:request:dc8faf06-91b7-4eca-9807-6d684459f7ce (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:14,501 [537] [DEBUG] [app] Starting request: urn:request:dc8faf06-91b7-4eca-9807-6d684459f7ce (/c1/_internal_ping) 2020-03-31 08:55:14,502 [537] [DEBUG] [app] Ending request: urn:request:dc8faf06-91b7-4eca-9807-6d684459f7ce (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:14,502 [537] [DEBUG] [app] Ending request: urn:request:dc8faf06-91b7-4eca-9807-6d684459f7ce (/c1/_internal_ping) 2020-03-31 08:55:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:14,503 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:14,505 [518] [DEBUG] [app] Ending request: urn:request:406aebb2-aecc-4e59-82ab-5436fd465f4c (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:55:14,503 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:14,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:55:14,505 [518] [DEBUG] [app] Ending request: urn:request:406aebb2-aecc-4e59-82ab-5436fd465f4c (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:14,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 08:55:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:55:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:33.366791+00:00 (in 18.826559 seconds) 2020-03-31 08:55:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:00:14 UTC)" (scheduled at 2020-03-31 08:55:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 08:55:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:33.366791+00:00 (in 18.826559 seconds) 2020-03-31 08:55:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:00:14 UTC)" (scheduled at 2020-03-31 08:55:14.539408+00:00) 2020-03-31 08:55:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 08:55:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), u'exportactionlogs/%']) 2020-03-31 08:55:14,557 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), True, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 08:55:14,557 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), True, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), 0, u'exportactionlogs/%']) 2020-03-31 08:55:14,562 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), True, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 08:55:14,562 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), True, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 8, 55, 14, 541035), u'exportactionlogs/%']) 2020-03-31 08:55:14,566 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 08:55:14,566 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:14,571 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 08:55:14,571 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:14,571 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:00:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:55:14,571 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:00:14 UTC)" executed successfully 2020-03-31 08:55:14,579 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 08:55:14,579 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:14,580 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 08:55:14,580 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 08:55:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:55:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:45.064514+00:00 (in 29.999176 seconds) 2020-03-31 08:55:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:45 UTC)" (scheduled at 2020-03-31 08:55:15.064514+00:00) gcworker stdout | 2020-03-31 08:55:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:45.064514+00:00 (in 29.999176 seconds) 2020-03-31 08:55:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:45 UTC)" (scheduled at 2020-03-31 08:55:15.064514+00:00) 2020-03-31 08:55:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435315, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:55:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435315, 1209600, 500, 1, 0]) 2020-03-31 08:55:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:55:15,082 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 08:55:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:55:15,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:55:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:45 UTC)" executed successfully 2020-03-31 08:55:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:55:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:26.924497+00:00 (in 9.999211 seconds) 2020-03-31 08:55:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:26 UTC)" (scheduled at 2020-03-31 08:55:16.924497+00:00) 2020-03-31 08:55:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:55:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:26.924497+00:00 (in 9.999211 seconds) 2020-03-31 08:55:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:26 UTC)" (scheduled at 2020-03-31 08:55:16.924497+00:00) 2020-03-31 08:55:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 16, 926234), True, datetime.datetime(2020, 3, 31, 8, 55, 16, 926234), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:55:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 16, 926234), True, datetime.datetime(2020, 3, 31, 8, 55, 16, 926234), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:55:16,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:55:16,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:16,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:55:16,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:26 UTC)" executed successfully 2020-03-31 08:55:18,273 [518] [DEBUG] [app] Starting request: urn:request:7dd33248-0095-4517-b06d-fbe375d63c6c (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:18,273 [518] [DEBUG] [app] Starting request: urn:request:7dd33248-0095-4517-b06d-fbe375d63c6c (/health/instance) 2020-03-31 08:55:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:18,278 [544] [DEBUG] [app] Starting request: urn:request:8a803aa5-167c-432a-ad65-52cf6abeb1d0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:18,278 [544] [DEBUG] [app] Starting request: urn:request:8a803aa5-167c-432a-ad65-52cf6abeb1d0 (/v1/_internal_ping) 2020-03-31 08:55:18,280 [544] [DEBUG] [app] Ending request: urn:request:8a803aa5-167c-432a-ad65-52cf6abeb1d0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:18,280 [544] [DEBUG] [app] Ending request: urn:request:8a803aa5-167c-432a-ad65-52cf6abeb1d0 (/v1/_internal_ping) 2020-03-31 08:55:18,280 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:55:18,280 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:18,282 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:55:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:55:18,282 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:18,284 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:18,284 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:18,285 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:18,285 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:18,311 [518] [DEBUG] [app] Starting request: urn:request:d57a56ed-3d34-4196-a83f-6ced10c7b0da (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:18,311 [518] [DEBUG] [app] Starting request: urn:request:d57a56ed-3d34-4196-a83f-6ced10c7b0da (/_internal_ping) 2020-03-31 08:55:18,312 [518] [DEBUG] [app] Ending request: urn:request:d57a56ed-3d34-4196-a83f-6ced10c7b0da (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:18,312 [518] [DEBUG] [app] Ending request: urn:request:d57a56ed-3d34-4196-a83f-6ced10c7b0da (/_internal_ping) 2020-03-31 08:55:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:55:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:55:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:55:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:55:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:55:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:18,325 [534] [DEBUG] [app] Starting request: urn:request:15e7683c-a7b7-4dd3-93a3-2264475547be (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:18,325 [534] [DEBUG] [app] Starting request: urn:request:15e7683c-a7b7-4dd3-93a3-2264475547be (/c1/_internal_ping) 2020-03-31 08:55:18,326 [534] [DEBUG] [app] Ending request: urn:request:15e7683c-a7b7-4dd3-93a3-2264475547be (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:18,326 [534] [DEBUG] [app] Ending request: urn:request:15e7683c-a7b7-4dd3-93a3-2264475547be (/c1/_internal_ping) 2020-03-31 08:55:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:18,327 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:18,328 [518] [DEBUG] [app] Ending request: urn:request:7dd33248-0095-4517-b06d-fbe375d63c6c (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:55:18,327 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:18,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 08:55:18,328 [518] [DEBUG] [app] Ending request: urn:request:7dd33248-0095-4517-b06d-fbe375d63c6c (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:18,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 08:55:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:55:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:31.440707+00:00 (in 9.995898 seconds) 2020-03-31 08:55:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:31 UTC)" (scheduled at 2020-03-31 08:55:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:55:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:31.440707+00:00 (in 9.995898 seconds) 2020-03-31 08:55:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:31 UTC)" (scheduled at 2020-03-31 08:55:21.440707+00:00) 2020-03-31 08:55:21,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:55:21,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:55:21,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:55:21,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:55:21,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:55:21,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:31 UTC)" executed successfully 2020-03-31 08:55:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:55:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:52.113221+00:00 (in 29.997398 seconds) 2020-03-31 08:55:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:52 UTC)" (scheduled at 2020-03-31 08:55:22.113221+00:00) securityworker stdout | 2020-03-31 08:55:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:52.113221+00:00 (in 29.997398 seconds) 2020-03-31 08:55:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:52 UTC)" (scheduled at 2020-03-31 08:55:22.113221+00:00) 2020-03-31 08:55:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:55:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:55:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:55:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:55:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:55:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:55:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:55:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:55:52 UTC)" executed successfully 2020-03-31 08:55:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:55:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:24 UTC)" (scheduled at 2020-03-31 08:55:24.323352+00:00) 2020-03-31 08:55:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:55:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:24 UTC)" (scheduled at 2020-03-31 08:55:24.323352+00:00) 2020-03-31 08:55:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 24, 324823), True, datetime.datetime(2020, 3, 31, 8, 55, 24, 324823), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:55:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 24, 324823), True, datetime.datetime(2020, 3, 31, 8, 55, 24, 324823), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:55:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:34.323352+00:00 (in 9.999266 seconds) notificationworker stdout | 2020-03-31 08:55:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:34.323352+00:00 (in 9.999266 seconds) 2020-03-31 08:55:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:55:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:55:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:34 UTC)" executed successfully 2020-03-31 08:55:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:55:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:33.330112+00:00 (in 8.918357 seconds) 2020-03-31 08:55:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:24 UTC)" (scheduled at 2020-03-31 08:55:24.410927+00:00) 2020-03-31 08:55:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:55:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:33.330112+00:00 (in 8.918357 seconds) 2020-03-31 08:55:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:24 UTC)" (scheduled at 2020-03-31 08:55:24.410927+00:00) 2020-03-31 08:55:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:24 UTC)" executed successfully 2020-03-31 08:55:24,446 [527] [DEBUG] [app] Starting request: urn:request:17ecdc40-119d-48f9-8160-b63f397a3975 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:24,446 [527] [DEBUG] [app] Starting request: urn:request:17ecdc40-119d-48f9-8160-b63f397a3975 (/health/instance) 2020-03-31 08:55:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:24,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:24,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:24,455 [538] [DEBUG] [app] Starting request: urn:request:c09de2cf-fc72-4259-a551-e9278a7e71e8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:24,455 [538] [DEBUG] [app] Starting request: urn:request:c09de2cf-fc72-4259-a551-e9278a7e71e8 (/v1/_internal_ping) 2020-03-31 08:55:24,456 [538] [DEBUG] [app] Ending request: urn:request:c09de2cf-fc72-4259-a551-e9278a7e71e8 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:55:24,456 [538] [DEBUG] [app] Ending request: urn:request:c09de2cf-fc72-4259-a551-e9278a7e71e8 (/v1/_internal_ping) 2020-03-31 08:55:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:24,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:24,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:55:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:24,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:55:24,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:55:24,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:55:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:24,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:24,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:24,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:24,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:24,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:24,488 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:24,488 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:24,501 [521] [DEBUG] [app] Starting request: urn:request:bd0afe65-f655-4da0-9ed3-70d5f158a04a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:24,501 [521] [DEBUG] [app] Starting request: urn:request:bd0afe65-f655-4da0-9ed3-70d5f158a04a (/_internal_ping) 2020-03-31 08:55:24,504 [521] [DEBUG] [app] Ending request: urn:request:bd0afe65-f655-4da0-9ed3-70d5f158a04a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:24,504 [521] [DEBUG] [app] Ending request: urn:request:bd0afe65-f655-4da0-9ed3-70d5f158a04a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.012 159 0.012) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.012 159 0.012) 2020-03-31 08:55:24,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:24,506 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:24,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:24,506 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:24,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:24,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:24,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:24,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:24,514 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:24,514 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:55:24,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:24,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:24,518 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:24,520 [539] [DEBUG] [app] Starting request: urn:request:f3c89b64-5503-4fd2-89d0-afae26847e74 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:24,518 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 08:55:24,520 [539] [DEBUG] [app] Starting request: urn:request:f3c89b64-5503-4fd2-89d0-afae26847e74 (/c1/_internal_ping) 2020-03-31 08:55:24,521 [539] [DEBUG] [app] Ending request: urn:request:f3c89b64-5503-4fd2-89d0-afae26847e74 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:24,521 [539] [DEBUG] [app] Ending request: urn:request:f3c89b64-5503-4fd2-89d0-afae26847e74 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:24,522 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:24,522 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:55:24,522 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:24,522 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:24,524 [527] [DEBUG] [app] Ending request: urn:request:17ecdc40-119d-48f9-8160-b63f397a3975 (/health/instance) 2020-03-31 08:55:24,524 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:24,524 [527] [DEBUG] [app] Ending request: urn:request:17ecdc40-119d-48f9-8160-b63f397a3975 (/health/instance) 2020-03-31 08:55:24,524 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:24,525 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.081 49 0.081) gunicorn-web stdout | 2020-03-31 08:55:24,525 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.081 49 0.081) 2020-03-31 08:55:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:55:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:36.924497+00:00 (in 9.999018 seconds) 2020-03-31 08:55:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:36 UTC)" (scheduled at 2020-03-31 08:55:26.924497+00:00) 2020-03-31 08:55:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:55:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:36.924497+00:00 (in 9.999018 seconds) 2020-03-31 08:55:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:36 UTC)" (scheduled at 2020-03-31 08:55:26.924497+00:00) 2020-03-31 08:55:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 26, 926172), True, datetime.datetime(2020, 3, 31, 8, 55, 26, 926172), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:55:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 26, 926172), True, datetime.datetime(2020, 3, 31, 8, 55, 26, 926172), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:55:26,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:55:26,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:55:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:36 UTC)" executed successfully 2020-03-31 08:55:28,273 [518] [DEBUG] [app] Starting request: urn:request:ba927a15-175b-4106-9d4c-b1553ec133f1 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:28,273 [518] [DEBUG] [app] Starting request: urn:request:ba927a15-175b-4106-9d4c-b1553ec133f1 (/health/instance) 2020-03-31 08:55:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:28,280 [556] [DEBUG] [app] Starting request: urn:request:a42fa27b-5981-4405-8126-10c6be60b7ff (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:28,280 [556] [DEBUG] [app] Starting request: urn:request:a42fa27b-5981-4405-8126-10c6be60b7ff (/v1/_internal_ping) 2020-03-31 08:55:28,281 [556] [DEBUG] [app] Ending request: urn:request:a42fa27b-5981-4405-8126-10c6be60b7ff (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:28,281 [556] [DEBUG] [app] Ending request: urn:request:a42fa27b-5981-4405-8126-10c6be60b7ff (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:28,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:55:28,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:55:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:55:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:28,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:28,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:28,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:28,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:28,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:28,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:28,314 [518] [DEBUG] [app] Starting request: urn:request:d521379f-00eb-49f6-9639-14f51b4c7fb4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:28,314 [518] [DEBUG] [app] Starting request: urn:request:d521379f-00eb-49f6-9639-14f51b4c7fb4 (/_internal_ping) 2020-03-31 08:55:28,315 [518] [DEBUG] [app] Ending request: urn:request:d521379f-00eb-49f6-9639-14f51b4c7fb4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:28,315 [518] [DEBUG] [app] Ending request: urn:request:d521379f-00eb-49f6-9639-14f51b4c7fb4 (/_internal_ping) 2020-03-31 08:55:28,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:28,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:55:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:55:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:55:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:28,328 [543] [DEBUG] [app] Starting request: urn:request:aec8f3b9-c38a-45e4-a44e-200e7f873173 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:28,328 [543] [DEBUG] [app] Starting request: urn:request:aec8f3b9-c38a-45e4-a44e-200e7f873173 (/c1/_internal_ping) 2020-03-31 08:55:28,328 [543] [DEBUG] [app] Ending request: urn:request:aec8f3b9-c38a-45e4-a44e-200e7f873173 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:28,328 [543] [DEBUG] [app] Ending request: urn:request:aec8f3b9-c38a-45e4-a44e-200e7f873173 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:28,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:28,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:28,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:55:28,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:28,330 [518] [DEBUG] [app] Ending request: urn:request:ba927a15-175b-4106-9d4c-b1553ec133f1 (/health/instance) 2020-03-31 08:55:28,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:28,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:55:28,330 [518] [DEBUG] [app] Ending request: urn:request:ba927a15-175b-4106-9d4c-b1553ec133f1 (/health/instance) 2020-03-31 08:55:28,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:28,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:55:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:55:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:00.337699+00:00 (in 29.999358 seconds) 2020-03-31 08:55:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:00 UTC)" (scheduled at 2020-03-31 08:55:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:55:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:00.337699+00:00 (in 29.999358 seconds) 2020-03-31 08:55:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:00 UTC)" (scheduled at 2020-03-31 08:55:30.337699+00:00) 2020-03-31 08:55:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 55, 30, 339066), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:55:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 55, 30, 339066), False, 50, 1, 0]) 2020-03-31 08:55:30,353 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:55:30,353 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:55:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:55:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:00 UTC)" executed successfully 2020-03-31 08:55:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:55:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:41.440707+00:00 (in 9.997243 seconds) 2020-03-31 08:55:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:41 UTC)" (scheduled at 2020-03-31 08:55:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:55:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:41.440707+00:00 (in 9.997243 seconds) 2020-03-31 08:55:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:41 UTC)" (scheduled at 2020-03-31 08:55:31.440707+00:00) 2020-03-31 08:55:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:55:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:55:31,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:55:31,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:55:31,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:31,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:55:31,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:31,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:41 UTC)" executed successfully 2020-03-31 08:55:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:55:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:03.330112+00:00 (in 29.999163 seconds) 2020-03-31 08:55:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:03 UTC)" (scheduled at 2020-03-31 08:55:33.330112+00:00) 2020-03-31 08:55:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 33, 331735), True, datetime.datetime(2020, 3, 31, 8, 55, 33, 331735), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:55:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:03.330112+00:00 (in 29.999163 seconds) 2020-03-31 08:55:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:03 UTC)" (scheduled at 2020-03-31 08:55:33.330112+00:00) 2020-03-31 08:55:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:55:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 33, 331735), True, datetime.datetime(2020, 3, 31, 8, 55, 33, 331735), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:55:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:55:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:55:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:03 UTC)" executed successfully 2020-03-31 08:55:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:55:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:43.539999+00:00 (in 10.172500 seconds) 2020-03-31 08:55:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:33 UTC)" (scheduled at 2020-03-31 08:55:33.366791+00:00) 2020-03-31 08:55:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:55:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:43.539999+00:00 (in 10.172500 seconds) 2020-03-31 08:55:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:33 UTC)" (scheduled at 2020-03-31 08:55:33.366791+00:00) 2020-03-31 08:55:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 33, 368081), True, datetime.datetime(2020, 3, 31, 8, 55, 33, 368081), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:55:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 33, 368081), True, datetime.datetime(2020, 3, 31, 8, 55, 33, 368081), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:55:33,386 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:55:33,386 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:55:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:33 UTC)" executed successfully 2020-03-31 08:55:34,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:34,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:44.323352+00:00 (in 9.995830 seconds) 2020-03-31 08:55:34,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:44 UTC)" (scheduled at 2020-03-31 08:55:34.323352+00:00) 2020-03-31 08:55:34,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:55:34,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:55:34,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:44.323352+00:00 (in 9.995830 seconds) 2020-03-31 08:55:34,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:44 UTC)" (scheduled at 2020-03-31 08:55:34.323352+00:00) 2020-03-31 08:55:34,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:34,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 34, 328367), True, datetime.datetime(2020, 3, 31, 8, 55, 34, 328367), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:55:34,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 34, 328367), True, datetime.datetime(2020, 3, 31, 8, 55, 34, 328367), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:55:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:55:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:55:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:44 UTC)" executed successfully 2020-03-31 08:55:34,446 [527] [DEBUG] [app] Starting request: urn:request:3ea48fd4-8111-4f40-893a-be2fd2a9173c (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:34,446 [527] [DEBUG] [app] Starting request: urn:request:3ea48fd4-8111-4f40-893a-be2fd2a9173c (/health/instance) 2020-03-31 08:55:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:34,454 [538] [DEBUG] [app] Starting request: urn:request:bb21c9c7-f80a-455a-b013-6286edf01800 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:34,454 [538] [DEBUG] [app] Starting request: urn:request:bb21c9c7-f80a-455a-b013-6286edf01800 (/v1/_internal_ping) 2020-03-31 08:55:34,455 [538] [DEBUG] [app] Ending request: urn:request:bb21c9c7-f80a-455a-b013-6286edf01800 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:34,455 [538] [DEBUG] [app] Ending request: urn:request:bb21c9c7-f80a-455a-b013-6286edf01800 (/v1/_internal_ping) 2020-03-31 08:55:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:55:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:34,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:34,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:34,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:55:34,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:55:34,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:55:34,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:34,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:34,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:34,463 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:34,463 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:34,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:34,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:34,488 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:34,488 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:34,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:34,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:34,494 [527] [DEBUG] [app] Starting request: urn:request:c84bd5ca-3e9d-48e8-8047-0e0f0a9bf57c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:34,494 [527] [DEBUG] [app] Starting request: urn:request:c84bd5ca-3e9d-48e8-8047-0e0f0a9bf57c (/_internal_ping) 2020-03-31 08:55:34,495 [527] [DEBUG] [app] Ending request: urn:request:c84bd5ca-3e9d-48e8-8047-0e0f0a9bf57c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:34,495 [527] [DEBUG] [app] Ending request: urn:request:c84bd5ca-3e9d-48e8-8047-0e0f0a9bf57c (/_internal_ping) 2020-03-31 08:55:34,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:55:34,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:55:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:34,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:34,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:55:34,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:34,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:34,648 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:34,648 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:34,650 [537] [DEBUG] [app] Starting request: urn:request:48ef1bb2-a9ce-4ed4-9f56-7193e21c73bc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:34,650 [537] [DEBUG] [app] Starting request: urn:request:48ef1bb2-a9ce-4ed4-9f56-7193e21c73bc (/c1/_internal_ping) 2020-03-31 08:55:34,651 [537] [DEBUG] [app] Ending request: urn:request:48ef1bb2-a9ce-4ed4-9f56-7193e21c73bc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:34,651 [537] [DEBUG] [app] Ending request: urn:request:48ef1bb2-a9ce-4ed4-9f56-7193e21c73bc (/c1/_internal_ping) 2020-03-31 08:55:34,652 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:34,651 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:34,652 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:34,653 [527] [DEBUG] [app] Ending request: urn:request:3ea48fd4-8111-4f40-893a-be2fd2a9173c (/health/instance) 2020-03-31 08:55:34,654 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:55:34,651 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:34,654 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.210 49 0.210) gunicorn-web stdout | 2020-03-31 08:55:34,653 [527] [DEBUG] [app] Ending request: urn:request:3ea48fd4-8111-4f40-893a-be2fd2a9173c (/health/instance) 2020-03-31 08:55:34,654 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:34,654 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.210 49 0.210) 2020-03-31 08:55:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:55:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:38.010934+00:00 (in 1.085654 seconds) storagereplication stdout | 2020-03-31 08:55:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:38.010934+00:00 (in 1.085654 seconds) 2020-03-31 08:55:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:46 UTC)" (scheduled at 2020-03-31 08:55:36.924497+00:00) 2020-03-31 08:55:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:55:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:46 UTC)" (scheduled at 2020-03-31 08:55:36.924497+00:00) 2020-03-31 08:55:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 36, 926551), True, datetime.datetime(2020, 3, 31, 8, 55, 36, 926551), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:55:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 36, 926551), True, datetime.datetime(2020, 3, 31, 8, 55, 36, 926551), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:55:36,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:55:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:55:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:46 UTC)" executed successfully 2020-03-31 08:55:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:55:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:46.924497+00:00 (in 8.912802 seconds) 2020-03-31 08:55:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:38 UTC)" (scheduled at 2020-03-31 08:55:38.010934+00:00) 2020-03-31 08:55:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 08:55:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:46.924497+00:00 (in 8.912802 seconds) 2020-03-31 08:55:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:38 UTC)" (scheduled at 2020-03-31 08:55:38.010934+00:00) 2020-03-31 08:55:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:55:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:38 UTC)" executed successfully 2020-03-31 08:55:38,273 [518] [DEBUG] [app] Starting request: urn:request:13ce6a66-d59a-42d4-9047-164b40ed7d30 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:38,273 [518] [DEBUG] [app] Starting request: urn:request:13ce6a66-d59a-42d4-9047-164b40ed7d30 (/health/instance) 2020-03-31 08:55:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:38,280 [546] [DEBUG] [app] Starting request: urn:request:527e3c60-ad79-4b19-9456-cef34e88d527 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:38,280 [546] [DEBUG] [app] Starting request: urn:request:527e3c60-ad79-4b19-9456-cef34e88d527 (/v1/_internal_ping) 2020-03-31 08:55:38,281 [546] [DEBUG] [app] Ending request: urn:request:527e3c60-ad79-4b19-9456-cef34e88d527 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:38,281 [546] [DEBUG] [app] Ending request: urn:request:527e3c60-ad79-4b19-9456-cef34e88d527 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:38,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:55:38,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:55:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:55:38,284 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:55:38,284 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:38,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:38,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:38,287 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:38,287 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:38,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:38,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:38,311 [518] [DEBUG] [app] Starting request: urn:request:24fbf967-e9f9-4b3c-93e8-31e15de8fedf (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:38,311 [518] [DEBUG] [app] Starting request: urn:request:24fbf967-e9f9-4b3c-93e8-31e15de8fedf (/_internal_ping) 2020-03-31 08:55:38,312 [518] [DEBUG] [app] Ending request: urn:request:24fbf967-e9f9-4b3c-93e8-31e15de8fedf (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:38,312 [518] [DEBUG] [app] Ending request: urn:request:24fbf967-e9f9-4b3c-93e8-31e15de8fedf (/_internal_ping) 2020-03-31 08:55:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:55:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:55:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:38,324 [534] [DEBUG] [app] Starting request: urn:request:9899ab4e-e89e-4d89-ae53-25ded575cf05 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:38,324 [534] [DEBUG] [app] Starting request: urn:request:9899ab4e-e89e-4d89-ae53-25ded575cf05 (/c1/_internal_ping) 2020-03-31 08:55:38,325 [534] [DEBUG] [app] Ending request: urn:request:9899ab4e-e89e-4d89-ae53-25ded575cf05 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:38,325 [534] [DEBUG] [app] Ending request: urn:request:9899ab4e-e89e-4d89-ae53-25ded575cf05 (/c1/_internal_ping) 2020-03-31 08:55:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:38,326 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:38,327 [518] [DEBUG] [app] Ending request: urn:request:13ce6a66-d59a-42d4-9047-164b40ed7d30 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:55:38,326 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:38,328 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 08:55:38,327 [518] [DEBUG] [app] Ending request: urn:request:13ce6a66-d59a-42d4-9047-164b40ed7d30 (/health/instance) 2020-03-31 08:55:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:55:38,328 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 08:55:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:55:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:55:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:51.440707+00:00 (in 9.998995 seconds) 2020-03-31 08:55:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:51 UTC)" (scheduled at 2020-03-31 08:55:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:55:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:51.440707+00:00 (in 9.998995 seconds) 2020-03-31 08:55:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:51 UTC)" (scheduled at 2020-03-31 08:55:41.440707+00:00) 2020-03-31 08:55:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:55:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:55:41,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:55:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:55:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:41,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:55:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:41,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:51 UTC)" executed successfully 2020-03-31 08:55:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:55:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:43,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:33.366791+00:00 (in 49.826133 seconds) 2020-03-31 08:55:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:43 UTC)" (scheduled at 2020-03-31 08:55:43.539999+00:00) 2020-03-31 08:55:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:55:43,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:33.366791+00:00 (in 49.826133 seconds) 2020-03-31 08:55:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:43 UTC)" (scheduled at 2020-03-31 08:55:43.539999+00:00) 2020-03-31 08:55:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:43 UTC)" executed successfully 2020-03-31 08:55:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:55:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:49.498966+00:00 (in 5.174355 seconds) 2020-03-31 08:55:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:54 UTC)" (scheduled at 2020-03-31 08:55:44.323352+00:00) 2020-03-31 08:55:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:55:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:49.498966+00:00 (in 5.174355 seconds) 2020-03-31 08:55:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:54 UTC)" (scheduled at 2020-03-31 08:55:44.323352+00:00) 2020-03-31 08:55:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 44, 325697), True, datetime.datetime(2020, 3, 31, 8, 55, 44, 325697), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:55:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 44, 325697), True, datetime.datetime(2020, 3, 31, 8, 55, 44, 325697), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:55:44,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:55:44,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:55:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:54 UTC)" executed successfully 2020-03-31 08:55:44,446 [527] [DEBUG] [app] Starting request: urn:request:ab70930b-2628-42be-96bb-3a7e9a1fae87 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:44,446 [527] [DEBUG] [app] Starting request: urn:request:ab70930b-2628-42be-96bb-3a7e9a1fae87 (/health/instance) 2020-03-31 08:55:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:44,454 [549] [DEBUG] [app] Starting request: urn:request:dfc8236e-52ef-448d-b178-9446a592f577 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:44,454 [549] [DEBUG] [app] Starting request: urn:request:dfc8236e-52ef-448d-b178-9446a592f577 (/v1/_internal_ping) 2020-03-31 08:55:44,455 [549] [DEBUG] [app] Ending request: urn:request:dfc8236e-52ef-448d-b178-9446a592f577 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:44,455 [549] [DEBUG] [app] Ending request: urn:request:dfc8236e-52ef-448d-b178-9446a592f577 (/v1/_internal_ping) 2020-03-31 08:55:44,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:55:44,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:55:44,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:55:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:55:44,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:44,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:44,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:44,461 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:44,461 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:44,486 [527] [DEBUG] [app] Starting request: urn:request:617f175a-19ff-4fe1-bc8b-2e829fdfc16e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:44,486 [527] [DEBUG] [app] Starting request: urn:request:617f175a-19ff-4fe1-bc8b-2e829fdfc16e (/_internal_ping) 2020-03-31 08:55:44,487 [527] [DEBUG] [app] Ending request: urn:request:617f175a-19ff-4fe1-bc8b-2e829fdfc16e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:44,487 [527] [DEBUG] [app] Ending request: urn:request:617f175a-19ff-4fe1-bc8b-2e829fdfc16e (/_internal_ping) 2020-03-31 08:55:44,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:44,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:44,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:44,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:55:44,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:44,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:44,502 [536] [DEBUG] [app] Starting request: urn:request:3310ab81-5165-4ada-98c0-8a726da79fef (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:44,502 [536] [DEBUG] [app] Starting request: urn:request:3310ab81-5165-4ada-98c0-8a726da79fef (/c1/_internal_ping) 2020-03-31 08:55:44,506 [536] [DEBUG] [app] Ending request: urn:request:3310ab81-5165-4ada-98c0-8a726da79fef (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:44,506 [536] [DEBUG] [app] Ending request: urn:request:3310ab81-5165-4ada-98c0-8a726da79fef (/c1/_internal_ping) 2020-03-31 08:55:44,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) gunicorn-web stdout | 2020-03-31 08:55:44,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:44,507 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) gunicorn-verbs stdout | 2020-03-31 08:55:44,507 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:44,508 [527] [DEBUG] [app] Ending request: urn:request:ab70930b-2628-42be-96bb-3a7e9a1fae87 (/health/instance) 2020-03-31 08:55:44,508 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:44,508 [527] [DEBUG] [app] Ending request: urn:request:ab70930b-2628-42be-96bb-3a7e9a1fae87 (/health/instance) 2020-03-31 08:55:44,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:44,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:55:44,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:55:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:55:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:15.064514+00:00 (in 29.999081 seconds) 2020-03-31 08:55:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:15 UTC)" (scheduled at 2020-03-31 08:55:45.064514+00:00) gcworker stdout | 2020-03-31 08:55:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:15.064514+00:00 (in 29.999081 seconds) 2020-03-31 08:55:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:15 UTC)" (scheduled at 2020-03-31 08:55:45.064514+00:00) 2020-03-31 08:55:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435345, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:55:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435345, 1209600, 500, 1, 0]) 2020-03-31 08:55:45,081 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:55:45,081 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:55:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:55:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:15 UTC)" executed successfully 2020-03-31 08:55:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:55:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:56.924497+00:00 (in 9.998841 seconds) 2020-03-31 08:55:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:56 UTC)" (scheduled at 2020-03-31 08:55:46.924497+00:00) 2020-03-31 08:55:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:55:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:56.924497+00:00 (in 9.998841 seconds) 2020-03-31 08:55:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:56 UTC)" (scheduled at 2020-03-31 08:55:46.924497+00:00) 2020-03-31 08:55:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 46, 926615), True, datetime.datetime(2020, 3, 31, 8, 55, 46, 926615), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:55:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 46, 926615), True, datetime.datetime(2020, 3, 31, 8, 55, 46, 926615), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:55:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:55:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:55:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:55:56 UTC)" executed successfully 2020-03-31 08:55:48,274 [527] [DEBUG] [app] Starting request: urn:request:18b3f471-aa5e-4ece-9826-38845ba0edc2 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:48,274 [527] [DEBUG] [app] Starting request: urn:request:18b3f471-aa5e-4ece-9826-38845ba0edc2 (/health/instance) 2020-03-31 08:55:48,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:48,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:48,279 [548] [DEBUG] [app] Starting request: urn:request:44e3d778-9dc5-44b0-a1c2-edf75f7d1765 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:48,279 [548] [DEBUG] [app] Starting request: urn:request:44e3d778-9dc5-44b0-a1c2-edf75f7d1765 (/v1/_internal_ping) 2020-03-31 08:55:48,280 [548] [DEBUG] [app] Ending request: urn:request:44e3d778-9dc5-44b0-a1c2-edf75f7d1765 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:48,280 [548] [DEBUG] [app] Ending request: urn:request:44e3d778-9dc5-44b0-a1c2-edf75f7d1765 (/v1/_internal_ping) 2020-03-31 08:55:48,281 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:48,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:55:48,281 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:48,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:48,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 08:55:48,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:55:48,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:55:48,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:48,285 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:48,285 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:48,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:48,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:48,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:48,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:48,313 [519] [DEBUG] [app] Starting request: urn:request:66c0aaaf-cfdb-42e3-be5c-d9cc3fde6fe2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:48,313 [519] [DEBUG] [app] Starting request: urn:request:66c0aaaf-cfdb-42e3-be5c-d9cc3fde6fe2 (/_internal_ping) 2020-03-31 08:55:48,314 [519] [DEBUG] [app] Ending request: urn:request:66c0aaaf-cfdb-42e3-be5c-d9cc3fde6fe2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:48,314 [519] [DEBUG] [app] Ending request: urn:request:66c0aaaf-cfdb-42e3-be5c-d9cc3fde6fe2 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:55:48,314 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:55:48,314 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:55:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:55:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:55:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:48,326 [539] [DEBUG] [app] Starting request: urn:request:f16a08e9-fac1-4d8c-8627-7cf880bbce52 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:48,326 [539] [DEBUG] [app] Starting request: urn:request:f16a08e9-fac1-4d8c-8627-7cf880bbce52 (/c1/_internal_ping) 2020-03-31 08:55:48,327 [539] [DEBUG] [app] Ending request: urn:request:f16a08e9-fac1-4d8c-8627-7cf880bbce52 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:48,327 [539] [DEBUG] [app] Ending request: urn:request:f16a08e9-fac1-4d8c-8627-7cf880bbce52 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:55:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:48,330 [527] [DEBUG] [app] Ending request: urn:request:18b3f471-aa5e-4ece-9826-38845ba0edc2 (/health/instance) 2020-03-31 08:55:48,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:48,330 [527] [DEBUG] [app] Ending request: urn:request:18b3f471-aa5e-4ece-9826-38845ba0edc2 (/health/instance) 2020-03-31 08:55:48,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:48,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 08:55:48,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 08:55:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:55:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:54.323352+00:00 (in 4.823298 seconds) 2020-03-31 08:55:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:49 UTC)" (scheduled at 2020-03-31 08:55:49.498966+00:00) 2020-03-31 08:55:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:55:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:54.323352+00:00 (in 4.823298 seconds) 2020-03-31 08:55:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:49 UTC)" (scheduled at 2020-03-31 08:55:49.498966+00:00) 2020-03-31 08:55:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:49 UTC)" executed successfully 2020-03-31 08:55:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:55:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:51 UTC)" (scheduled at 2020-03-31 08:55:51.110610+00:00) 2020-03-31 08:55:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:59.560326+00:00 (in 8.448910 seconds) 2020-03-31 08:55:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:55:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:51 UTC)" (scheduled at 2020-03-31 08:55:51.110610+00:00) 2020-03-31 08:55:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:55:59.560326+00:00 (in 8.448910 seconds) 2020-03-31 08:55:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 51, 112540), True, datetime.datetime(2020, 3, 31, 8, 55, 51, 112540), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:55:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 51, 112540), True, datetime.datetime(2020, 3, 31, 8, 55, 51, 112540), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:55:51,135 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:51,135 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:51,135 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:55:51,135 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:51,135 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:51,135 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:51 UTC)" executed successfully 2020-03-31 08:55:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:55:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:01.440707+00:00 (in 9.999021 seconds) 2020-03-31 08:55:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:01 UTC)" (scheduled at 2020-03-31 08:55:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:55:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:01.440707+00:00 (in 9.999021 seconds) 2020-03-31 08:55:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:01 UTC)" (scheduled at 2020-03-31 08:55:51.440707+00:00) 2020-03-31 08:55:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:55:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:55:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:55:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:55:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:55:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:01 UTC)" executed successfully 2020-03-31 08:55:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:55:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:22.113221+00:00 (in 29.998802 seconds) 2020-03-31 08:55:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:22 UTC)" (scheduled at 2020-03-31 08:55:52.113221+00:00) securityworker stdout | 2020-03-31 08:55:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:22.113221+00:00 (in 29.998802 seconds) 2020-03-31 08:55:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:22 UTC)" (scheduled at 2020-03-31 08:55:52.113221+00:00) 2020-03-31 08:55:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:55:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:55:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:55:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:55:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:55:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:55:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:55:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:22 UTC)" executed successfully 2020-03-31 08:55:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:55:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:04.323352+00:00 (in 9.998822 seconds) 2020-03-31 08:55:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:04 UTC)" (scheduled at 2020-03-31 08:55:54.323352+00:00) 2020-03-31 08:55:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:55:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:04.323352+00:00 (in 9.998822 seconds) 2020-03-31 08:55:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:04 UTC)" (scheduled at 2020-03-31 08:55:54.323352+00:00) 2020-03-31 08:55:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 54, 325429), True, datetime.datetime(2020, 3, 31, 8, 55, 54, 325429), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:55:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 54, 325429), True, datetime.datetime(2020, 3, 31, 8, 55, 54, 325429), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:55:54,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:55:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:55:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:04 UTC)" executed successfully 2020-03-31 08:55:54,446 [518] [DEBUG] [app] Starting request: urn:request:b93794ae-55d0-4df3-91dd-b826b816df42 (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:54,446 [518] [DEBUG] [app] Starting request: urn:request:b93794ae-55d0-4df3-91dd-b826b816df42 (/health/instance) 2020-03-31 08:55:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:54,455 [549] [DEBUG] [app] Starting request: urn:request:fdc4568d-593a-440d-9097-8a764b3e5bc2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:54,455 [549] [DEBUG] [app] Starting request: urn:request:fdc4568d-593a-440d-9097-8a764b3e5bc2 (/v1/_internal_ping) 2020-03-31 08:55:54,456 [549] [DEBUG] [app] Ending request: urn:request:fdc4568d-593a-440d-9097-8a764b3e5bc2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:54,456 [549] [DEBUG] [app] Ending request: urn:request:fdc4568d-593a-440d-9097-8a764b3e5bc2 (/v1/_internal_ping) 2020-03-31 08:55:54,457 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:55:54,457 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:54,460 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:55:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:55:54,460 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:54,461 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:54,461 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:54,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:54,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:54,486 [518] [DEBUG] [app] Starting request: urn:request:71c1ead2-ac46-4f83-a21a-f3e184869396 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:54,486 [518] [DEBUG] [app] Starting request: urn:request:71c1ead2-ac46-4f83-a21a-f3e184869396 (/_internal_ping) 2020-03-31 08:55:54,487 [518] [DEBUG] [app] Ending request: urn:request:71c1ead2-ac46-4f83-a21a-f3e184869396 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:54,487 [518] [DEBUG] [app] Ending request: urn:request:71c1ead2-ac46-4f83-a21a-f3e184869396 (/_internal_ping) 2020-03-31 08:55:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:55:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:55:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:54,499 [539] [DEBUG] [app] Starting request: urn:request:c7d18824-d709-4658-b4bb-f2bebac651f7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:54,499 [539] [DEBUG] [app] Starting request: urn:request:c7d18824-d709-4658-b4bb-f2bebac651f7 (/c1/_internal_ping) 2020-03-31 08:55:54,500 [539] [DEBUG] [app] Ending request: urn:request:c7d18824-d709-4658-b4bb-f2bebac651f7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:54,500 [539] [DEBUG] [app] Ending request: urn:request:c7d18824-d709-4658-b4bb-f2bebac651f7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:54,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:55:54,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:54,502 [518] [DEBUG] [app] Ending request: urn:request:b93794ae-55d0-4df3-91dd-b826b816df42 (/health/instance) 2020-03-31 08:55:54,502 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 08:55:54,502 [518] [DEBUG] [app] Ending request: urn:request:b93794ae-55d0-4df3-91dd-b826b816df42 (/health/instance) 2020-03-31 08:55:54,502 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 08:55:54,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:55:54,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:55:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:06.924497+00:00 (in 9.998886 seconds) 2020-03-31 08:55:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:06 UTC)" (scheduled at 2020-03-31 08:55:56.924497+00:00) 2020-03-31 08:55:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:55:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:06.924497+00:00 (in 9.998886 seconds) 2020-03-31 08:55:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:06 UTC)" (scheduled at 2020-03-31 08:55:56.924497+00:00) 2020-03-31 08:55:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:55:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 56, 926408), True, datetime.datetime(2020, 3, 31, 8, 55, 56, 926408), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:55:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 55, 56, 926408), True, datetime.datetime(2020, 3, 31, 8, 55, 56, 926408), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:55:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:55:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:55:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:55:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:06 UTC)" executed successfully 2020-03-31 08:55:58,273 [527] [DEBUG] [app] Starting request: urn:request:e52a88da-7962-497c-9fa7-a603d916d65f (/health/instance) gunicorn-web stdout | 2020-03-31 08:55:58,273 [527] [DEBUG] [app] Starting request: urn:request:e52a88da-7962-497c-9fa7-a603d916d65f (/health/instance) 2020-03-31 08:55:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:58,281 [538] [DEBUG] [app] Starting request: urn:request:26ffc11a-2fe0-4fae-82d1-340722c5f9f7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:58,281 [538] [DEBUG] [app] Starting request: urn:request:26ffc11a-2fe0-4fae-82d1-340722c5f9f7 (/v1/_internal_ping) 2020-03-31 08:55:58,282 [538] [DEBUG] [app] Ending request: urn:request:26ffc11a-2fe0-4fae-82d1-340722c5f9f7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:55:58,282 [538] [DEBUG] [app] Ending request: urn:request:26ffc11a-2fe0-4fae-82d1-340722c5f9f7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:55:58,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:55:58,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:55:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:55:58,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:55:58,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:55:58,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:55:58,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:55:58,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:55:58,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:55:58,305 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:55:58,305 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:55:58,327 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:55:58,327 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:55:58,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:58,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:58,333 [527] [DEBUG] [app] Starting request: urn:request:ce71d6c9-ae9c-4f11-8a99-b15d52b1092f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:58,333 [527] [DEBUG] [app] Starting request: urn:request:ce71d6c9-ae9c-4f11-8a99-b15d52b1092f (/_internal_ping) 2020-03-31 08:55:58,334 [527] [DEBUG] [app] Ending request: urn:request:ce71d6c9-ae9c-4f11-8a99-b15d52b1092f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:55:58,334 [527] [DEBUG] [app] Ending request: urn:request:ce71d6c9-ae9c-4f11-8a99-b15d52b1092f (/_internal_ping) 2020-03-31 08:55:58,334 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:58,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:55:58,334 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:58,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:55:58,339 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:58,339 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:58,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:55:58,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:55:58,343 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:58,343 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:55:58,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:55:58,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:55:58,348 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:55:58,348 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:55:58,349 [536] [DEBUG] [app] Starting request: urn:request:99eb3ccf-df4c-4519-b6fd-6d1b54ede9fb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:58,349 [536] [DEBUG] [app] Starting request: urn:request:99eb3ccf-df4c-4519-b6fd-6d1b54ede9fb (/c1/_internal_ping) 2020-03-31 08:55:58,350 [536] [DEBUG] [app] Ending request: urn:request:99eb3ccf-df4c-4519-b6fd-6d1b54ede9fb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:55:58,350 [536] [DEBUG] [app] Ending request: urn:request:99eb3ccf-df4c-4519-b6fd-6d1b54ede9fb (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:55:58,351 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:55:58,351 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:55:58,351 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:55:58,351 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:55:58,353 [527] [DEBUG] [app] Ending request: urn:request:e52a88da-7962-497c-9fa7-a603d916d65f (/health/instance) 2020-03-31 08:55:58,353 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:55:58,353 [527] [DEBUG] [app] Ending request: urn:request:e52a88da-7962-497c-9fa7-a603d916d65f (/health/instance) 2020-03-31 08:55:58,353 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:55:58,354 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) gunicorn-web stdout | 2020-03-31 08:55:58,354 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:55:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:55:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) 2020-03-31 08:55:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:55:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:55:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:51.110610+00:00 (in 51.548926 seconds) 2020-03-31 08:55:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:59 UTC)" (scheduled at 2020-03-31 08:55:59.560326+00:00) 2020-03-31 08:55:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:55:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:51.110610+00:00 (in 51.548926 seconds) 2020-03-31 08:55:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:59 UTC)" (scheduled at 2020-03-31 08:55:59.560326+00:00) 2020-03-31 08:55:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:55:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:56:59 UTC)" executed successfully 2020-03-31 08:56:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:56:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:30.337699+00:00 (in 29.998994 seconds) 2020-03-31 08:56:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:30 UTC)" (scheduled at 2020-03-31 08:56:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:56:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:30.337699+00:00 (in 29.998994 seconds) 2020-03-31 08:56:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:30 UTC)" (scheduled at 2020-03-31 08:56:00.337699+00:00) 2020-03-31 08:56:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 56, 0, 339660), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:56:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 56, 0, 339660), False, 50, 1, 0]) 2020-03-31 08:56:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:56:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:56:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:56:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:30 UTC)" executed successfully 2020-03-31 08:56:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:56:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:01 UTC)" (scheduled at 2020-03-31 08:56:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:56:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:01 UTC)" (scheduled at 2020-03-31 08:56:01.440707+00:00) 2020-03-31 08:56:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:56:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:56:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:11.440707+00:00 (in 9.998986 seconds) repositoryactioncounter stdout | 2020-03-31 08:56:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:11.440707+00:00 (in 9.998986 seconds) 2020-03-31 08:56:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:56:01,465 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 08:56:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:56:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:56:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:11 UTC)" executed successfully 2020-03-31 08:56:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:56:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:24.410927+00:00 (in 21.080052 seconds) 2020-03-31 08:56:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:33 UTC)" (scheduled at 2020-03-31 08:56:03.330112+00:00) 2020-03-31 08:56:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:56:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:24.410927+00:00 (in 21.080052 seconds) 2020-03-31 08:56:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:33 UTC)" (scheduled at 2020-03-31 08:56:03.330112+00:00) 2020-03-31 08:56:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 3, 331752), True, datetime.datetime(2020, 3, 31, 8, 56, 3, 331752), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:56:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 3, 331752), True, datetime.datetime(2020, 3, 31, 8, 56, 3, 331752), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:56:03,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:56:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:56:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:33 UTC)" executed successfully 2020-03-31 08:56:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:56:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:14.323352+00:00 (in 9.999205 seconds) 2020-03-31 08:56:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:14 UTC)" (scheduled at 2020-03-31 08:56:04.323352+00:00) 2020-03-31 08:56:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:56:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:14.323352+00:00 (in 9.999205 seconds) 2020-03-31 08:56:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:14 UTC)" (scheduled at 2020-03-31 08:56:04.323352+00:00) 2020-03-31 08:56:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 4, 325242), True, datetime.datetime(2020, 3, 31, 8, 56, 4, 325242), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:56:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 4, 325242), True, datetime.datetime(2020, 3, 31, 8, 56, 4, 325242), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:56:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:56:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:56:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:14 UTC)" executed successfully 2020-03-31 08:56:04,446 [526] [DEBUG] [app] Starting request: urn:request:aefc99b9-368f-46e3-9d04-cae0766c9879 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:04,446 [526] [DEBUG] [app] Starting request: urn:request:aefc99b9-368f-46e3-9d04-cae0766c9879 (/health/instance) 2020-03-31 08:56:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:04,454 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:04,454 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:04,455 [551] [DEBUG] [app] Starting request: urn:request:e5718842-12a1-4d99-b12c-df8ba81b1226 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:04,455 [551] [DEBUG] [app] Starting request: urn:request:e5718842-12a1-4d99-b12c-df8ba81b1226 (/v1/_internal_ping) 2020-03-31 08:56:04,457 [551] [DEBUG] [app] Ending request: urn:request:e5718842-12a1-4d99-b12c-df8ba81b1226 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:04,457 [551] [DEBUG] [app] Ending request: urn:request:e5718842-12a1-4d99-b12c-df8ba81b1226 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:04,457 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 08:56:04,457 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:04,458 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:04,458 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:04,459 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:04,459 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:04,461 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:56:04,461 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:04,463 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:04,463 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:04,463 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:04,463 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:04,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:04,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:04,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:04,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:04,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:04,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:04,492 [527] [DEBUG] [app] Starting request: urn:request:cb67f2e4-91d9-47f2-8790-ede50db074ba (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:04,492 [527] [DEBUG] [app] Starting request: urn:request:cb67f2e4-91d9-47f2-8790-ede50db074ba (/_internal_ping) 2020-03-31 08:56:04,493 [527] [DEBUG] [app] Ending request: urn:request:cb67f2e4-91d9-47f2-8790-ede50db074ba (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:04,493 [527] [DEBUG] [app] Ending request: urn:request:cb67f2e4-91d9-47f2-8790-ede50db074ba (/_internal_ping) 2020-03-31 08:56:04,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:56:04,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:56:04,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:04,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:04,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:56:04,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:04,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:04,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:56:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:04,506 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:04,506 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:04,508 [536] [DEBUG] [app] Starting request: urn:request:bd423f94-40da-4ad1-b502-03a021b5d6c0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:04,508 [536] [DEBUG] [app] Starting request: urn:request:bd423f94-40da-4ad1-b502-03a021b5d6c0 (/c1/_internal_ping) 2020-03-31 08:56:04,509 [536] [DEBUG] [app] Ending request: urn:request:bd423f94-40da-4ad1-b502-03a021b5d6c0 (/c1/_internal_ping) 2020-03-31 08:56:04,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:04,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:04,511 [526] [DEBUG] [app] Ending request: urn:request:aefc99b9-368f-46e3-9d04-cae0766c9879 (/health/instance) 2020-03-31 08:56:04,511 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:56:04,509 [536] [DEBUG] [app] Ending request: urn:request:bd423f94-40da-4ad1-b502-03a021b5d6c0 (/c1/_internal_ping) 2020-03-31 08:56:04,512 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 08:56:04,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:56:04,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:04,511 [526] [DEBUG] [app] Ending request: urn:request:aefc99b9-368f-46e3-9d04-cae0766c9879 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:04,511 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:04,512 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 08:56:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:56:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:16.924497+00:00 (in 9.999203 seconds) 2020-03-31 08:56:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:16 UTC)" (scheduled at 2020-03-31 08:56:06.924497+00:00) 2020-03-31 08:56:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:56:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:16.924497+00:00 (in 9.999203 seconds) 2020-03-31 08:56:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:16 UTC)" (scheduled at 2020-03-31 08:56:06.924497+00:00) 2020-03-31 08:56:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 6, 925933), True, datetime.datetime(2020, 3, 31, 8, 56, 6, 925933), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:56:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 6, 925933), True, datetime.datetime(2020, 3, 31, 8, 56, 6, 925933), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:56:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:56:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:56:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:16 UTC)" executed successfully 2020-03-31 08:56:08,272 [518] [DEBUG] [app] Starting request: urn:request:01917bd4-6562-4357-a21b-41d5bce35746 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:08,272 [518] [DEBUG] [app] Starting request: urn:request:01917bd4-6562-4357-a21b-41d5bce35746 (/health/instance) 2020-03-31 08:56:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:08,280 [546] [DEBUG] [app] Starting request: urn:request:659869c8-dc01-4f41-bda2-506126152a95 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:08,280 [546] [DEBUG] [app] Starting request: urn:request:659869c8-dc01-4f41-bda2-506126152a95 (/v1/_internal_ping) 2020-03-31 08:56:08,281 [546] [DEBUG] [app] Ending request: urn:request:659869c8-dc01-4f41-bda2-506126152a95 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:08,281 [546] [DEBUG] [app] Ending request: urn:request:659869c8-dc01-4f41-bda2-506126152a95 (/v1/_internal_ping) 2020-03-31 08:56:08,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:56:08,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:56:08,285 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:56:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:08,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:08,285 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:08,287 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:08,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:08,287 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:08,313 [518] [DEBUG] [app] Starting request: urn:request:ec1bb146-2909-471c-90b8-94c6016fadcc (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:08,313 [518] [DEBUG] [app] Starting request: urn:request:ec1bb146-2909-471c-90b8-94c6016fadcc (/_internal_ping) 2020-03-31 08:56:08,313 [518] [DEBUG] [app] Ending request: urn:request:ec1bb146-2909-471c-90b8-94c6016fadcc (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:08,313 [518] [DEBUG] [app] Ending request: urn:request:ec1bb146-2909-471c-90b8-94c6016fadcc (/_internal_ping) 2020-03-31 08:56:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:56:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:56:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:08,327 [536] [DEBUG] [app] Starting request: urn:request:529242cf-302d-4265-9a0e-80c7dba5f73a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:08,327 [536] [DEBUG] [app] Starting request: urn:request:529242cf-302d-4265-9a0e-80c7dba5f73a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:08,328 [536] [DEBUG] [app] Ending request: urn:request:529242cf-302d-4265-9a0e-80c7dba5f73a (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:08,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:56:08,328 [536] [DEBUG] [app] Ending request: urn:request:529242cf-302d-4265-9a0e-80c7dba5f73a (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:08,330 [518] [DEBUG] [app] Ending request: urn:request:01917bd4-6562-4357-a21b-41d5bce35746 (/health/instance) 2020-03-31 08:56:08,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:56:08,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:56:08,330 [518] [DEBUG] [app] Ending request: urn:request:01917bd4-6562-4357-a21b-41d5bce35746 (/health/instance) 2020-03-31 08:56:08,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:56:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:56:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:21.440707+00:00 (in 9.997215 seconds) 2020-03-31 08:56:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:21 UTC)" (scheduled at 2020-03-31 08:56:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:56:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:21.440707+00:00 (in 9.997215 seconds) 2020-03-31 08:56:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:21 UTC)" (scheduled at 2020-03-31 08:56:11.440707+00:00) 2020-03-31 08:56:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:56:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:56:11,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:56:11,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:56:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:56:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:21 UTC)" executed successfully 2020-03-31 08:56:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:56:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:24.323352+00:00 (in 9.999096 seconds) 2020-03-31 08:56:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:24 UTC)" (scheduled at 2020-03-31 08:56:14.323352+00:00) 2020-03-31 08:56:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:56:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:24.323352+00:00 (in 9.999096 seconds) 2020-03-31 08:56:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:24 UTC)" (scheduled at 2020-03-31 08:56:14.323352+00:00) 2020-03-31 08:56:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 14, 325299), True, datetime.datetime(2020, 3, 31, 8, 56, 14, 325299), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:56:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 14, 325299), True, datetime.datetime(2020, 3, 31, 8, 56, 14, 325299), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:56:14,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:14,352 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:56:14,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:14,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:14,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:56:14,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:24 UTC)" executed successfully 2020-03-31 08:56:14,446 [527] [DEBUG] [app] Starting request: urn:request:1a7e0ace-a3ca-4ffc-8f2e-8fcaf76a2a98 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:14,446 [527] [DEBUG] [app] Starting request: urn:request:1a7e0ace-a3ca-4ffc-8f2e-8fcaf76a2a98 (/health/instance) 2020-03-31 08:56:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:14,453 [553] [DEBUG] [app] Starting request: urn:request:6e5ce36a-55ff-4700-ba09-94f6cb4e034a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:14,453 [553] [DEBUG] [app] Starting request: urn:request:6e5ce36a-55ff-4700-ba09-94f6cb4e034a (/v1/_internal_ping) 2020-03-31 08:56:14,454 [553] [DEBUG] [app] Ending request: urn:request:6e5ce36a-55ff-4700-ba09-94f6cb4e034a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:14,454 [553] [DEBUG] [app] Ending request: urn:request:6e5ce36a-55ff-4700-ba09-94f6cb4e034a (/v1/_internal_ping) 2020-03-31 08:56:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:56:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:56:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:14,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:14,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:14,478 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:14,478 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:14,502 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:14,502 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:14,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:14,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:14,507 [525] [DEBUG] [app] Starting request: urn:request:c3cddb5c-83e2-434a-9017-b68b7496079c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:14,507 [525] [DEBUG] [app] Starting request: urn:request:c3cddb5c-83e2-434a-9017-b68b7496079c (/_internal_ping) 2020-03-31 08:56:14,508 [525] [DEBUG] [app] Ending request: urn:request:c3cddb5c-83e2-434a-9017-b68b7496079c (/_internal_ping) 2020-03-31 08:56:14,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:14,509 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:14,508 [525] [DEBUG] [app] Ending request: urn:request:c3cddb5c-83e2-434a-9017-b68b7496079c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:14,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:14,509 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:56:14,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:14,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:14,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:56:14,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:56:14,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:56:14,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:14,516 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:56:14,516 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:14,520 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:14,520 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:14,521 [536] [DEBUG] [app] Starting request: urn:request:1733b8ea-0c50-4c2d-9dfb-f5df9e7b584c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:14,521 [536] [DEBUG] [app] Starting request: urn:request:1733b8ea-0c50-4c2d-9dfb-f5df9e7b584c (/c1/_internal_ping) 2020-03-31 08:56:14,522 [536] [DEBUG] [app] Ending request: urn:request:1733b8ea-0c50-4c2d-9dfb-f5df9e7b584c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:14,522 [536] [DEBUG] [app] Ending request: urn:request:1733b8ea-0c50-4c2d-9dfb-f5df9e7b584c (/c1/_internal_ping) 2020-03-31 08:56:14,523 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:14,522 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:14,523 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:14,524 [527] [DEBUG] [app] Ending request: urn:request:1a7e0ace-a3ca-4ffc-8f2e-8fcaf76a2a98 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:56:14,522 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:14,525 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:14,525 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) gunicorn-web stdout | 2020-03-31 08:56:14,524 [527] [DEBUG] [app] Ending request: urn:request:1a7e0ace-a3ca-4ffc-8f2e-8fcaf76a2a98 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:14,525 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:14,525 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) 2020-03-31 08:56:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:56:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:45.064514+00:00 (in 29.998830 seconds) 2020-03-31 08:56:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:45 UTC)" (scheduled at 2020-03-31 08:56:15.064514+00:00) gcworker stdout | 2020-03-31 08:56:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:45.064514+00:00 (in 29.998830 seconds) 2020-03-31 08:56:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:45 UTC)" (scheduled at 2020-03-31 08:56:15.064514+00:00) 2020-03-31 08:56:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435375, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:56:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435375, 1209600, 500, 1, 0]) 2020-03-31 08:56:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:56:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:56:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:56:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:45 UTC)" executed successfully 2020-03-31 08:56:16,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:56:16,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:26 UTC)" (scheduled at 2020-03-31 08:56:16.924497+00:00) 2020-03-31 08:56:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:26.924497+00:00 (in 9.999388 seconds) storagereplication stdout | 2020-03-31 08:56:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:26 UTC)" (scheduled at 2020-03-31 08:56:16.924497+00:00) 2020-03-31 08:56:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:26.924497+00:00 (in 9.999388 seconds) 2020-03-31 08:56:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:56:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 16, 926043), True, datetime.datetime(2020, 3, 31, 8, 56, 16, 926043), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:56:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 16, 926043), True, datetime.datetime(2020, 3, 31, 8, 56, 16, 926043), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:56:16,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:56:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:56:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:26 UTC)" executed successfully 2020-03-31 08:56:18,273 [526] [DEBUG] [app] Starting request: urn:request:06bb93ff-7efe-4492-b20b-51a459ed31c3 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:18,273 [526] [DEBUG] [app] Starting request: urn:request:06bb93ff-7efe-4492-b20b-51a459ed31c3 (/health/instance) 2020-03-31 08:56:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:18,280 [550] [DEBUG] [app] Starting request: urn:request:79780102-c61d-4030-afa5-44984082fa4a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:18,280 [550] [DEBUG] [app] Starting request: urn:request:79780102-c61d-4030-afa5-44984082fa4a (/v1/_internal_ping) 2020-03-31 08:56:18,282 [550] [DEBUG] [app] Ending request: urn:request:79780102-c61d-4030-afa5-44984082fa4a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:18,282 [550] [DEBUG] [app] Ending request: urn:request:79780102-c61d-4030-afa5-44984082fa4a (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:18,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:18,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:56:18,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:18,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:18,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:18,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:18,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:56:18,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:18,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:18,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:18,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:18,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:18,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:18,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:18,312 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:18,312 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:18,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:18,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:18,317 [518] [DEBUG] [app] Starting request: urn:request:eaed120a-a4f9-44a7-b0d7-8588ab7f0f6f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:18,317 [518] [DEBUG] [app] Starting request: urn:request:eaed120a-a4f9-44a7-b0d7-8588ab7f0f6f (/_internal_ping) 2020-03-31 08:56:18,318 [518] [DEBUG] [app] Ending request: urn:request:eaed120a-a4f9-44a7-b0d7-8588ab7f0f6f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:18,318 [518] [DEBUG] [app] Ending request: urn:request:eaed120a-a4f9-44a7-b0d7-8588ab7f0f6f (/_internal_ping) 2020-03-31 08:56:18,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:18,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:18,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:18,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:56:18,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:18,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:56:18,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:56:18,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:18,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:56:18,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:18,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:56:18,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:18,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:18,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:18,331 [543] [DEBUG] [app] Starting request: urn:request:85ed0d73-6543-4a2f-b51a-c23d23bed2f7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:18,331 [543] [DEBUG] [app] Starting request: urn:request:85ed0d73-6543-4a2f-b51a-c23d23bed2f7 (/c1/_internal_ping) 2020-03-31 08:56:18,332 [543] [DEBUG] [app] Ending request: urn:request:85ed0d73-6543-4a2f-b51a-c23d23bed2f7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:18,332 [543] [DEBUG] [app] Ending request: urn:request:85ed0d73-6543-4a2f-b51a-c23d23bed2f7 (/c1/_internal_ping) 2020-03-31 08:56:18,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:18,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:18,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:18,335 [526] [DEBUG] [app] Ending request: urn:request:06bb93ff-7efe-4492-b20b-51a459ed31c3 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:56:18,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:18,335 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:18,336 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:56:18,335 [526] [DEBUG] [app] Ending request: urn:request:06bb93ff-7efe-4492-b20b-51a459ed31c3 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:18,335 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:18,336 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:56:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:56:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:31.440707+00:00 (in 9.999237 seconds) 2020-03-31 08:56:21,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:31 UTC)" (scheduled at 2020-03-31 08:56:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:56:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:31.440707+00:00 (in 9.999237 seconds) 2020-03-31 08:56:21,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:31 UTC)" (scheduled at 2020-03-31 08:56:21.440707+00:00) 2020-03-31 08:56:21,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:56:21,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:56:21,476 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:56:21,476 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:56:21,476 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:21,476 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:56:21,476 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:21,476 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:31 UTC)" executed successfully 2020-03-31 08:56:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:56:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:52.113221+00:00 (in 29.998902 seconds) 2020-03-31 08:56:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:52 UTC)" (scheduled at 2020-03-31 08:56:22.113221+00:00) securityworker stdout | 2020-03-31 08:56:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:52.113221+00:00 (in 29.998902 seconds) 2020-03-31 08:56:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:52 UTC)" (scheduled at 2020-03-31 08:56:22.113221+00:00) 2020-03-31 08:56:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:56:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:56:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:56:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:56:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:56:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:56:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:56:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:56:52 UTC)" executed successfully 2020-03-31 08:56:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:56:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:34.323352+00:00 (in 9.998713 seconds) 2020-03-31 08:56:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:34 UTC)" (scheduled at 2020-03-31 08:56:24.323352+00:00) 2020-03-31 08:56:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:56:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:34.323352+00:00 (in 9.998713 seconds) 2020-03-31 08:56:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:34 UTC)" (scheduled at 2020-03-31 08:56:24.323352+00:00) 2020-03-31 08:56:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 24, 325533), True, datetime.datetime(2020, 3, 31, 8, 56, 24, 325533), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:56:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 24, 325533), True, datetime.datetime(2020, 3, 31, 8, 56, 24, 325533), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:56:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:56:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:56:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:34 UTC)" executed successfully 2020-03-31 08:56:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:56:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:33.330112+00:00 (in 8.918269 seconds) 2020-03-31 08:56:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:24 UTC)" (scheduled at 2020-03-31 08:56:24.410927+00:00) 2020-03-31 08:56:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 08:56:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:33.330112+00:00 (in 8.918269 seconds) 2020-03-31 08:56:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:24 UTC)" (scheduled at 2020-03-31 08:56:24.410927+00:00) 2020-03-31 08:56:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:56:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:56:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:24 UTC)" executed successfully 2020-03-31 08:56:24,445 [526] [DEBUG] [app] Starting request: urn:request:854aa13d-c25e-4024-a62c-2bbc4a35ab36 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:24,445 [526] [DEBUG] [app] Starting request: urn:request:854aa13d-c25e-4024-a62c-2bbc4a35ab36 (/health/instance) 2020-03-31 08:56:24,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:24,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:24,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:24,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:24,453 [554] [DEBUG] [app] Starting request: urn:request:3c43f74b-c4f2-4091-9e6c-4d806e1455b5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:24,453 [554] [DEBUG] [app] Starting request: urn:request:3c43f74b-c4f2-4091-9e6c-4d806e1455b5 (/v1/_internal_ping) 2020-03-31 08:56:24,454 [554] [DEBUG] [app] Ending request: urn:request:3c43f74b-c4f2-4091-9e6c-4d806e1455b5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:24,454 [554] [DEBUG] [app] Ending request: urn:request:3c43f74b-c4f2-4091-9e6c-4d806e1455b5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:24,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:24,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:56:24,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:24,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:24,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:24,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:24,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:56:24,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:24,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:24,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:24,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:24,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:24,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:24,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:24,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:24,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:24,483 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:24,483 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:24,484 [526] [DEBUG] [app] Starting request: urn:request:80f14c95-764e-452c-bcb1-3dc1c7bd6de0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:24,484 [526] [DEBUG] [app] Starting request: urn:request:80f14c95-764e-452c-bcb1-3dc1c7bd6de0 (/_internal_ping) 2020-03-31 08:56:24,485 [526] [DEBUG] [app] Ending request: urn:request:80f14c95-764e-452c-bcb1-3dc1c7bd6de0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:24,485 [526] [DEBUG] [app] Ending request: urn:request:80f14c95-764e-452c-bcb1-3dc1c7bd6de0 (/_internal_ping) 2020-03-31 08:56:24,486 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:24,486 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:24,486 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:24,486 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:24,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:24,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:24,490 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:56:24,490 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:56:24,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:24,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:56:24,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:24,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:24,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:24,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:24,498 [534] [DEBUG] [app] Starting request: urn:request:40b04ae0-5f77-4f30-a442-683f26df8fe8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:24,498 [534] [DEBUG] [app] Starting request: urn:request:40b04ae0-5f77-4f30-a442-683f26df8fe8 (/c1/_internal_ping) 2020-03-31 08:56:24,499 [534] [DEBUG] [app] Ending request: urn:request:40b04ae0-5f77-4f30-a442-683f26df8fe8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:24,499 [534] [DEBUG] [app] Ending request: urn:request:40b04ae0-5f77-4f30-a442-683f26df8fe8 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:24,500 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:24,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:56:24,500 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:24,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:24,502 [526] [DEBUG] [app] Ending request: urn:request:854aa13d-c25e-4024-a62c-2bbc4a35ab36 (/health/instance) 2020-03-31 08:56:24,502 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:24,502 [526] [DEBUG] [app] Ending request: urn:request:854aa13d-c25e-4024-a62c-2bbc4a35ab36 (/health/instance) 2020-03-31 08:56:24,502 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:24,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:56:24,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:56:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:56:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:36.924497+00:00 (in 9.998551 seconds) storagereplication stdout | 2020-03-31 08:56:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:36.924497+00:00 (in 9.998551 seconds) 2020-03-31 08:56:26,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:36 UTC)" (scheduled at 2020-03-31 08:56:26.924497+00:00) 2020-03-31 08:56:26,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:56:26,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:36 UTC)" (scheduled at 2020-03-31 08:56:26.924497+00:00) 2020-03-31 08:56:26,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:26,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 26, 929514), True, datetime.datetime(2020, 3, 31, 8, 56, 26, 929514), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:56:26,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 26, 929514), True, datetime.datetime(2020, 3, 31, 8, 56, 26, 929514), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:56:26,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:56:26,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:26,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:26,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:56:26,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:26,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:36 UTC)" executed successfully 2020-03-31 08:56:28,273 [518] [DEBUG] [app] Starting request: urn:request:19d1f018-f390-4a68-8496-29b382160b6b (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:28,273 [518] [DEBUG] [app] Starting request: urn:request:19d1f018-f390-4a68-8496-29b382160b6b (/health/instance) 2020-03-31 08:56:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:28,282 [555] [DEBUG] [app] Starting request: urn:request:5e54fd57-4631-430e-95cc-a95f02e4b874 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:28,282 [555] [DEBUG] [app] Starting request: urn:request:5e54fd57-4631-430e-95cc-a95f02e4b874 (/v1/_internal_ping) 2020-03-31 08:56:28,283 [555] [DEBUG] [app] Ending request: urn:request:5e54fd57-4631-430e-95cc-a95f02e4b874 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:28,283 [555] [DEBUG] [app] Ending request: urn:request:5e54fd57-4631-430e-95cc-a95f02e4b874 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:28,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:28,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:56:28,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:28,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:28,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:28,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:28,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:56:28,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:28,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:28,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:28,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:28,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:28,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:28,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:28,316 [518] [DEBUG] [app] Starting request: urn:request:12fc809c-6393-4eca-879c-3aaa9fac4077 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:28,316 [518] [DEBUG] [app] Starting request: urn:request:12fc809c-6393-4eca-879c-3aaa9fac4077 (/_internal_ping) 2020-03-31 08:56:28,317 [518] [DEBUG] [app] Ending request: urn:request:12fc809c-6393-4eca-879c-3aaa9fac4077 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:28,317 [518] [DEBUG] [app] Ending request: urn:request:12fc809c-6393-4eca-879c-3aaa9fac4077 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:56:28,318 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:28,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:28,318 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:28,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:56:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:56:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:28,332 [536] [DEBUG] [app] Starting request: urn:request:b971ac13-2d66-487e-9e36-844b73991dad (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:28,332 [536] [DEBUG] [app] Starting request: urn:request:b971ac13-2d66-487e-9e36-844b73991dad (/c1/_internal_ping) 2020-03-31 08:56:28,333 [536] [DEBUG] [app] Ending request: urn:request:b971ac13-2d66-487e-9e36-844b73991dad (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:28,333 [536] [DEBUG] [app] Ending request: urn:request:b971ac13-2d66-487e-9e36-844b73991dad (/c1/_internal_ping) 2020-03-31 08:56:28,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:28,333 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:28,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:28,335 [518] [DEBUG] [app] Ending request: urn:request:19d1f018-f390-4a68-8496-29b382160b6b (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:56:28,333 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:28,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:28,335 [518] [DEBUG] [app] Ending request: urn:request:19d1f018-f390-4a68-8496-29b382160b6b (/health/instance) 2020-03-31 08:56:28,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 08:56:28,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:28,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 08:56:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:56:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:00.337699+00:00 (in 29.998998 seconds) 2020-03-31 08:56:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:00 UTC)" (scheduled at 2020-03-31 08:56:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:56:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:00.337699+00:00 (in 29.998998 seconds) 2020-03-31 08:56:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:00 UTC)" (scheduled at 2020-03-31 08:56:30.337699+00:00) 2020-03-31 08:56:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 56, 30, 339649), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:56:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 56, 30, 339649), False, 50, 1, 0]) 2020-03-31 08:56:30,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:56:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:56:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:56:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:00 UTC)" executed successfully 2020-03-31 08:56:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:56:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:41.440707+00:00 (in 9.999293 seconds) 2020-03-31 08:56:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:41 UTC)" (scheduled at 2020-03-31 08:56:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:56:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:41.440707+00:00 (in 9.999293 seconds) 2020-03-31 08:56:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:41 UTC)" (scheduled at 2020-03-31 08:56:31.440707+00:00) 2020-03-31 08:56:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:56:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:56:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:56:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:56:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:56:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:41 UTC)" executed successfully 2020-03-31 08:56:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:56:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:03 UTC)" (scheduled at 2020-03-31 08:56:33.330112+00:00) 2020-03-31 08:56:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:56:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:03 UTC)" (scheduled at 2020-03-31 08:56:33.330112+00:00) 2020-03-31 08:56:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:03.330112+00:00 (in 29.999063 seconds) security_notification_worker stdout | 2020-03-31 08:56:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:03.330112+00:00 (in 29.999063 seconds) 2020-03-31 08:56:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 33, 332063), True, datetime.datetime(2020, 3, 31, 8, 56, 33, 332063), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:56:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 33, 332063), True, datetime.datetime(2020, 3, 31, 8, 56, 33, 332063), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:56:33,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:56:33,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:56:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:03 UTC)" executed successfully 2020-03-31 08:56:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:56:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:43.539999+00:00 (in 10.171512 seconds) 2020-03-31 08:56:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:33 UTC)" (scheduled at 2020-03-31 08:56:33.366791+00:00) 2020-03-31 08:56:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:56:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:43.539999+00:00 (in 10.171512 seconds) 2020-03-31 08:56:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:33 UTC)" (scheduled at 2020-03-31 08:56:33.366791+00:00) 2020-03-31 08:56:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 33, 369082), True, datetime.datetime(2020, 3, 31, 8, 56, 33, 369082), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:56:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 33, 369082), True, datetime.datetime(2020, 3, 31, 8, 56, 33, 369082), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:56:33,386 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:56:33,386 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:56:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:33 UTC)" executed successfully 2020-03-31 08:56:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:56:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:44.323352+00:00 (in 9.998787 seconds) 2020-03-31 08:56:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:44 UTC)" (scheduled at 2020-03-31 08:56:34.323352+00:00) 2020-03-31 08:56:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:56:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:44.323352+00:00 (in 9.998787 seconds) 2020-03-31 08:56:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:44 UTC)" (scheduled at 2020-03-31 08:56:34.323352+00:00) 2020-03-31 08:56:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 34, 325510), True, datetime.datetime(2020, 3, 31, 8, 56, 34, 325510), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:56:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 34, 325510), True, datetime.datetime(2020, 3, 31, 8, 56, 34, 325510), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:56:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:56:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:56:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:44 UTC)" executed successfully 2020-03-31 08:56:34,445 [518] [DEBUG] [app] Starting request: urn:request:f10fe6d8-0f97-43c8-92d8-8d2792c619c4 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:34,445 [518] [DEBUG] [app] Starting request: urn:request:f10fe6d8-0f97-43c8-92d8-8d2792c619c4 (/health/instance) 2020-03-31 08:56:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:34,454 [549] [DEBUG] [app] Starting request: urn:request:ce3cc23d-9625-419f-a6a1-7793f84abca4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:34,454 [549] [DEBUG] [app] Starting request: urn:request:ce3cc23d-9625-419f-a6a1-7793f84abca4 (/v1/_internal_ping) 2020-03-31 08:56:34,455 [549] [DEBUG] [app] Ending request: urn:request:ce3cc23d-9625-419f-a6a1-7793f84abca4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:34,455 [549] [DEBUG] [app] Ending request: urn:request:ce3cc23d-9625-419f-a6a1-7793f84abca4 (/v1/_internal_ping) 2020-03-31 08:56:34,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:56:34,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:34,459 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:56:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:56:34,459 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:34,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:34,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:34,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:34,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:34,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:34,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:34,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:34,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:34,493 [518] [DEBUG] [app] Starting request: urn:request:cd4b60a3-4c82-439e-ad5e-034b68f45ea2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:34,493 [518] [DEBUG] [app] Starting request: urn:request:cd4b60a3-4c82-439e-ad5e-034b68f45ea2 (/_internal_ping) 2020-03-31 08:56:34,494 [518] [DEBUG] [app] Ending request: urn:request:cd4b60a3-4c82-439e-ad5e-034b68f45ea2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:34,494 [518] [DEBUG] [app] Ending request: urn:request:cd4b60a3-4c82-439e-ad5e-034b68f45ea2 (/_internal_ping) 2020-03-31 08:56:34,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:34,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:56:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:56:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:56:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:34,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:34,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:34,508 [534] [DEBUG] [app] Starting request: urn:request:c47fbc7a-e597-46c9-aeb0-d01812b32c65 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:34,508 [534] [DEBUG] [app] Starting request: urn:request:c47fbc7a-e597-46c9-aeb0-d01812b32c65 (/c1/_internal_ping) 2020-03-31 08:56:34,509 [534] [DEBUG] [app] Ending request: urn:request:c47fbc7a-e597-46c9-aeb0-d01812b32c65 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:34,509 [534] [DEBUG] [app] Ending request: urn:request:c47fbc7a-e597-46c9-aeb0-d01812b32c65 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:34,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:34,510 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:34,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:56:34,510 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:34,511 [518] [DEBUG] [app] Ending request: urn:request:f10fe6d8-0f97-43c8-92d8-8d2792c619c4 (/health/instance) 2020-03-31 08:56:34,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:34,511 [518] [DEBUG] [app] Ending request: urn:request:f10fe6d8-0f97-43c8-92d8-8d2792c619c4 (/health/instance) 2020-03-31 08:56:34,512 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:34,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 08:56:34,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 08:56:36,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:56:36,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:36,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:38.010934+00:00 (in 1.083004 seconds) 2020-03-31 08:56:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:46 UTC)" (scheduled at 2020-03-31 08:56:36.924497+00:00) 2020-03-31 08:56:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:56:36,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:38.010934+00:00 (in 1.083004 seconds) 2020-03-31 08:56:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:46 UTC)" (scheduled at 2020-03-31 08:56:36.924497+00:00) 2020-03-31 08:56:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 36, 928713), True, datetime.datetime(2020, 3, 31, 8, 56, 36, 928713), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:56:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 36, 928713), True, datetime.datetime(2020, 3, 31, 8, 56, 36, 928713), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:56:36,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:56:36,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:36,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:36,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:56:36,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:36,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:46 UTC)" executed successfully 2020-03-31 08:56:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:56:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:38 UTC)" (scheduled at 2020-03-31 08:56:38.010934+00:00) 2020-03-31 08:56:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 08:56:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:38 UTC)" (scheduled at 2020-03-31 08:56:38.010934+00:00) 2020-03-31 08:56:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:56:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:46.924497+00:00 (in 8.912696 seconds) 2020-03-31 08:56:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:56:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:46.924497+00:00 (in 8.912696 seconds) 2020-03-31 08:56:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:38 UTC)" executed successfully 2020-03-31 08:56:38,272 [527] [DEBUG] [app] Starting request: urn:request:ba9c520e-6675-4443-9da9-f876c3b3309f (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:38,272 [527] [DEBUG] [app] Starting request: urn:request:ba9c520e-6675-4443-9da9-f876c3b3309f (/health/instance) 2020-03-31 08:56:38,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:38,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:38,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:38,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:38,279 [552] [DEBUG] [app] Starting request: urn:request:a0185667-bc92-48da-bc74-6dccc1c51444 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:38,279 [552] [DEBUG] [app] Starting request: urn:request:a0185667-bc92-48da-bc74-6dccc1c51444 (/v1/_internal_ping) 2020-03-31 08:56:38,280 [552] [DEBUG] [app] Ending request: urn:request:a0185667-bc92-48da-bc74-6dccc1c51444 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:38,280 [552] [DEBUG] [app] Ending request: urn:request:a0185667-bc92-48da-bc74-6dccc1c51444 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:38,281 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:38,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:56:38,281 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:38,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:38,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:38,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:38,284 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:56:38,284 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:38,285 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:38,285 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:38,286 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:38,286 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:38,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:38,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:38,313 [527] [DEBUG] [app] Starting request: urn:request:bbcfc054-ad82-4a4e-904e-5e8ac2bfbcd4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:38,313 [527] [DEBUG] [app] Starting request: urn:request:bbcfc054-ad82-4a4e-904e-5e8ac2bfbcd4 (/_internal_ping) 2020-03-31 08:56:38,314 [527] [DEBUG] [app] Ending request: urn:request:bbcfc054-ad82-4a4e-904e-5e8ac2bfbcd4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:38,314 [527] [DEBUG] [app] Ending request: urn:request:bbcfc054-ad82-4a4e-904e-5e8ac2bfbcd4 (/_internal_ping) 2020-03-31 08:56:38,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:38,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:56:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:56:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:56:38,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:38,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:38,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:38,327 [536] [DEBUG] [app] Starting request: urn:request:bfd6e62a-aa78-4252-abe0-da874702121e (/c1/_internal_ping) 2020-03-31 08:56:38,328 [536] [DEBUG] [app] Ending request: urn:request:bfd6e62a-aa78-4252-abe0-da874702121e (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:38,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:56:38,327 [536] [DEBUG] [app] Starting request: urn:request:bfd6e62a-aa78-4252-abe0-da874702121e (/c1/_internal_ping) 2020-03-31 08:56:38,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:38,329 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:56:38,328 [536] [DEBUG] [app] Ending request: urn:request:bfd6e62a-aa78-4252-abe0-da874702121e (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:38,331 [527] [DEBUG] [app] Ending request: urn:request:ba9c520e-6675-4443-9da9-f876c3b3309f (/health/instance) 2020-03-31 08:56:38,331 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:38,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:38,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-verbs stdout | 2020-03-31 08:56:38,329 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:38,331 [527] [DEBUG] [app] Ending request: urn:request:ba9c520e-6675-4443-9da9-f876c3b3309f (/health/instance) 2020-03-31 08:56:38,331 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:38,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 08:56:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:56:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:51.440707+00:00 (in 9.999109 seconds) 2020-03-31 08:56:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:51 UTC)" (scheduled at 2020-03-31 08:56:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:56:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:51.440707+00:00 (in 9.999109 seconds) 2020-03-31 08:56:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:51 UTC)" (scheduled at 2020-03-31 08:56:41.440707+00:00) 2020-03-31 08:56:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:56:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:56:41,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:56:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:56:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:56:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:51 UTC)" executed successfully 2020-03-31 08:56:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:56:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:33.366791+00:00 (in 49.825968 seconds) 2020-03-31 08:56:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:43 UTC)" (scheduled at 2020-03-31 08:56:43.539999+00:00) 2020-03-31 08:56:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 08:56:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:33.366791+00:00 (in 49.825968 seconds) 2020-03-31 08:56:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:43 UTC)" (scheduled at 2020-03-31 08:56:43.539999+00:00) 2020-03-31 08:56:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:56:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:56:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:43 UTC)" executed successfully 2020-03-31 08:56:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:56:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:54 UTC)" (scheduled at 2020-03-31 08:56:44.323352+00:00) 2020-03-31 08:56:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:49.498966+00:00 (in 5.174560 seconds) notificationworker stdout | 2020-03-31 08:56:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:54 UTC)" (scheduled at 2020-03-31 08:56:44.323352+00:00) 2020-03-31 08:56:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:49.498966+00:00 (in 5.174560 seconds) 2020-03-31 08:56:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 44, 325539), True, datetime.datetime(2020, 3, 31, 8, 56, 44, 325539), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:56:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 44, 325539), True, datetime.datetime(2020, 3, 31, 8, 56, 44, 325539), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:56:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:56:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:56:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:54 UTC)" executed successfully 2020-03-31 08:56:44,446 [526] [DEBUG] [app] Starting request: urn:request:1f0b71e0-e2ed-4079-b722-a36ba31763b6 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:44,446 [526] [DEBUG] [app] Starting request: urn:request:1f0b71e0-e2ed-4079-b722-a36ba31763b6 (/health/instance) 2020-03-31 08:56:44,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:44,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:44,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:44,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:44,453 [551] [DEBUG] [app] Starting request: urn:request:5a4ff069-ae8d-46d1-b222-a9e24a9549c6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:44,453 [551] [DEBUG] [app] Starting request: urn:request:5a4ff069-ae8d-46d1-b222-a9e24a9549c6 (/v1/_internal_ping) 2020-03-31 08:56:44,455 [551] [DEBUG] [app] Ending request: urn:request:5a4ff069-ae8d-46d1-b222-a9e24a9549c6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:44,455 [551] [DEBUG] [app] Ending request: urn:request:5a4ff069-ae8d-46d1-b222-a9e24a9549c6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:44,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:56:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:44,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:44,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:44,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:56:44,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:56:44,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:44,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:44,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:44,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:44,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:44,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:44,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:44,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:44,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:44,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:44,491 [518] [DEBUG] [app] Starting request: urn:request:edfa6e0d-dbd1-49ff-8d4d-09a432d414a9 (/_internal_ping) 2020-03-31 08:56:44,492 [518] [DEBUG] [app] Ending request: urn:request:edfa6e0d-dbd1-49ff-8d4d-09a432d414a9 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:44,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:44,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:44,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:56:44,491 [518] [DEBUG] [app] Starting request: urn:request:edfa6e0d-dbd1-49ff-8d4d-09a432d414a9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:44,492 [518] [DEBUG] [app] Ending request: urn:request:edfa6e0d-dbd1-49ff-8d4d-09a432d414a9 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:56:44,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:44,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:44,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:44,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:44,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:56:44,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:44,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:44,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:56:44,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:44,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:44,507 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:44,507 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:44,508 [536] [DEBUG] [app] Starting request: urn:request:2e9e95b9-8216-4fe7-8cd9-f435509c03ee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:44,508 [536] [DEBUG] [app] Starting request: urn:request:2e9e95b9-8216-4fe7-8cd9-f435509c03ee (/c1/_internal_ping) 2020-03-31 08:56:44,509 [536] [DEBUG] [app] Ending request: urn:request:2e9e95b9-8216-4fe7-8cd9-f435509c03ee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:44,509 [536] [DEBUG] [app] Ending request: urn:request:2e9e95b9-8216-4fe7-8cd9-f435509c03ee (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:44,510 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:44,510 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:44,510 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:56:44,510 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:44,511 [526] [DEBUG] [app] Ending request: urn:request:1f0b71e0-e2ed-4079-b722-a36ba31763b6 (/health/instance) 2020-03-31 08:56:44,512 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:44,511 [526] [DEBUG] [app] Ending request: urn:request:1f0b71e0-e2ed-4079-b722-a36ba31763b6 (/health/instance) 2020-03-31 08:56:44,512 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:44,512 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 08:56:44,512 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 08:56:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:56:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:15.064514+00:00 (in 29.999102 seconds) 2020-03-31 08:56:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:15 UTC)" (scheduled at 2020-03-31 08:56:45.064514+00:00) gcworker stdout | 2020-03-31 08:56:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:15.064514+00:00 (in 29.999102 seconds) 2020-03-31 08:56:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:15 UTC)" (scheduled at 2020-03-31 08:56:45.064514+00:00) 2020-03-31 08:56:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435405, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:56:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435405, 1209600, 500, 1, 0]) 2020-03-31 08:56:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:56:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:56:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:56:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:15 UTC)" executed successfully 2020-03-31 08:56:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:56:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:56.924497+00:00 (in 9.998340 seconds) 2020-03-31 08:56:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:56 UTC)" (scheduled at 2020-03-31 08:56:46.924497+00:00) 2020-03-31 08:56:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:56:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:56.924497+00:00 (in 9.998340 seconds) 2020-03-31 08:56:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:56 UTC)" (scheduled at 2020-03-31 08:56:46.924497+00:00) 2020-03-31 08:56:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 46, 926891), True, datetime.datetime(2020, 3, 31, 8, 56, 46, 926891), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:56:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 46, 926891), True, datetime.datetime(2020, 3, 31, 8, 56, 46, 926891), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:56:46,950 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:56:46,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:46,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:46,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:56:46,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:46,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:56:56 UTC)" executed successfully 2020-03-31 08:56:48,272 [527] [DEBUG] [app] Starting request: urn:request:a88eac18-56f4-4a0e-b97d-1f4cf8dab75b (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:48,272 [527] [DEBUG] [app] Starting request: urn:request:a88eac18-56f4-4a0e-b97d-1f4cf8dab75b (/health/instance) 2020-03-31 08:56:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:48,280 [547] [DEBUG] [app] Starting request: urn:request:86a9651c-9b69-4699-9bc4-3483fda19acc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:48,280 [547] [DEBUG] [app] Starting request: urn:request:86a9651c-9b69-4699-9bc4-3483fda19acc (/v1/_internal_ping) 2020-03-31 08:56:48,281 [547] [DEBUG] [app] Ending request: urn:request:86a9651c-9b69-4699-9bc4-3483fda19acc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:48,281 [547] [DEBUG] [app] Ending request: urn:request:86a9651c-9b69-4699-9bc4-3483fda19acc (/v1/_internal_ping) 2020-03-31 08:56:48,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:56:48,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 08:56:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:48,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:56:48,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:48,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:48,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:48,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:48,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:48,303 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:48,303 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:48,318 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:48,318 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:48,322 [527] [DEBUG] [app] Starting request: urn:request:cf1cc871-f32f-451a-950c-f9d6ccdac28a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:48,322 [527] [DEBUG] [app] Starting request: urn:request:cf1cc871-f32f-451a-950c-f9d6ccdac28a (/_internal_ping) 2020-03-31 08:56:48,323 [527] [DEBUG] [app] Ending request: urn:request:cf1cc871-f32f-451a-950c-f9d6ccdac28a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:48,323 [527] [DEBUG] [app] Ending request: urn:request:cf1cc871-f32f-451a-950c-f9d6ccdac28a (/_internal_ping) 2020-03-31 08:56:48,324 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:56:48,324 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:48,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:56:48,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:56:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:48,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:48,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:56:48,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:48,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:48,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:48,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:48,337 [541] [DEBUG] [app] Starting request: urn:request:bab361d0-69f0-4fff-910a-2835b7e99330 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:48,337 [541] [DEBUG] [app] Starting request: urn:request:bab361d0-69f0-4fff-910a-2835b7e99330 (/c1/_internal_ping) 2020-03-31 08:56:48,338 [541] [DEBUG] [app] Ending request: urn:request:bab361d0-69f0-4fff-910a-2835b7e99330 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:48,338 [541] [DEBUG] [app] Ending request: urn:request:bab361d0-69f0-4fff-910a-2835b7e99330 (/c1/_internal_ping) 2020-03-31 08:56:48,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:48,338 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:48,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:56:48,338 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:48,340 [527] [DEBUG] [app] Ending request: urn:request:a88eac18-56f4-4a0e-b97d-1f4cf8dab75b (/health/instance) 2020-03-31 08:56:48,341 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 08:56:48,340 [527] [DEBUG] [app] Ending request: urn:request:a88eac18-56f4-4a0e-b97d-1f4cf8dab75b (/health/instance) 2020-03-31 08:56:48,341 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:48,341 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 08:56:48,341 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:56:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:56:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:54.323352+00:00 (in 4.822926 seconds) 2020-03-31 08:56:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:49 UTC)" (scheduled at 2020-03-31 08:56:49.498966+00:00) 2020-03-31 08:56:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:56:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:56:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:54.323352+00:00 (in 4.822926 seconds) 2020-03-31 08:56:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:49 UTC)" (scheduled at 2020-03-31 08:56:49.498966+00:00) 2020-03-31 08:56:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:56:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:49 UTC)" executed successfully 2020-03-31 08:56:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:56:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:51 UTC)" (scheduled at 2020-03-31 08:56:51.110610+00:00) 2020-03-31 08:56:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:59.560326+00:00 (in 8.448931 seconds) 2020-03-31 08:56:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:56:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:51 UTC)" (scheduled at 2020-03-31 08:56:51.110610+00:00) 2020-03-31 08:56:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:56:59.560326+00:00 (in 8.448931 seconds) 2020-03-31 08:56:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 51, 112607), True, datetime.datetime(2020, 3, 31, 8, 56, 51, 112607), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:56:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 51, 112607), True, datetime.datetime(2020, 3, 31, 8, 56, 51, 112607), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:56:51,131 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:56:51,131 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:56:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:51 UTC)" executed successfully 2020-03-31 08:56:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:56:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:01.440707+00:00 (in 9.999157 seconds) 2020-03-31 08:56:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:01 UTC)" (scheduled at 2020-03-31 08:56:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:56:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:01.440707+00:00 (in 9.999157 seconds) 2020-03-31 08:56:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:01 UTC)" (scheduled at 2020-03-31 08:56:51.440707+00:00) 2020-03-31 08:56:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:56:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:56:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:56:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:56:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:56:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:01 UTC)" executed successfully 2020-03-31 08:56:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:56:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:22.113221+00:00 (in 29.998923 seconds) 2020-03-31 08:56:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:22 UTC)" (scheduled at 2020-03-31 08:56:52.113221+00:00) securityworker stdout | 2020-03-31 08:56:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:22.113221+00:00 (in 29.998923 seconds) 2020-03-31 08:56:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:22 UTC)" (scheduled at 2020-03-31 08:56:52.113221+00:00) 2020-03-31 08:56:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:56:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:56:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:56:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:56:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:56:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:56:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:56:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:22 UTC)" executed successfully 2020-03-31 08:56:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:56:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:04.323352+00:00 (in 9.999014 seconds) 2020-03-31 08:56:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:04 UTC)" (scheduled at 2020-03-31 08:56:54.323352+00:00) 2020-03-31 08:56:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:56:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:04.323352+00:00 (in 9.999014 seconds) 2020-03-31 08:56:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:04 UTC)" (scheduled at 2020-03-31 08:56:54.323352+00:00) 2020-03-31 08:56:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 54, 325378), True, datetime.datetime(2020, 3, 31, 8, 56, 54, 325378), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:56:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 54, 325378), True, datetime.datetime(2020, 3, 31, 8, 56, 54, 325378), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:56:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:56:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:56:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:04 UTC)" executed successfully 2020-03-31 08:56:54,446 [518] [DEBUG] [app] Starting request: urn:request:17b1dcbd-5e99-4fed-b70c-52904e0a82e8 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:54,446 [518] [DEBUG] [app] Starting request: urn:request:17b1dcbd-5e99-4fed-b70c-52904e0a82e8 (/health/instance) 2020-03-31 08:56:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:54,454 [548] [DEBUG] [app] Starting request: urn:request:77b26cff-5003-4d61-acff-dd65b1873c19 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:54,454 [548] [DEBUG] [app] Starting request: urn:request:77b26cff-5003-4d61-acff-dd65b1873c19 (/v1/_internal_ping) 2020-03-31 08:56:54,455 [548] [DEBUG] [app] Ending request: urn:request:77b26cff-5003-4d61-acff-dd65b1873c19 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:54,455 [548] [DEBUG] [app] Ending request: urn:request:77b26cff-5003-4d61-acff-dd65b1873c19 (/v1/_internal_ping) 2020-03-31 08:56:54,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:56:54,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:56:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:56:54,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:56:54,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:54,460 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:54,460 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:54,462 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:54,462 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:54,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:54,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:54,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:54,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:54,489 [526] [DEBUG] [app] Starting request: urn:request:0a2691db-487c-4d13-82e0-b5dcf196fb1a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:54,489 [526] [DEBUG] [app] Starting request: urn:request:0a2691db-487c-4d13-82e0-b5dcf196fb1a (/_internal_ping) 2020-03-31 08:56:54,490 [526] [DEBUG] [app] Ending request: urn:request:0a2691db-487c-4d13-82e0-b5dcf196fb1a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:56:54,490 [526] [DEBUG] [app] Ending request: urn:request:0a2691db-487c-4d13-82e0-b5dcf196fb1a (/_internal_ping) 2020-03-31 08:56:54,491 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:56:54,491 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:56:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:56:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:56:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:54,504 [541] [DEBUG] [app] Starting request: urn:request:27771e3f-8d66-43f9-827c-bea627d00d79 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:54,504 [541] [DEBUG] [app] Starting request: urn:request:27771e3f-8d66-43f9-827c-bea627d00d79 (/c1/_internal_ping) 2020-03-31 08:56:54,505 [541] [DEBUG] [app] Ending request: urn:request:27771e3f-8d66-43f9-827c-bea627d00d79 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:54,505 [541] [DEBUG] [app] Ending request: urn:request:27771e3f-8d66-43f9-827c-bea627d00d79 (/c1/_internal_ping) 2020-03-31 08:56:54,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:54,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:54,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:56:54,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:54,507 [518] [DEBUG] [app] Ending request: urn:request:17b1dcbd-5e99-4fed-b70c-52904e0a82e8 (/health/instance) 2020-03-31 08:56:54,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:54,507 [518] [DEBUG] [app] Ending request: urn:request:17b1dcbd-5e99-4fed-b70c-52904e0a82e8 (/health/instance) 2020-03-31 08:56:54,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:54,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 08:56:54,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 08:56:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:03.010318+00:00 (in 6.085005 seconds) 2020-03-31 08:56:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:06 UTC)" (scheduled at 2020-03-31 08:56:56.924497+00:00) 2020-03-31 08:56:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:56:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:56:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:03.010318+00:00 (in 6.085005 seconds) 2020-03-31 08:56:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:06 UTC)" (scheduled at 2020-03-31 08:56:56.924497+00:00) 2020-03-31 08:56:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:56:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 56, 926504), True, datetime.datetime(2020, 3, 31, 8, 56, 56, 926504), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:56:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 56, 56, 926504), True, datetime.datetime(2020, 3, 31, 8, 56, 56, 926504), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:56:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:56:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:56:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:56:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:06 UTC)" executed successfully 2020-03-31 08:56:58,273 [518] [DEBUG] [app] Starting request: urn:request:f5d097ad-7144-4db5-820e-121abdedc9b9 (/health/instance) gunicorn-web stdout | 2020-03-31 08:56:58,273 [518] [DEBUG] [app] Starting request: urn:request:f5d097ad-7144-4db5-820e-121abdedc9b9 (/health/instance) 2020-03-31 08:56:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:58,278 [549] [DEBUG] [app] Starting request: urn:request:ace77e58-1c9c-49b4-8bad-b0ca62e12ed7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:58,278 [549] [DEBUG] [app] Starting request: urn:request:ace77e58-1c9c-49b4-8bad-b0ca62e12ed7 (/v1/_internal_ping) 2020-03-31 08:56:58,279 [549] [DEBUG] [app] Ending request: urn:request:ace77e58-1c9c-49b4-8bad-b0ca62e12ed7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:56:58,279 [549] [DEBUG] [app] Ending request: urn:request:ace77e58-1c9c-49b4-8bad-b0ca62e12ed7 (/v1/_internal_ping) 2020-03-31 08:56:58,280 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:56:58,280 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:56:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:56:58,282 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:56:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:56:58,282 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:56:58,284 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:56:58,284 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:56:58,285 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:56:58,285 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:56:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:56:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:56:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:56:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:56:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:58,314 [527] [DEBUG] [app] Starting request: urn:request:ca312b00-09d7-42b5-a8ec-9d75af2450c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:58,314 [527] [DEBUG] [app] Starting request: urn:request:ca312b00-09d7-42b5-a8ec-9d75af2450c2 (/_internal_ping) 2020-03-31 08:56:58,315 [527] [DEBUG] [app] Ending request: urn:request:ca312b00-09d7-42b5-a8ec-9d75af2450c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:56:58,315 [527] [DEBUG] [app] Ending request: urn:request:ca312b00-09d7-42b5-a8ec-9d75af2450c2 (/_internal_ping) 2020-03-31 08:56:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:56:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:56:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:56:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:56:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:56:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:56:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:56:58,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:56:58,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:56:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:56:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:56:58,328 [543] [DEBUG] [app] Starting request: urn:request:7bd4c1c2-8e7b-46fe-9fd5-7636247e897d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:58,328 [543] [DEBUG] [app] Starting request: urn:request:7bd4c1c2-8e7b-46fe-9fd5-7636247e897d (/c1/_internal_ping) 2020-03-31 08:56:58,329 [543] [DEBUG] [app] Ending request: urn:request:7bd4c1c2-8e7b-46fe-9fd5-7636247e897d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:56:58,329 [543] [DEBUG] [app] Ending request: urn:request:7bd4c1c2-8e7b-46fe-9fd5-7636247e897d (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:58,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:56:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:56:58,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:56:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:56:58,331 [518] [DEBUG] [app] Ending request: urn:request:f5d097ad-7144-4db5-820e-121abdedc9b9 (/health/instance) 2020-03-31 08:56:58,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:56:58,331 [518] [DEBUG] [app] Ending request: urn:request:f5d097ad-7144-4db5-820e-121abdedc9b9 (/health/instance) 2020-03-31 08:56:58,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:56:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:56:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:56:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:56:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:56:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:56:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:56:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:51.110610+00:00 (in 51.548902 seconds) 2020-03-31 08:56:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:59 UTC)" (scheduled at 2020-03-31 08:56:59.560326+00:00) 2020-03-31 08:56:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:56:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:56:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:51.110610+00:00 (in 51.548902 seconds) 2020-03-31 08:56:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:59 UTC)" (scheduled at 2020-03-31 08:56:59.560326+00:00) 2020-03-31 08:56:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:56:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:59 UTC)" executed successfully 2020-03-31 08:57:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:57:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:30.337699+00:00 (in 29.999088 seconds) 2020-03-31 08:57:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:30 UTC)" (scheduled at 2020-03-31 08:57:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:57:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:30.337699+00:00 (in 29.999088 seconds) 2020-03-31 08:57:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:30 UTC)" (scheduled at 2020-03-31 08:57:00.337699+00:00) 2020-03-31 08:57:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 57, 0, 339519), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:57:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 57, 0, 339519), False, 50, 1, 0]) 2020-03-31 08:57:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:57:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:57:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:57:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:30 UTC)" executed successfully 2020-03-31 08:57:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:57:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:11.440707+00:00 (in 9.998965 seconds) 2020-03-31 08:57:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:11 UTC)" (scheduled at 2020-03-31 08:57:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:57:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:11.440707+00:00 (in 9.998965 seconds) 2020-03-31 08:57:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:11 UTC)" (scheduled at 2020-03-31 08:57:01.440707+00:00) 2020-03-31 08:57:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:57:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:57:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:01,459 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 08:57:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:57:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:11 UTC)" executed successfully 2020-03-31 08:57:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:06.924497+00:00 (in 3.913418 seconds) 2020-03-31 08:57:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:03 UTC)" (scheduled at 2020-03-31 08:57:03.010318+00:00) storagereplication stdout | 2020-03-31 08:57:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:06.924497+00:00 (in 3.913418 seconds) 2020-03-31 08:57:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:03 UTC)" (scheduled at 2020-03-31 08:57:03.010318+00:00) 2020-03-31 08:57:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 08:57:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), u'imagestoragereplication/%']) 2020-03-31 08:57:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), True, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 08:57:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), True, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), 0, u'imagestoragereplication/%']) 2020-03-31 08:57:03,032 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), True, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 08:57:03,032 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), True, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 8, 57, 3, 12102), u'imagestoragereplication/%']) 2020-03-31 08:57:03,036 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 08:57:03,036 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:03,038 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 08:57:03,038 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:03,039 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:03,039 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:03 UTC)" executed successfully 2020-03-31 08:57:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 08:57:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 08:57:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 08:57:03,332 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:57:03,332 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:03,333 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:24.410927+00:00 (in 21.078213 seconds) 2020-03-31 08:57:03,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:33 UTC)" (scheduled at 2020-03-31 08:57:03.330112+00:00) 2020-03-31 08:57:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:57:03,333 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:24.410927+00:00 (in 21.078213 seconds) 2020-03-31 08:57:03,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:33 UTC)" (scheduled at 2020-03-31 08:57:03.330112+00:00) 2020-03-31 08:57:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:03,336 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 3, 333711), True, datetime.datetime(2020, 3, 31, 8, 57, 3, 333711), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:57:03,336 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 3, 333711), True, datetime.datetime(2020, 3, 31, 8, 57, 3, 333711), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:57:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:57:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:57:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:33 UTC)" executed successfully 2020-03-31 08:57:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:14.323352+00:00 (in 9.998610 seconds) 2020-03-31 08:57:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:14 UTC)" (scheduled at 2020-03-31 08:57:04.323352+00:00) 2020-03-31 08:57:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:57:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:14.323352+00:00 (in 9.998610 seconds) 2020-03-31 08:57:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:14 UTC)" (scheduled at 2020-03-31 08:57:04.323352+00:00) 2020-03-31 08:57:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 4, 325719), True, datetime.datetime(2020, 3, 31, 8, 57, 4, 325719), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:57:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 4, 325719), True, datetime.datetime(2020, 3, 31, 8, 57, 4, 325719), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:57:04,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:57:04,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:14 UTC)" executed successfully 2020-03-31 08:57:04,446 [518] [DEBUG] [app] Starting request: urn:request:a3599332-5683-4a89-9d29-7ac05755adbb (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:04,446 [518] [DEBUG] [app] Starting request: urn:request:a3599332-5683-4a89-9d29-7ac05755adbb (/health/instance) 2020-03-31 08:57:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:04,454 [553] [DEBUG] [app] Starting request: urn:request:ae9f612b-1674-431c-bf80-64d394d8efc3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:04,454 [553] [DEBUG] [app] Starting request: urn:request:ae9f612b-1674-431c-bf80-64d394d8efc3 (/v1/_internal_ping) 2020-03-31 08:57:04,455 [553] [DEBUG] [app] Ending request: urn:request:ae9f612b-1674-431c-bf80-64d394d8efc3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:04,455 [553] [DEBUG] [app] Ending request: urn:request:ae9f612b-1674-431c-bf80-64d394d8efc3 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:57:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:04,532 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:04,532 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:04,536 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:04,536 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:04,537 [518] [DEBUG] [app] Starting request: urn:request:b50378b9-3e0c-4c12-9a31-e8a0c1d62361 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:04,537 [518] [DEBUG] [app] Starting request: urn:request:b50378b9-3e0c-4c12-9a31-e8a0c1d62361 (/_internal_ping) 2020-03-31 08:57:04,538 [518] [DEBUG] [app] Ending request: urn:request:b50378b9-3e0c-4c12-9a31-e8a0c1d62361 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:04,538 [518] [DEBUG] [app] Ending request: urn:request:b50378b9-3e0c-4c12-9a31-e8a0c1d62361 (/_internal_ping) 2020-03-31 08:57:04,539 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:57:04,539 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:04,539 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:57:04,539 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:04,542 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:04,542 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:04,543 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:04,543 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:04,545 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:04,545 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:04,546 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:57:04,546 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:57:04,549 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:04,549 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:04,550 [539] [DEBUG] [app] Starting request: urn:request:458976e7-4a05-494d-9376-3fc74f479d0d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:04,550 [539] [DEBUG] [app] Starting request: urn:request:458976e7-4a05-494d-9376-3fc74f479d0d (/c1/_internal_ping) 2020-03-31 08:57:04,552 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:04,551 [539] [DEBUG] [app] Ending request: urn:request:458976e7-4a05-494d-9376-3fc74f479d0d (/c1/_internal_ping) 2020-03-31 08:57:04,552 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:04,552 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:04,554 [518] [DEBUG] [app] Ending request: urn:request:a3599332-5683-4a89-9d29-7ac05755adbb (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:57:04,551 [539] [DEBUG] [app] Ending request: urn:request:458976e7-4a05-494d-9376-3fc74f479d0d (/c1/_internal_ping) 2020-03-31 08:57:04,552 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:04,554 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:04,555 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.109 49 0.109) gunicorn-web stdout | 2020-03-31 08:57:04,554 [518] [DEBUG] [app] Ending request: urn:request:a3599332-5683-4a89-9d29-7ac05755adbb (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:04,554 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:04,555 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.109 49 0.109) 2020-03-31 08:57:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:16.924497+00:00 (in 9.998352 seconds) 2020-03-31 08:57:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:16 UTC)" (scheduled at 2020-03-31 08:57:06.924497+00:00) 2020-03-31 08:57:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:57:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:16.924497+00:00 (in 9.998352 seconds) 2020-03-31 08:57:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:16 UTC)" (scheduled at 2020-03-31 08:57:06.924497+00:00) 2020-03-31 08:57:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 6, 927102), True, datetime.datetime(2020, 3, 31, 8, 57, 6, 927102), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:57:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 6, 927102), True, datetime.datetime(2020, 3, 31, 8, 57, 6, 927102), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:57:06,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:57:06,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:06,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:06,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:16 UTC)" executed successfully 2020-03-31 08:57:08,273 [518] [DEBUG] [app] Starting request: urn:request:7cfa36eb-d468-4f4e-82ce-a08dc388d0dd (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:08,273 [518] [DEBUG] [app] Starting request: urn:request:7cfa36eb-d468-4f4e-82ce-a08dc388d0dd (/health/instance) 2020-03-31 08:57:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:08,278 [538] [DEBUG] [app] Starting request: urn:request:6b8279f4-804e-4da6-9567-5daa48307834 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:08,278 [538] [DEBUG] [app] Starting request: urn:request:6b8279f4-804e-4da6-9567-5daa48307834 (/v1/_internal_ping) 2020-03-31 08:57:08,279 [538] [DEBUG] [app] Ending request: urn:request:6b8279f4-804e-4da6-9567-5daa48307834 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:08,279 [538] [DEBUG] [app] Ending request: urn:request:6b8279f4-804e-4da6-9567-5daa48307834 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:08,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:57:08,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:57:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:08,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:08,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:08,313 [518] [DEBUG] [app] Starting request: urn:request:b480cb7c-4b27-46af-968a-83e4de5417e8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:08,313 [518] [DEBUG] [app] Starting request: urn:request:b480cb7c-4b27-46af-968a-83e4de5417e8 (/_internal_ping) 2020-03-31 08:57:08,314 [518] [DEBUG] [app] Ending request: urn:request:b480cb7c-4b27-46af-968a-83e4de5417e8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:08,314 [518] [DEBUG] [app] Ending request: urn:request:b480cb7c-4b27-46af-968a-83e4de5417e8 (/_internal_ping) 2020-03-31 08:57:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:57:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:08,327 [541] [DEBUG] [app] Starting request: urn:request:7fabea5c-a1eb-48c2-ad4f-a8c7cdb6d1c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:08,327 [541] [DEBUG] [app] Starting request: urn:request:7fabea5c-a1eb-48c2-ad4f-a8c7cdb6d1c2 (/c1/_internal_ping) 2020-03-31 08:57:08,328 [541] [DEBUG] [app] Ending request: urn:request:7fabea5c-a1eb-48c2-ad4f-a8c7cdb6d1c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:08,328 [541] [DEBUG] [app] Ending request: urn:request:7fabea5c-a1eb-48c2-ad4f-a8c7cdb6d1c2 (/c1/_internal_ping) 2020-03-31 08:57:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:08,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:08,330 [518] [DEBUG] [app] Ending request: urn:request:7cfa36eb-d468-4f4e-82ce-a08dc388d0dd (/health/instance) 2020-03-31 08:57:08,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:57:08,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:57:08,330 [518] [DEBUG] [app] Ending request: urn:request:7cfa36eb-d468-4f4e-82ce-a08dc388d0dd (/health/instance) 2020-03-31 08:57:08,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:57:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:57:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:57:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:21.440707+00:00 (in 9.999037 seconds) 2020-03-31 08:57:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:21 UTC)" (scheduled at 2020-03-31 08:57:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:57:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:21.440707+00:00 (in 9.999037 seconds) 2020-03-31 08:57:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:21 UTC)" (scheduled at 2020-03-31 08:57:11.440707+00:00) 2020-03-31 08:57:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:57:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:57:11,473 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:11,473 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 08:57:11,473 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:11,473 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:11,473 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:57:11,473 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:21 UTC)" executed successfully 2020-03-31 08:57:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:24.323352+00:00 (in 9.999044 seconds) 2020-03-31 08:57:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:24 UTC)" (scheduled at 2020-03-31 08:57:14.323352+00:00) 2020-03-31 08:57:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:57:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:24.323352+00:00 (in 9.999044 seconds) 2020-03-31 08:57:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:24 UTC)" (scheduled at 2020-03-31 08:57:14.323352+00:00) 2020-03-31 08:57:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 14, 325255), True, datetime.datetime(2020, 3, 31, 8, 57, 14, 325255), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:57:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 14, 325255), True, datetime.datetime(2020, 3, 31, 8, 57, 14, 325255), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:57:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:14,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:57:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:24 UTC)" executed successfully 2020-03-31 08:57:14,447 [526] [DEBUG] [app] Starting request: urn:request:d2cf0e0a-601e-40f8-853b-1e58dfb5b181 (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:14,447 [526] [DEBUG] [app] Starting request: urn:request:d2cf0e0a-601e-40f8-853b-1e58dfb5b181 (/health/instance) 2020-03-31 08:57:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:14,454 [552] [DEBUG] [app] Starting request: urn:request:de71f29b-8ba2-4e29-8f16-2bdee5b2be07 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:14,454 [552] [DEBUG] [app] Starting request: urn:request:de71f29b-8ba2-4e29-8f16-2bdee5b2be07 (/v1/_internal_ping) 2020-03-31 08:57:14,455 [552] [DEBUG] [app] Ending request: urn:request:de71f29b-8ba2-4e29-8f16-2bdee5b2be07 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:14,455 [552] [DEBUG] [app] Ending request: urn:request:de71f29b-8ba2-4e29-8f16-2bdee5b2be07 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:57:14,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:14,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:57:14,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:14,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:14,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:14,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:14,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:14,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:14,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:14,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:14,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:14,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:14,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:14,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:14,487 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:14,487 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:14,488 [526] [DEBUG] [app] Starting request: urn:request:34b0ab48-e8e4-47c6-892e-557ef9292cbe (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:14,488 [526] [DEBUG] [app] Starting request: urn:request:34b0ab48-e8e4-47c6-892e-557ef9292cbe (/_internal_ping) 2020-03-31 08:57:14,489 [526] [DEBUG] [app] Ending request: urn:request:34b0ab48-e8e4-47c6-892e-557ef9292cbe (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:14,489 [526] [DEBUG] [app] Ending request: urn:request:34b0ab48-e8e4-47c6-892e-557ef9292cbe (/_internal_ping) 2020-03-31 08:57:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:14,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:14,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:14,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:14,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:14,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:14,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:57:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:57:14,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:14,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:14,501 [541] [DEBUG] [app] Starting request: urn:request:93d1a9e9-28d1-4f35-afb7-bf742f019bcf (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:14,501 [541] [DEBUG] [app] Starting request: urn:request:93d1a9e9-28d1-4f35-afb7-bf742f019bcf (/c1/_internal_ping) 2020-03-31 08:57:14,502 [541] [DEBUG] [app] Ending request: urn:request:93d1a9e9-28d1-4f35-afb7-bf742f019bcf (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:14,502 [541] [DEBUG] [app] Ending request: urn:request:93d1a9e9-28d1-4f35-afb7-bf742f019bcf (/c1/_internal_ping) 2020-03-31 08:57:14,502 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:14,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:14,502 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:14,504 [526] [DEBUG] [app] Ending request: urn:request:d2cf0e0a-601e-40f8-853b-1e58dfb5b181 (/health/instance) 2020-03-31 08:57:14,504 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:57:14,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:57:14,504 [526] [DEBUG] [app] Ending request: urn:request:d2cf0e0a-601e-40f8-853b-1e58dfb5b181 (/health/instance) 2020-03-31 08:57:14,504 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:57:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:57:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:45 UTC)" (scheduled at 2020-03-31 08:57:15.064514+00:00) gcworker stdout | 2020-03-31 08:57:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:45 UTC)" (scheduled at 2020-03-31 08:57:15.064514+00:00) 2020-03-31 08:57:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:45.064514+00:00 (in 29.998904 seconds) gcworker stdout | 2020-03-31 08:57:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:45.064514+00:00 (in 29.998904 seconds) 2020-03-31 08:57:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435435, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:57:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435435, 1209600, 500, 1, 0]) 2020-03-31 08:57:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:57:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:57:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:57:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:45 UTC)" executed successfully 2020-03-31 08:57:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:26.924497+00:00 (in 9.999175 seconds) 2020-03-31 08:57:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:26 UTC)" (scheduled at 2020-03-31 08:57:16.924497+00:00) 2020-03-31 08:57:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:57:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:26.924497+00:00 (in 9.999175 seconds) 2020-03-31 08:57:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:26 UTC)" (scheduled at 2020-03-31 08:57:16.924497+00:00) 2020-03-31 08:57:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 16, 926140), True, datetime.datetime(2020, 3, 31, 8, 57, 16, 926140), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:57:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 16, 926140), True, datetime.datetime(2020, 3, 31, 8, 57, 16, 926140), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:57:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:57:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:26 UTC)" executed successfully 2020-03-31 08:57:18,273 [518] [DEBUG] [app] Starting request: urn:request:737a279a-96b3-4030-a86e-24e7b3b26d2d (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:18,273 [518] [DEBUG] [app] Starting request: urn:request:737a279a-96b3-4030-a86e-24e7b3b26d2d (/health/instance) 2020-03-31 08:57:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:18,281 [538] [DEBUG] [app] Starting request: urn:request:b3524072-a78d-4165-a2b2-2d71350050d5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:18,281 [538] [DEBUG] [app] Starting request: urn:request:b3524072-a78d-4165-a2b2-2d71350050d5 (/v1/_internal_ping) 2020-03-31 08:57:18,282 [538] [DEBUG] [app] Ending request: urn:request:b3524072-a78d-4165-a2b2-2d71350050d5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:18,282 [538] [DEBUG] [app] Ending request: urn:request:b3524072-a78d-4165-a2b2-2d71350050d5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:18,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:57:18,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:57:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:57:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:18,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:18,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:18,314 [527] [DEBUG] [app] Starting request: urn:request:baddb890-51b2-4985-9028-3e1ab7ff0860 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:18,314 [527] [DEBUG] [app] Starting request: urn:request:baddb890-51b2-4985-9028-3e1ab7ff0860 (/_internal_ping) 2020-03-31 08:57:18,315 [527] [DEBUG] [app] Ending request: urn:request:baddb890-51b2-4985-9028-3e1ab7ff0860 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:18,315 [527] [DEBUG] [app] Ending request: urn:request:baddb890-51b2-4985-9028-3e1ab7ff0860 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:57:18,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:57:18,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:57:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:18,327 [539] [DEBUG] [app] Starting request: urn:request:e3f908b3-b457-4c44-b252-c4f8689bd5b7 (/c1/_internal_ping) 2020-03-31 08:57:18,328 [539] [DEBUG] [app] Ending request: urn:request:e3f908b3-b457-4c44-b252-c4f8689bd5b7 (/c1/_internal_ping) 2020-03-31 08:57:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:18,329 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 08:57:18,327 [539] [DEBUG] [app] Starting request: urn:request:e3f908b3-b457-4c44-b252-c4f8689bd5b7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:18,328 [539] [DEBUG] [app] Ending request: urn:request:e3f908b3-b457-4c44-b252-c4f8689bd5b7 (/c1/_internal_ping) 2020-03-31 08:57:18,330 [518] [DEBUG] [app] Ending request: urn:request:737a279a-96b3-4030-a86e-24e7b3b26d2d (/health/instance) 2020-03-31 08:57:18,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:57:18,329 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:18,330 [518] [DEBUG] [app] Ending request: urn:request:737a279a-96b3-4030-a86e-24e7b3b26d2d (/health/instance) 2020-03-31 08:57:18,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:57:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:57:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:31.440707+00:00 (in 9.999102 seconds) 2020-03-31 08:57:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:31 UTC)" (scheduled at 2020-03-31 08:57:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:57:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:31.440707+00:00 (in 9.999102 seconds) 2020-03-31 08:57:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:31 UTC)" (scheduled at 2020-03-31 08:57:21.440707+00:00) 2020-03-31 08:57:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:57:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:57:21,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:57:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:57:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:31 UTC)" executed successfully 2020-03-31 08:57:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:57:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:52.113221+00:00 (in 29.998988 seconds) 2020-03-31 08:57:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:52 UTC)" (scheduled at 2020-03-31 08:57:22.113221+00:00) securityworker stdout | 2020-03-31 08:57:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:52.113221+00:00 (in 29.998988 seconds) 2020-03-31 08:57:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:52 UTC)" (scheduled at 2020-03-31 08:57:22.113221+00:00) 2020-03-31 08:57:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:57:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:57:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:57:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:57:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:57:22,133 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 08:57:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:57:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:57:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:57:52 UTC)" executed successfully 2020-03-31 08:57:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:24 UTC)" (scheduled at 2020-03-31 08:57:24.323352+00:00) 2020-03-31 08:57:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:57:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:24 UTC)" (scheduled at 2020-03-31 08:57:24.323352+00:00) 2020-03-31 08:57:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:31.498397+00:00 (in 7.174181 seconds) notificationworker stdout | 2020-03-31 08:57:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:31.498397+00:00 (in 7.174181 seconds) 2020-03-31 08:57:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 24, 325101), True, datetime.datetime(2020, 3, 31, 8, 57, 24, 325101), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:57:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 24, 325101), True, datetime.datetime(2020, 3, 31, 8, 57, 24, 325101), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:57:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:57:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:34 UTC)" executed successfully 2020-03-31 08:57:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:57:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:33.330112+00:00 (in 8.917950 seconds) 2020-03-31 08:57:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:24 UTC)" (scheduled at 2020-03-31 08:57:24.410927+00:00) 2020-03-31 08:57:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:57:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:33.330112+00:00 (in 8.917950 seconds) 2020-03-31 08:57:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:24 UTC)" (scheduled at 2020-03-31 08:57:24.410927+00:00) 2020-03-31 08:57:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:24 UTC)" executed successfully 2020-03-31 08:57:24,446 [518] [DEBUG] [app] Starting request: urn:request:a9a1d7cc-3e2e-486d-8842-3c2a36e46051 (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:24,446 [518] [DEBUG] [app] Starting request: urn:request:a9a1d7cc-3e2e-486d-8842-3c2a36e46051 (/health/instance) 2020-03-31 08:57:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:24,454 [538] [DEBUG] [app] Starting request: urn:request:88cae756-34e6-4639-99a7-78f228063aeb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:24,454 [538] [DEBUG] [app] Starting request: urn:request:88cae756-34e6-4639-99a7-78f228063aeb (/v1/_internal_ping) 2020-03-31 08:57:24,455 [538] [DEBUG] [app] Ending request: urn:request:88cae756-34e6-4639-99a7-78f228063aeb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:24,455 [538] [DEBUG] [app] Ending request: urn:request:88cae756-34e6-4639-99a7-78f228063aeb (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:57:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:57:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:24,488 [518] [DEBUG] [app] Starting request: urn:request:fa319f41-b64e-4dba-b2fa-baedfe7e2652 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:24,488 [518] [DEBUG] [app] Starting request: urn:request:fa319f41-b64e-4dba-b2fa-baedfe7e2652 (/_internal_ping) 2020-03-31 08:57:24,489 [518] [DEBUG] [app] Ending request: urn:request:fa319f41-b64e-4dba-b2fa-baedfe7e2652 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:24,489 [518] [DEBUG] [app] Ending request: urn:request:fa319f41-b64e-4dba-b2fa-baedfe7e2652 (/_internal_ping) 2020-03-31 08:57:24,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:24,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:57:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:57:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:57:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:57:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:57:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:24,503 [536] [DEBUG] [app] Starting request: urn:request:810b9933-1630-4d20-a4fa-b117a9cb1d3a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:24,503 [536] [DEBUG] [app] Starting request: urn:request:810b9933-1630-4d20-a4fa-b117a9cb1d3a (/c1/_internal_ping) 2020-03-31 08:57:24,504 [536] [DEBUG] [app] Ending request: urn:request:810b9933-1630-4d20-a4fa-b117a9cb1d3a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:24,504 [536] [DEBUG] [app] Ending request: urn:request:810b9933-1630-4d20-a4fa-b117a9cb1d3a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:24,505 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:24,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:57:24,505 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:24,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:24,506 [518] [DEBUG] [app] Ending request: urn:request:a9a1d7cc-3e2e-486d-8842-3c2a36e46051 (/health/instance) 2020-03-31 08:57:24,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:24,506 [518] [DEBUG] [app] Ending request: urn:request:a9a1d7cc-3e2e-486d-8842-3c2a36e46051 (/health/instance) 2020-03-31 08:57:24,506 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:24,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 08:57:24,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 08:57:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:36.924497+00:00 (in 9.999177 seconds) 2020-03-31 08:57:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:36 UTC)" (scheduled at 2020-03-31 08:57:26.924497+00:00) 2020-03-31 08:57:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:57:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:36.924497+00:00 (in 9.999177 seconds) 2020-03-31 08:57:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:36 UTC)" (scheduled at 2020-03-31 08:57:26.924497+00:00) 2020-03-31 08:57:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 26, 925919), True, datetime.datetime(2020, 3, 31, 8, 57, 26, 925919), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:57:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 26, 925919), True, datetime.datetime(2020, 3, 31, 8, 57, 26, 925919), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:57:26,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:57:26,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:36 UTC)" executed successfully 2020-03-31 08:57:28,274 [518] [DEBUG] [app] Starting request: urn:request:42f3026c-4355-478b-a485-68a74889050f (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:28,274 [518] [DEBUG] [app] Starting request: urn:request:42f3026c-4355-478b-a485-68a74889050f (/health/instance) 2020-03-31 08:57:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:28,279 [556] [DEBUG] [app] Starting request: urn:request:a1dcc504-09eb-496d-b726-b8dd0cc6ed78 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:28,279 [556] [DEBUG] [app] Starting request: urn:request:a1dcc504-09eb-496d-b726-b8dd0cc6ed78 (/v1/_internal_ping) 2020-03-31 08:57:28,281 [556] [DEBUG] [app] Ending request: urn:request:a1dcc504-09eb-496d-b726-b8dd0cc6ed78 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:28,281 [556] [DEBUG] [app] Ending request: urn:request:a1dcc504-09eb-496d-b726-b8dd0cc6ed78 (/v1/_internal_ping) 2020-03-31 08:57:28,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:57:28,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:28,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:57:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:28,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:28,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:28,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:28,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:28,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:28,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:28,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:28,314 [518] [DEBUG] [app] Starting request: urn:request:796a3e56-4c9f-4223-bb14-89c624bc9d05 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:28,314 [518] [DEBUG] [app] Starting request: urn:request:796a3e56-4c9f-4223-bb14-89c624bc9d05 (/_internal_ping) 2020-03-31 08:57:28,315 [518] [DEBUG] [app] Ending request: urn:request:796a3e56-4c9f-4223-bb14-89c624bc9d05 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:28,315 [518] [DEBUG] [app] Ending request: urn:request:796a3e56-4c9f-4223-bb14-89c624bc9d05 (/_internal_ping) 2020-03-31 08:57:28,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:28,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:57:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:28,329 [536] [DEBUG] [app] Starting request: urn:request:4afa9bf1-7872-4ee7-b576-582392530661 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:28,329 [536] [DEBUG] [app] Starting request: urn:request:4afa9bf1-7872-4ee7-b576-582392530661 (/c1/_internal_ping) 2020-03-31 08:57:28,329 [536] [DEBUG] [app] Ending request: urn:request:4afa9bf1-7872-4ee7-b576-582392530661 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:28,329 [536] [DEBUG] [app] Ending request: urn:request:4afa9bf1-7872-4ee7-b576-582392530661 (/c1/_internal_ping) 2020-03-31 08:57:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:28,330 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:28,331 [518] [DEBUG] [app] Ending request: urn:request:42f3026c-4355-478b-a485-68a74889050f (/health/instance) 2020-03-31 08:57:28,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:57:28,330 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:57:28,331 [518] [DEBUG] [app] Ending request: urn:request:42f3026c-4355-478b-a485-68a74889050f (/health/instance) 2020-03-31 08:57:28,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:28,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:57:28,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:57:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:57:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:00.337699+00:00 (in 29.998844 seconds) 2020-03-31 08:57:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:00 UTC)" (scheduled at 2020-03-31 08:57:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:57:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:00.337699+00:00 (in 29.998844 seconds) 2020-03-31 08:57:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:00 UTC)" (scheduled at 2020-03-31 08:57:30.337699+00:00) 2020-03-31 08:57:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 57, 30, 339684), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:57:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 57, 30, 339684), False, 50, 1, 0]) 2020-03-31 08:57:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:57:30,357 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 08:57:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:57:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:57:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:00 UTC)" executed successfully 2020-03-31 08:57:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:57:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:41.440707+00:00 (in 9.999107 seconds) 2020-03-31 08:57:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:41 UTC)" (scheduled at 2020-03-31 08:57:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:57:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:41.440707+00:00 (in 9.999107 seconds) 2020-03-31 08:57:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:41 UTC)" (scheduled at 2020-03-31 08:57:31.440707+00:00) 2020-03-31 08:57:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:57:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:57:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:57:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:57:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:41 UTC)" executed successfully 2020-03-31 08:57:31,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:31,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:31,502 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:31 UTC)" (scheduled at 2020-03-31 08:57:31.498397+00:00) notificationworker stdout | 2020-03-31 08:57:31,502 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:31 UTC)" (scheduled at 2020-03-31 08:57:31.498397+00:00) 2020-03-31 08:57:31,503 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), u'notification/%']) 2020-03-31 08:57:31,504 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:34.323352+00:00 (in 2.821252 seconds) notificationworker stdout | 2020-03-31 08:57:31,503 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), u'notification/%']) 2020-03-31 08:57:31,504 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:34.323352+00:00 (in 2.821252 seconds) 2020-03-31 08:57:31,525 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), True, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), 0, u'notification/%']) notificationworker stdout | 2020-03-31 08:57:31,525 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), True, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), 0, u'notification/%']) 2020-03-31 08:57:31,532 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), True, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), u'notification/%']) notificationworker stdout | 2020-03-31 08:57:31,532 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), True, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 8, 57, 31, 502799), u'notification/%']) 2020-03-31 08:57:31,540 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:57:31,540 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:31,541 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:31,541 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:02:31 UTC)" executed successfully 2020-03-31 08:57:31,542 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 08:57:31,542 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:31,543 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 08:57:31,543 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:31,544 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics notificationworker stdout | 2020-03-31 08:57:31,544 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:31,546 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 08:57:31,546 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:31,547 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 08:57:31,547 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:31,548 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics notificationworker stdout | 2020-03-31 08:57:31,548 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 08:57:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:57:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:03.330112+00:00 (in 29.999209 seconds) 2020-03-31 08:57:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:03 UTC)" (scheduled at 2020-03-31 08:57:33.330112+00:00) 2020-03-31 08:57:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:57:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:03.330112+00:00 (in 29.999209 seconds) 2020-03-31 08:57:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:03 UTC)" (scheduled at 2020-03-31 08:57:33.330112+00:00) 2020-03-31 08:57:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 33, 331908), True, datetime.datetime(2020, 3, 31, 8, 57, 33, 331908), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:57:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 33, 331908), True, datetime.datetime(2020, 3, 31, 8, 57, 33, 331908), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:57:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:57:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:57:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:03 UTC)" executed successfully 2020-03-31 08:57:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:57:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:33 UTC)" (scheduled at 2020-03-31 08:57:33.366791+00:00) 2020-03-31 08:57:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:57:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:57:33 UTC)" (scheduled at 2020-03-31 08:57:33.366791+00:00) 2020-03-31 08:57:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:33,369 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:43.539999+00:00 (in 10.172002 seconds) exportactionlogsworker stdout | 2020-03-31 08:57:33,369 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:43.539999+00:00 (in 10.172002 seconds) 2020-03-31 08:57:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 33, 368713), True, datetime.datetime(2020, 3, 31, 8, 57, 33, 368713), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:57:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 33, 368713), True, datetime.datetime(2020, 3, 31, 8, 57, 33, 368713), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:57:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:57:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:57:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:33 UTC)" executed successfully 2020-03-31 08:57:34,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:34,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:34 UTC)" (scheduled at 2020-03-31 08:57:34.323352+00:00) 2020-03-31 08:57:34,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:57:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:34 UTC)" (scheduled at 2020-03-31 08:57:34.323352+00:00) 2020-03-31 08:57:34,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:34,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:44.323352+00:00 (in 9.997374 seconds) notificationworker stdout | 2020-03-31 08:57:34,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:44.323352+00:00 (in 9.997374 seconds) 2020-03-31 08:57:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 34, 326733), True, datetime.datetime(2020, 3, 31, 8, 57, 34, 326733), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:57:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 34, 326733), True, datetime.datetime(2020, 3, 31, 8, 57, 34, 326733), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:57:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:34,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:57:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:44 UTC)" executed successfully 2020-03-31 08:57:34,446 [527] [DEBUG] [app] Starting request: urn:request:64ed66a9-ad2a-4649-87ed-5c25fdbe098f (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:34,446 [527] [DEBUG] [app] Starting request: urn:request:64ed66a9-ad2a-4649-87ed-5c25fdbe098f (/health/instance) 2020-03-31 08:57:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:34,455 [550] [DEBUG] [app] Starting request: urn:request:a202521c-9886-4f18-81ef-453219dce8fb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:34,455 [550] [DEBUG] [app] Starting request: urn:request:a202521c-9886-4f18-81ef-453219dce8fb (/v1/_internal_ping) 2020-03-31 08:57:34,456 [550] [DEBUG] [app] Ending request: urn:request:a202521c-9886-4f18-81ef-453219dce8fb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:34,456 [550] [DEBUG] [app] Ending request: urn:request:a202521c-9886-4f18-81ef-453219dce8fb (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:34,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:57:34,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:34,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:34,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:34,463 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:34,463 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:34,464 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:34,464 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:34,465 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:34,465 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:34,474 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:34,474 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:34,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:34,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:34,491 [518] [DEBUG] [app] Starting request: urn:request:dd14534b-436a-4200-ab77-b489ca79dfaf (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:34,491 [518] [DEBUG] [app] Starting request: urn:request:dd14534b-436a-4200-ab77-b489ca79dfaf (/_internal_ping) 2020-03-31 08:57:34,492 [518] [DEBUG] [app] Ending request: urn:request:dd14534b-436a-4200-ab77-b489ca79dfaf (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:34,492 [518] [DEBUG] [app] Ending request: urn:request:dd14534b-436a-4200-ab77-b489ca79dfaf (/_internal_ping) 2020-03-31 08:57:34,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:57:34,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:57:34,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:34,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:34,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:34,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:57:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:57:34,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:34,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:34,508 [537] [DEBUG] [app] Starting request: urn:request:94c9f9e8-cfaf-47d4-80e0-b15f66fc0e97 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:34,508 [537] [DEBUG] [app] Starting request: urn:request:94c9f9e8-cfaf-47d4-80e0-b15f66fc0e97 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:34,508 [537] [DEBUG] [app] Ending request: urn:request:94c9f9e8-cfaf-47d4-80e0-b15f66fc0e97 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:34,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:34,509 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:57:34,508 [537] [DEBUG] [app] Ending request: urn:request:94c9f9e8-cfaf-47d4-80e0-b15f66fc0e97 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:34,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:34,510 [527] [DEBUG] [app] Ending request: urn:request:64ed66a9-ad2a-4649-87ed-5c25fdbe098f (/health/instance) 2020-03-31 08:57:34,511 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:57:34,509 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:34,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 08:57:34,510 [527] [DEBUG] [app] Ending request: urn:request:64ed66a9-ad2a-4649-87ed-5c25fdbe098f (/health/instance) 2020-03-31 08:57:34,511 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:34,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 08:57:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:38.010934+00:00 (in 1.085753 seconds) 2020-03-31 08:57:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:46 UTC)" (scheduled at 2020-03-31 08:57:36.924497+00:00) 2020-03-31 08:57:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:57:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:38.010934+00:00 (in 1.085753 seconds) 2020-03-31 08:57:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:46 UTC)" (scheduled at 2020-03-31 08:57:36.924497+00:00) 2020-03-31 08:57:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 36, 925976), True, datetime.datetime(2020, 3, 31, 8, 57, 36, 925976), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:57:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 36, 925976), True, datetime.datetime(2020, 3, 31, 8, 57, 36, 925976), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:57:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:57:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:46 UTC)" executed successfully 2020-03-31 08:57:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:46.924497+00:00 (in 8.912809 seconds) 2020-03-31 08:57:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:38 UTC)" (scheduled at 2020-03-31 08:57:38.010934+00:00) 2020-03-31 08:57:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:46.924497+00:00 (in 8.912809 seconds) 2020-03-31 08:57:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:38 UTC)" (scheduled at 2020-03-31 08:57:38.010934+00:00) 2020-03-31 08:57:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:38 UTC)" executed successfully 2020-03-31 08:57:38,273 [518] [DEBUG] [app] Starting request: urn:request:f7ca8630-b30f-482e-98da-0f17f6175c3d (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:38,273 [518] [DEBUG] [app] Starting request: urn:request:f7ca8630-b30f-482e-98da-0f17f6175c3d (/health/instance) 2020-03-31 08:57:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:38,280 [552] [DEBUG] [app] Starting request: urn:request:6f932d52-c6d0-4ba4-85d6-0b157e792249 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:38,280 [552] [DEBUG] [app] Starting request: urn:request:6f932d52-c6d0-4ba4-85d6-0b157e792249 (/v1/_internal_ping) 2020-03-31 08:57:38,282 [552] [DEBUG] [app] Ending request: urn:request:6f932d52-c6d0-4ba4-85d6-0b157e792249 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:38,282 [552] [DEBUG] [app] Ending request: urn:request:6f932d52-c6d0-4ba4-85d6-0b157e792249 (/v1/_internal_ping) 2020-03-31 08:57:38,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:38,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:57:38,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:38,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:38,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:57:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:57:38,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:38,287 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:38,287 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:38,288 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:38,288 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:38,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:38,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:38,313 [518] [DEBUG] [app] Starting request: urn:request:778c9ef2-e5fe-4f2d-a13a-669bd5f11e7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:38,313 [518] [DEBUG] [app] Starting request: urn:request:778c9ef2-e5fe-4f2d-a13a-669bd5f11e7b (/_internal_ping) 2020-03-31 08:57:38,314 [518] [DEBUG] [app] Ending request: urn:request:778c9ef2-e5fe-4f2d-a13a-669bd5f11e7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:38,314 [518] [DEBUG] [app] Ending request: urn:request:778c9ef2-e5fe-4f2d-a13a-669bd5f11e7b (/_internal_ping) 2020-03-31 08:57:38,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:57:38,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:57:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:57:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:38,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:38,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:38,327 [541] [DEBUG] [app] Starting request: urn:request:d8398f02-4d86-4ad5-9b12-30f7e1cea595 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:38,327 [541] [DEBUG] [app] Starting request: urn:request:d8398f02-4d86-4ad5-9b12-30f7e1cea595 (/c1/_internal_ping) 2020-03-31 08:57:38,328 [541] [DEBUG] [app] Ending request: urn:request:d8398f02-4d86-4ad5-9b12-30f7e1cea595 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:38,328 [541] [DEBUG] [app] Ending request: urn:request:d8398f02-4d86-4ad5-9b12-30f7e1cea595 (/c1/_internal_ping) 2020-03-31 08:57:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:38,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:57:38,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:38,330 [518] [DEBUG] [app] Ending request: urn:request:f7ca8630-b30f-482e-98da-0f17f6175c3d (/health/instance) 2020-03-31 08:57:38,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:38,330 [518] [DEBUG] [app] Ending request: urn:request:f7ca8630-b30f-482e-98da-0f17f6175c3d (/health/instance) 2020-03-31 08:57:38,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:57:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:57:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:57:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:51.440707+00:00 (in 9.999190 seconds) 2020-03-31 08:57:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:51 UTC)" (scheduled at 2020-03-31 08:57:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:57:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:51.440707+00:00 (in 9.999190 seconds) 2020-03-31 08:57:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:51 UTC)" (scheduled at 2020-03-31 08:57:41.440707+00:00) 2020-03-31 08:57:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:57:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:57:41,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:57:41,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:41,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:57:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:41,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:51 UTC)" executed successfully 2020-03-31 08:57:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:57:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:33.366791+00:00 (in 49.825135 seconds) 2020-03-31 08:57:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:43 UTC)" (scheduled at 2020-03-31 08:57:43.539999+00:00) 2020-03-31 08:57:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:57:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:33.366791+00:00 (in 49.825135 seconds) 2020-03-31 08:57:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:43 UTC)" (scheduled at 2020-03-31 08:57:43.539999+00:00) 2020-03-31 08:57:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:43 UTC)" executed successfully 2020-03-31 08:57:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:49.498966+00:00 (in 5.173568 seconds) 2020-03-31 08:57:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:54 UTC)" (scheduled at 2020-03-31 08:57:44.323352+00:00) 2020-03-31 08:57:44,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:57:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:49.498966+00:00 (in 5.173568 seconds) 2020-03-31 08:57:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:54 UTC)" (scheduled at 2020-03-31 08:57:44.323352+00:00) 2020-03-31 08:57:44,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 44, 326333), True, datetime.datetime(2020, 3, 31, 8, 57, 44, 326333), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:57:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 44, 326333), True, datetime.datetime(2020, 3, 31, 8, 57, 44, 326333), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:57:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:57:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:54 UTC)" executed successfully 2020-03-31 08:57:44,446 [518] [DEBUG] [app] Starting request: urn:request:523d7de8-6158-474c-917f-1a5751ffa61c (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:44,446 [518] [DEBUG] [app] Starting request: urn:request:523d7de8-6158-474c-917f-1a5751ffa61c (/health/instance) 2020-03-31 08:57:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:44,454 [552] [DEBUG] [app] Starting request: urn:request:71257bab-fb16-4f14-bbaf-9e50953d72b3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:44,454 [552] [DEBUG] [app] Starting request: urn:request:71257bab-fb16-4f14-bbaf-9e50953d72b3 (/v1/_internal_ping) 2020-03-31 08:57:44,455 [552] [DEBUG] [app] Ending request: urn:request:71257bab-fb16-4f14-bbaf-9e50953d72b3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:44,455 [552] [DEBUG] [app] Ending request: urn:request:71257bab-fb16-4f14-bbaf-9e50953d72b3 (/v1/_internal_ping) 2020-03-31 08:57:44,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:57:44,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:57:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:44,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:44,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:44,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:44,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:44,459 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:44,459 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:44,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:44,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:44,485 [527] [DEBUG] [app] Starting request: urn:request:081afade-a305-4017-a8bf-0bf30090e022 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:44,485 [527] [DEBUG] [app] Starting request: urn:request:081afade-a305-4017-a8bf-0bf30090e022 (/_internal_ping) 2020-03-31 08:57:44,486 [527] [DEBUG] [app] Ending request: urn:request:081afade-a305-4017-a8bf-0bf30090e022 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:44,486 [527] [DEBUG] [app] Ending request: urn:request:081afade-a305-4017-a8bf-0bf30090e022 (/_internal_ping) 2020-03-31 08:57:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:57:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:57:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:57:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:44,499 [537] [DEBUG] [app] Starting request: urn:request:2b47eaca-6b54-49e0-b6bd-a293c529532a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:44,499 [537] [DEBUG] [app] Starting request: urn:request:2b47eaca-6b54-49e0-b6bd-a293c529532a (/c1/_internal_ping) 2020-03-31 08:57:44,500 [537] [DEBUG] [app] Ending request: urn:request:2b47eaca-6b54-49e0-b6bd-a293c529532a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:44,500 [537] [DEBUG] [app] Ending request: urn:request:2b47eaca-6b54-49e0-b6bd-a293c529532a (/c1/_internal_ping) 2020-03-31 08:57:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:44,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:57:44,503 [518] [DEBUG] [app] Ending request: urn:request:523d7de8-6158-474c-917f-1a5751ffa61c (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:57:44,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:44,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 08:57:44,503 [518] [DEBUG] [app] Ending request: urn:request:523d7de8-6158-474c-917f-1a5751ffa61c (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:44,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 08:57:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:57:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:15.064514+00:00 (in 29.996257 seconds) 2020-03-31 08:57:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:15 UTC)" (scheduled at 2020-03-31 08:57:45.064514+00:00) gcworker stdout | 2020-03-31 08:57:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:15.064514+00:00 (in 29.996257 seconds) 2020-03-31 08:57:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:15 UTC)" (scheduled at 2020-03-31 08:57:45.064514+00:00) 2020-03-31 08:57:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435465, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:57:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435465, 1209600, 500, 1, 0]) 2020-03-31 08:57:45,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:57:45,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:57:45,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:57:45,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:15 UTC)" executed successfully 2020-03-31 08:57:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:56.924497+00:00 (in 9.999029 seconds) 2020-03-31 08:57:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:56 UTC)" (scheduled at 2020-03-31 08:57:46.924497+00:00) 2020-03-31 08:57:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:57:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:56.924497+00:00 (in 9.999029 seconds) 2020-03-31 08:57:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:56 UTC)" (scheduled at 2020-03-31 08:57:46.924497+00:00) 2020-03-31 08:57:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 46, 926153), True, datetime.datetime(2020, 3, 31, 8, 57, 46, 926153), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:57:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 46, 926153), True, datetime.datetime(2020, 3, 31, 8, 57, 46, 926153), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:57:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:57:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:57:56 UTC)" executed successfully 2020-03-31 08:57:48,272 [526] [DEBUG] [app] Starting request: urn:request:afe37f70-96e5-4f94-a35c-fbd5bc33d8f8 (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:48,272 [526] [DEBUG] [app] Starting request: urn:request:afe37f70-96e5-4f94-a35c-fbd5bc33d8f8 (/health/instance) 2020-03-31 08:57:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:48,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:48,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:48,281 [548] [DEBUG] [app] Starting request: urn:request:a5eb12b4-3a7b-4cb0-867d-a8be328d70f5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:48,281 [548] [DEBUG] [app] Starting request: urn:request:a5eb12b4-3a7b-4cb0-867d-a8be328d70f5 (/v1/_internal_ping) 2020-03-31 08:57:48,282 [548] [DEBUG] [app] Ending request: urn:request:a5eb12b4-3a7b-4cb0-867d-a8be328d70f5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:48,282 [548] [DEBUG] [app] Ending request: urn:request:a5eb12b4-3a7b-4cb0-867d-a8be328d70f5 (/v1/_internal_ping) 2020-03-31 08:57:48,283 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:57:48,283 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 08:57:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:48,285 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:48,285 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:48,287 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:48,287 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:48,288 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:48,288 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:48,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:48,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:48,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:48,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:48,310 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:48,310 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:48,311 [526] [DEBUG] [app] Starting request: urn:request:28660840-5645-4010-8584-e7b0b9e8d309 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:48,311 [526] [DEBUG] [app] Starting request: urn:request:28660840-5645-4010-8584-e7b0b9e8d309 (/_internal_ping) 2020-03-31 08:57:48,312 [526] [DEBUG] [app] Ending request: urn:request:28660840-5645-4010-8584-e7b0b9e8d309 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:48,312 [526] [DEBUG] [app] Ending request: urn:request:28660840-5645-4010-8584-e7b0b9e8d309 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:57:48,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:48,313 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:57:48,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:48,313 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:57:48,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:48,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:48,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:57:48,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:57:48,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:48,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:57:48,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:48,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:48,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:48,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:48,325 [541] [DEBUG] [app] Starting request: urn:request:f95c08f6-5508-4304-bf2a-46ec9121f805 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:48,325 [541] [DEBUG] [app] Starting request: urn:request:f95c08f6-5508-4304-bf2a-46ec9121f805 (/c1/_internal_ping) 2020-03-31 08:57:48,327 [541] [DEBUG] [app] Ending request: urn:request:f95c08f6-5508-4304-bf2a-46ec9121f805 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:48,327 [541] [DEBUG] [app] Ending request: urn:request:f95c08f6-5508-4304-bf2a-46ec9121f805 (/c1/_internal_ping) 2020-03-31 08:57:48,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:48,327 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:48,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:48,329 [526] [DEBUG] [app] Ending request: urn:request:afe37f70-96e5-4f94-a35c-fbd5bc33d8f8 (/health/instance) 2020-03-31 08:57:48,329 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:57:48,327 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:48,330 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 08:57:48,329 [526] [DEBUG] [app] Ending request: urn:request:afe37f70-96e5-4f94-a35c-fbd5bc33d8f8 (/health/instance) 2020-03-31 08:57:48,329 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:48,330 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 08:57:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:54.323352+00:00 (in 4.823333 seconds) 2020-03-31 08:57:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:49 UTC)" (scheduled at 2020-03-31 08:57:49.498966+00:00) 2020-03-31 08:57:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:54.323352+00:00 (in 4.823333 seconds) 2020-03-31 08:57:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:49 UTC)" (scheduled at 2020-03-31 08:57:49.498966+00:00) 2020-03-31 08:57:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:49 UTC)" executed successfully 2020-03-31 08:57:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:57:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:59.560326+00:00 (in 8.447943 seconds) 2020-03-31 08:57:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:51 UTC)" (scheduled at 2020-03-31 08:57:51.110610+00:00) 2020-03-31 08:57:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:57:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:57:59.560326+00:00 (in 8.447943 seconds) 2020-03-31 08:57:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:51 UTC)" (scheduled at 2020-03-31 08:57:51.110610+00:00) 2020-03-31 08:57:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 51, 113126), True, datetime.datetime(2020, 3, 31, 8, 57, 51, 113126), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:57:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 51, 113126), True, datetime.datetime(2020, 3, 31, 8, 57, 51, 113126), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:57:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:57:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:57:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:51 UTC)" executed successfully 2020-03-31 08:57:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:57:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:01.440707+00:00 (in 9.998735 seconds) 2020-03-31 08:57:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:01 UTC)" (scheduled at 2020-03-31 08:57:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:57:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:01.440707+00:00 (in 9.998735 seconds) 2020-03-31 08:57:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:01 UTC)" (scheduled at 2020-03-31 08:57:51.440707+00:00) 2020-03-31 08:57:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:57:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:57:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:57:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:57:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:57:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:01 UTC)" executed successfully 2020-03-31 08:57:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:57:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:22.113221+00:00 (in 29.998891 seconds) 2020-03-31 08:57:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:22 UTC)" (scheduled at 2020-03-31 08:57:52.113221+00:00) securityworker stdout | 2020-03-31 08:57:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:22.113221+00:00 (in 29.998891 seconds) 2020-03-31 08:57:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:22 UTC)" (scheduled at 2020-03-31 08:57:52.113221+00:00) 2020-03-31 08:57:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:57:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:57:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:57:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:57:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:57:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:57:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:57:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:22 UTC)" executed successfully 2020-03-31 08:57:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:57:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:04.323352+00:00 (in 9.999046 seconds) 2020-03-31 08:57:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:04 UTC)" (scheduled at 2020-03-31 08:57:54.323352+00:00) 2020-03-31 08:57:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:57:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:04.323352+00:00 (in 9.999046 seconds) 2020-03-31 08:57:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:04 UTC)" (scheduled at 2020-03-31 08:57:54.323352+00:00) 2020-03-31 08:57:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 54, 325256), True, datetime.datetime(2020, 3, 31, 8, 57, 54, 325256), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:57:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 54, 325256), True, datetime.datetime(2020, 3, 31, 8, 57, 54, 325256), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:57:54,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:57:54,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:54,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:54,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:57:54,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:54,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:04 UTC)" executed successfully 2020-03-31 08:57:54,446 [527] [DEBUG] [app] Starting request: urn:request:2e1ab498-030b-4357-aa33-c9a47abe5039 (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:54,446 [527] [DEBUG] [app] Starting request: urn:request:2e1ab498-030b-4357-aa33-c9a47abe5039 (/health/instance) 2020-03-31 08:57:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:54,454 [555] [DEBUG] [app] Starting request: urn:request:375409be-cdf7-47e4-bb1b-df838a5de70f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:54,454 [555] [DEBUG] [app] Starting request: urn:request:375409be-cdf7-47e4-bb1b-df838a5de70f (/v1/_internal_ping) 2020-03-31 08:57:54,456 [555] [DEBUG] [app] Ending request: urn:request:375409be-cdf7-47e4-bb1b-df838a5de70f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:54,456 [555] [DEBUG] [app] Ending request: urn:request:375409be-cdf7-47e4-bb1b-df838a5de70f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:54,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:57:54,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:57:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:54,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:54,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:54,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:54,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:54,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:54,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:54,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:54,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:54,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:54,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:54,485 [518] [DEBUG] [app] Starting request: urn:request:751c40f8-fd48-468d-ac43-8ac8eb664e65 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:54,485 [518] [DEBUG] [app] Starting request: urn:request:751c40f8-fd48-468d-ac43-8ac8eb664e65 (/_internal_ping) 2020-03-31 08:57:54,486 [518] [DEBUG] [app] Ending request: urn:request:751c40f8-fd48-468d-ac43-8ac8eb664e65 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:57:54,486 [518] [DEBUG] [app] Ending request: urn:request:751c40f8-fd48-468d-ac43-8ac8eb664e65 (/_internal_ping) 2020-03-31 08:57:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:54,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:54,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:57:54,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:54,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:57:54,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:54,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:54,500 [534] [DEBUG] [app] Starting request: urn:request:79950bfc-76ae-4474-80ee-e338245fe09a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:54,500 [534] [DEBUG] [app] Starting request: urn:request:79950bfc-76ae-4474-80ee-e338245fe09a (/c1/_internal_ping) 2020-03-31 08:57:54,501 [534] [DEBUG] [app] Ending request: urn:request:79950bfc-76ae-4474-80ee-e338245fe09a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:54,501 [534] [DEBUG] [app] Ending request: urn:request:79950bfc-76ae-4474-80ee-e338245fe09a (/c1/_internal_ping) 2020-03-31 08:57:54,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:54,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:54,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:54,504 [527] [DEBUG] [app] Ending request: urn:request:2e1ab498-030b-4357-aa33-c9a47abe5039 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:57:54,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:54,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:54,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:57:54,504 [527] [DEBUG] [app] Ending request: urn:request:2e1ab498-030b-4357-aa33-c9a47abe5039 (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:54,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:57:54,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:57:56,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:57:56,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:06.924497+00:00 (in 9.999352 seconds) 2020-03-31 08:57:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:06 UTC)" (scheduled at 2020-03-31 08:57:56.924497+00:00) 2020-03-31 08:57:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:57:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:06.924497+00:00 (in 9.999352 seconds) 2020-03-31 08:57:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:06 UTC)" (scheduled at 2020-03-31 08:57:56.924497+00:00) 2020-03-31 08:57:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:57:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 56, 925904), True, datetime.datetime(2020, 3, 31, 8, 57, 56, 925904), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:57:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 57, 56, 925904), True, datetime.datetime(2020, 3, 31, 8, 57, 56, 925904), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:57:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:57:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:57:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:57:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:06 UTC)" executed successfully 2020-03-31 08:57:58,273 [518] [DEBUG] [app] Starting request: urn:request:199114d5-0a2a-4a3a-ae35-cf90bd643a6f (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:58,273 [518] [DEBUG] [app] Starting request: urn:request:199114d5-0a2a-4a3a-ae35-cf90bd643a6f (/health/instance) 2020-03-31 08:57:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:58,280 [549] [DEBUG] [app] Starting request: urn:request:34a34330-15ee-46ea-81cf-61d224562732 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:58,280 [549] [DEBUG] [app] Starting request: urn:request:34a34330-15ee-46ea-81cf-61d224562732 (/v1/_internal_ping) 2020-03-31 08:57:58,281 [549] [DEBUG] [app] Ending request: urn:request:34a34330-15ee-46ea-81cf-61d224562732 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:57:58,281 [549] [DEBUG] [app] Ending request: urn:request:34a34330-15ee-46ea-81cf-61d224562732 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:58,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:57:58,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:58,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:57:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:57:58,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:57:58,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:57:58,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:57:58,285 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:57:58,285 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:57:58,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:57:58,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:57:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:57:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:57:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:57:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:57:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:58,314 [527] [DEBUG] [app] Starting request: urn:request:e0c6adb6-13cd-41fb-8828-57440f6984fa (/_internal_ping) 2020-03-31 08:57:58,315 [527] [DEBUG] [app] Ending request: urn:request:e0c6adb6-13cd-41fb-8828-57440f6984fa (/_internal_ping) 2020-03-31 08:57:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:57:58,314 [527] [DEBUG] [app] Starting request: urn:request:e0c6adb6-13cd-41fb-8828-57440f6984fa (/_internal_ping) 2020-03-31 08:57:58,315 [527] [DEBUG] [app] Ending request: urn:request:e0c6adb6-13cd-41fb-8828-57440f6984fa (/_internal_ping) 2020-03-31 08:57:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:57:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:57:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:57:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:57:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:57:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:57:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:57:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:57:58,326 [536] [DEBUG] [app] Starting request: urn:request:176b5984-9f13-4fcb-9bf1-a6d37c3fe132 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:58,326 [536] [DEBUG] [app] Starting request: urn:request:176b5984-9f13-4fcb-9bf1-a6d37c3fe132 (/c1/_internal_ping) 2020-03-31 08:57:58,327 [536] [DEBUG] [app] Ending request: urn:request:176b5984-9f13-4fcb-9bf1-a6d37c3fe132 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:57:58,327 [536] [DEBUG] [app] Ending request: urn:request:176b5984-9f13-4fcb-9bf1-a6d37c3fe132 (/c1/_internal_ping) 2020-03-31 08:57:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:58,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:57:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:57:58,330 [518] [DEBUG] [app] Ending request: urn:request:199114d5-0a2a-4a3a-ae35-cf90bd643a6f (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:57:58,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:57:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:57:58,330 [518] [DEBUG] [app] Ending request: urn:request:199114d5-0a2a-4a3a-ae35-cf90bd643a6f (/health/instance) gunicorn-web stdout | 2020-03-31 08:57:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:57:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:57:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:57:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:57:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:57:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:57:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:51.110610+00:00 (in 51.549371 seconds) 2020-03-31 08:57:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:59 UTC)" (scheduled at 2020-03-31 08:57:59.560326+00:00) 2020-03-31 08:57:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 08:57:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:51.110610+00:00 (in 51.549371 seconds) 2020-03-31 08:57:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:59 UTC)" (scheduled at 2020-03-31 08:57:59.560326+00:00) 2020-03-31 08:57:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:57:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:57:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:58:59 UTC)" executed successfully 2020-03-31 08:58:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:58:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:30.337699+00:00 (in 29.999195 seconds) 2020-03-31 08:58:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:30 UTC)" (scheduled at 2020-03-31 08:58:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:58:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:30.337699+00:00 (in 29.999195 seconds) 2020-03-31 08:58:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:30 UTC)" (scheduled at 2020-03-31 08:58:00.337699+00:00) 2020-03-31 08:58:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 58, 0, 339316), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:58:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 58, 0, 339316), False, 50, 1, 0]) 2020-03-31 08:58:00,360 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:58:00,360 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:58:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:58:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:30 UTC)" executed successfully 2020-03-31 08:58:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:58:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:11.440707+00:00 (in 9.999208 seconds) 2020-03-31 08:58:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:11 UTC)" (scheduled at 2020-03-31 08:58:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:58:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:11.440707+00:00 (in 9.999208 seconds) 2020-03-31 08:58:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:11 UTC)" (scheduled at 2020-03-31 08:58:01.440707+00:00) 2020-03-31 08:58:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:58:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:58:01,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:58:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:58:01,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:58:01,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:11 UTC)" executed successfully 2020-03-31 08:58:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:58:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:24.410927+00:00 (in 21.079768 seconds) 2020-03-31 08:58:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:33 UTC)" (scheduled at 2020-03-31 08:58:03.330112+00:00) 2020-03-31 08:58:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:58:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:24.410927+00:00 (in 21.079768 seconds) 2020-03-31 08:58:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:33 UTC)" (scheduled at 2020-03-31 08:58:03.330112+00:00) 2020-03-31 08:58:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 3, 332013), True, datetime.datetime(2020, 3, 31, 8, 58, 3, 332013), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:58:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 3, 332013), True, datetime.datetime(2020, 3, 31, 8, 58, 3, 332013), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:58:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:58:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:58:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:33 UTC)" executed successfully 2020-03-31 08:58:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:58:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:14.323352+00:00 (in 9.998912 seconds) 2020-03-31 08:58:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:14 UTC)" (scheduled at 2020-03-31 08:58:04.323352+00:00) 2020-03-31 08:58:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:58:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:14.323352+00:00 (in 9.998912 seconds) 2020-03-31 08:58:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:14 UTC)" (scheduled at 2020-03-31 08:58:04.323352+00:00) 2020-03-31 08:58:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 4, 325275), True, datetime.datetime(2020, 3, 31, 8, 58, 4, 325275), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:58:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 4, 325275), True, datetime.datetime(2020, 3, 31, 8, 58, 4, 325275), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:58:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:58:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:58:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:14 UTC)" executed successfully 2020-03-31 08:58:04,446 [518] [DEBUG] [app] Starting request: urn:request:5e228181-0128-4eca-b2e2-543f8bf1dbd1 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:04,446 [518] [DEBUG] [app] Starting request: urn:request:5e228181-0128-4eca-b2e2-543f8bf1dbd1 (/health/instance) 2020-03-31 08:58:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:04,455 [556] [DEBUG] [app] Starting request: urn:request:adf54254-a9d5-4d64-a02d-99d042823695 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:04,455 [556] [DEBUG] [app] Starting request: urn:request:adf54254-a9d5-4d64-a02d-99d042823695 (/v1/_internal_ping) 2020-03-31 08:58:04,456 [556] [DEBUG] [app] Ending request: urn:request:adf54254-a9d5-4d64-a02d-99d042823695 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:04,456 [556] [DEBUG] [app] Ending request: urn:request:adf54254-a9d5-4d64-a02d-99d042823695 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:04,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:58:04,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:04,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:04,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:04,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:04,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:04,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:04,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:04,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:04,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:04,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:04,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:04,495 [518] [DEBUG] [app] Starting request: urn:request:6ad41e96-fcd3-408c-a46e-4c476a6e2c3d (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:04,495 [518] [DEBUG] [app] Starting request: urn:request:6ad41e96-fcd3-408c-a46e-4c476a6e2c3d (/_internal_ping) 2020-03-31 08:58:04,496 [518] [DEBUG] [app] Ending request: urn:request:6ad41e96-fcd3-408c-a46e-4c476a6e2c3d (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:04,496 [518] [DEBUG] [app] Ending request: urn:request:6ad41e96-fcd3-408c-a46e-4c476a6e2c3d (/_internal_ping) 2020-03-31 08:58:04,497 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:04,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:58:04,497 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:04,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:58:04,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:04,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:04,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:04,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:04,512 [534] [DEBUG] [app] Starting request: urn:request:008bf565-8cf4-4b41-9108-7ed2ba88de65 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:04,512 [534] [DEBUG] [app] Starting request: urn:request:008bf565-8cf4-4b41-9108-7ed2ba88de65 (/c1/_internal_ping) 2020-03-31 08:58:04,513 [534] [DEBUG] [app] Ending request: urn:request:008bf565-8cf4-4b41-9108-7ed2ba88de65 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:04,513 [534] [DEBUG] [app] Ending request: urn:request:008bf565-8cf4-4b41-9108-7ed2ba88de65 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:04,513 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:58:04,513 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:04,524 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:04,524 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:04,526 [518] [DEBUG] [app] Ending request: urn:request:5e228181-0128-4eca-b2e2-543f8bf1dbd1 (/health/instance) 2020-03-31 08:58:04,526 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:04,526 [518] [DEBUG] [app] Ending request: urn:request:5e228181-0128-4eca-b2e2-543f8bf1dbd1 (/health/instance) 2020-03-31 08:58:04,526 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:04,526 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:58:04,526 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) 2020-03-31 08:58:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:16.924497+00:00 (in 9.999177 seconds) 2020-03-31 08:58:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:16 UTC)" (scheduled at 2020-03-31 08:58:06.924497+00:00) 2020-03-31 08:58:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:58:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:58:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:16.924497+00:00 (in 9.999177 seconds) 2020-03-31 08:58:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:16 UTC)" (scheduled at 2020-03-31 08:58:06.924497+00:00) 2020-03-31 08:58:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 6, 926030), True, datetime.datetime(2020, 3, 31, 8, 58, 6, 926030), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:58:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 6, 926030), True, datetime.datetime(2020, 3, 31, 8, 58, 6, 926030), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:58:06,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:58:06,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:06,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:58:06,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:16 UTC)" executed successfully 2020-03-31 08:58:08,273 [518] [DEBUG] [app] Starting request: urn:request:a83b49c9-1e0a-4526-a950-9f0c11dde921 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:08,273 [518] [DEBUG] [app] Starting request: urn:request:a83b49c9-1e0a-4526-a950-9f0c11dde921 (/health/instance) 2020-03-31 08:58:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:08,279 [538] [DEBUG] [app] Starting request: urn:request:d46462f5-2349-49eb-ad26-f67cc6cdb504 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:08,279 [538] [DEBUG] [app] Starting request: urn:request:d46462f5-2349-49eb-ad26-f67cc6cdb504 (/v1/_internal_ping) 2020-03-31 08:58:08,280 [538] [DEBUG] [app] Ending request: urn:request:d46462f5-2349-49eb-ad26-f67cc6cdb504 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:08,280 [538] [DEBUG] [app] Ending request: urn:request:d46462f5-2349-49eb-ad26-f67cc6cdb504 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:58:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:08,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:08,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:08,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:08,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:08,312 [525] [DEBUG] [app] Starting request: urn:request:0804bf3a-71a8-4f1b-82a9-d3f64aa13cc2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:08,312 [525] [DEBUG] [app] Starting request: urn:request:0804bf3a-71a8-4f1b-82a9-d3f64aa13cc2 (/_internal_ping) 2020-03-31 08:58:08,313 [525] [DEBUG] [app] Ending request: urn:request:0804bf3a-71a8-4f1b-82a9-d3f64aa13cc2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:08,313 [525] [DEBUG] [app] Ending request: urn:request:0804bf3a-71a8-4f1b-82a9-d3f64aa13cc2 (/_internal_ping) 2020-03-31 08:58:08,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:08,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:58:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:08,328 [534] [DEBUG] [app] Starting request: urn:request:d69db830-5125-4e16-8d7b-9d9a2934bedd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:08,328 [534] [DEBUG] [app] Starting request: urn:request:d69db830-5125-4e16-8d7b-9d9a2934bedd (/c1/_internal_ping) 2020-03-31 08:58:08,329 [534] [DEBUG] [app] Ending request: urn:request:d69db830-5125-4e16-8d7b-9d9a2934bedd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:08,329 [534] [DEBUG] [app] Ending request: urn:request:d69db830-5125-4e16-8d7b-9d9a2934bedd (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:08,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:58:08,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:08,331 [518] [DEBUG] [app] Ending request: urn:request:a83b49c9-1e0a-4526-a950-9f0c11dde921 (/health/instance) 2020-03-31 08:58:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:08,331 [518] [DEBUG] [app] Ending request: urn:request:a83b49c9-1e0a-4526-a950-9f0c11dde921 (/health/instance) 2020-03-31 08:58:08,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:58:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:58:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:58:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:21.440707+00:00 (in 9.998980 seconds) 2020-03-31 08:58:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:21 UTC)" (scheduled at 2020-03-31 08:58:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:58:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:21.440707+00:00 (in 9.998980 seconds) 2020-03-31 08:58:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:21 UTC)" (scheduled at 2020-03-31 08:58:11.440707+00:00) 2020-03-31 08:58:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:58:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:58:11,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:58:11,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:58:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:58:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:21 UTC)" executed successfully 2020-03-31 08:58:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:58:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:24.323352+00:00 (in 9.998826 seconds) 2020-03-31 08:58:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:24 UTC)" (scheduled at 2020-03-31 08:58:14.323352+00:00) 2020-03-31 08:58:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:58:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:24.323352+00:00 (in 9.998826 seconds) 2020-03-31 08:58:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:24 UTC)" (scheduled at 2020-03-31 08:58:14.323352+00:00) 2020-03-31 08:58:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 14, 325624), True, datetime.datetime(2020, 3, 31, 8, 58, 14, 325624), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:58:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 14, 325624), True, datetime.datetime(2020, 3, 31, 8, 58, 14, 325624), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:58:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:58:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:14,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:14,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:58:14,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:14,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:24 UTC)" executed successfully 2020-03-31 08:58:14,445 [527] [DEBUG] [app] Starting request: urn:request:4dc95448-37ff-4d40-b531-bd1dcd98e8ed (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:14,445 [527] [DEBUG] [app] Starting request: urn:request:4dc95448-37ff-4d40-b531-bd1dcd98e8ed (/health/instance) 2020-03-31 08:58:14,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:14,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:14,453 [544] [DEBUG] [app] Starting request: urn:request:dbf5e090-6f4e-4b3a-9899-146aee668670 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:14,453 [544] [DEBUG] [app] Starting request: urn:request:dbf5e090-6f4e-4b3a-9899-146aee668670 (/v1/_internal_ping) 2020-03-31 08:58:14,454 [544] [DEBUG] [app] Ending request: urn:request:dbf5e090-6f4e-4b3a-9899-146aee668670 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:14,454 [544] [DEBUG] [app] Ending request: urn:request:dbf5e090-6f4e-4b3a-9899-146aee668670 (/v1/_internal_ping) 2020-03-31 08:58:14,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:58:14,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:14,457 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:14,457 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:14,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:14,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:14,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:14,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:14,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:14,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:14,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:14,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:14,487 [526] [DEBUG] [app] Starting request: urn:request:2f58fa97-fd40-4d6d-abfd-3c8257e2afd9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:14,487 [526] [DEBUG] [app] Starting request: urn:request:2f58fa97-fd40-4d6d-abfd-3c8257e2afd9 (/_internal_ping) 2020-03-31 08:58:14,488 [526] [DEBUG] [app] Ending request: urn:request:2f58fa97-fd40-4d6d-abfd-3c8257e2afd9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:14,488 [526] [DEBUG] [app] Ending request: urn:request:2f58fa97-fd40-4d6d-abfd-3c8257e2afd9 (/_internal_ping) 2020-03-31 08:58:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:58:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:58:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:58:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:58:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:58:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:14,502 [534] [DEBUG] [app] Starting request: urn:request:02688f14-5b8c-4e63-a774-795b2683c3eb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:14,502 [534] [DEBUG] [app] Starting request: urn:request:02688f14-5b8c-4e63-a774-795b2683c3eb (/c1/_internal_ping) 2020-03-31 08:58:14,503 [534] [DEBUG] [app] Ending request: urn:request:02688f14-5b8c-4e63-a774-795b2683c3eb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:14,503 [534] [DEBUG] [app] Ending request: urn:request:02688f14-5b8c-4e63-a774-795b2683c3eb (/c1/_internal_ping) 2020-03-31 08:58:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:14,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:14,506 [527] [DEBUG] [app] Ending request: urn:request:4dc95448-37ff-4d40-b531-bd1dcd98e8ed (/health/instance) 2020-03-31 08:58:14,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:58:14,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:14,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 08:58:14,506 [527] [DEBUG] [app] Ending request: urn:request:4dc95448-37ff-4d40-b531-bd1dcd98e8ed (/health/instance) 2020-03-31 08:58:14,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:14,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 08:58:15,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:58:15,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:45.064514+00:00 (in 29.996374 seconds) 2020-03-31 08:58:15,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:45 UTC)" (scheduled at 2020-03-31 08:58:15.064514+00:00) gcworker stdout | 2020-03-31 08:58:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:45.064514+00:00 (in 29.996374 seconds) 2020-03-31 08:58:15,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:45 UTC)" (scheduled at 2020-03-31 08:58:15.064514+00:00) 2020-03-31 08:58:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435495, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:58:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435495, 1209600, 500, 1, 0]) 2020-03-31 08:58:15,086 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:58:15,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:58:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:58:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:45 UTC)" executed successfully 2020-03-31 08:58:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:58:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:26.924497+00:00 (in 9.999171 seconds) 2020-03-31 08:58:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:26 UTC)" (scheduled at 2020-03-31 08:58:16.924497+00:00) 2020-03-31 08:58:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:58:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:26.924497+00:00 (in 9.999171 seconds) 2020-03-31 08:58:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:26 UTC)" (scheduled at 2020-03-31 08:58:16.924497+00:00) 2020-03-31 08:58:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 16, 926039), True, datetime.datetime(2020, 3, 31, 8, 58, 16, 926039), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:58:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 16, 926039), True, datetime.datetime(2020, 3, 31, 8, 58, 16, 926039), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:58:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:58:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:58:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:26 UTC)" executed successfully 2020-03-31 08:58:18,272 [527] [DEBUG] [app] Starting request: urn:request:4ae0249c-03d0-4034-bba4-26985ee411b6 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:18,272 [527] [DEBUG] [app] Starting request: urn:request:4ae0249c-03d0-4034-bba4-26985ee411b6 (/health/instance) 2020-03-31 08:58:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:18,281 [545] [DEBUG] [app] Starting request: urn:request:a7d9a11a-3dca-4180-94aa-92e68cd38252 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:18,281 [545] [DEBUG] [app] Starting request: urn:request:a7d9a11a-3dca-4180-94aa-92e68cd38252 (/v1/_internal_ping) 2020-03-31 08:58:18,284 [545] [DEBUG] [app] Ending request: urn:request:a7d9a11a-3dca-4180-94aa-92e68cd38252 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:18,284 [545] [DEBUG] [app] Ending request: urn:request:a7d9a11a-3dca-4180-94aa-92e68cd38252 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 08:58:18,285 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:18,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:58:18,285 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:18,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:18,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:18,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:18,292 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:18,292 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:18,296 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:18,296 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:18,297 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:18,297 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:18,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:18,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:18,319 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:18,319 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:18,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:18,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:18,324 [527] [DEBUG] [app] Starting request: urn:request:401cc221-1d4f-4fed-ade5-c64bf786be86 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:18,324 [527] [DEBUG] [app] Starting request: urn:request:401cc221-1d4f-4fed-ade5-c64bf786be86 (/_internal_ping) 2020-03-31 08:58:18,325 [527] [DEBUG] [app] Ending request: urn:request:401cc221-1d4f-4fed-ade5-c64bf786be86 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:18,325 [527] [DEBUG] [app] Ending request: urn:request:401cc221-1d4f-4fed-ade5-c64bf786be86 (/_internal_ping) 2020-03-31 08:58:18,326 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:18,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:18,326 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:18,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:58:18,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:18,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:18,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:18,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:18,334 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:18,334 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:18,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:18,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:18,339 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:18,339 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:18,340 [543] [DEBUG] [app] Starting request: urn:request:ec30c485-a3f5-41f9-8c33-0b73b669a509 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:18,340 [543] [DEBUG] [app] Starting request: urn:request:ec30c485-a3f5-41f9-8c33-0b73b669a509 (/c1/_internal_ping) 2020-03-31 08:58:18,341 [543] [DEBUG] [app] Ending request: urn:request:ec30c485-a3f5-41f9-8c33-0b73b669a509 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:18,341 [543] [DEBUG] [app] Ending request: urn:request:ec30c485-a3f5-41f9-8c33-0b73b669a509 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:18,342 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:18,341 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:18,342 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:58:18,341 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:18,343 [527] [DEBUG] [app] Ending request: urn:request:4ae0249c-03d0-4034-bba4-26985ee411b6 (/health/instance) 2020-03-31 08:58:18,343 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:18,343 [527] [DEBUG] [app] Ending request: urn:request:4ae0249c-03d0-4034-bba4-26985ee411b6 (/health/instance) 2020-03-31 08:58:18,343 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:18,343 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 08:58:18,343 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 08:58:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:58:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:31.440707+00:00 (in 9.998858 seconds) 2020-03-31 08:58:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:31 UTC)" (scheduled at 2020-03-31 08:58:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:58:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:31.440707+00:00 (in 9.998858 seconds) 2020-03-31 08:58:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:31 UTC)" (scheduled at 2020-03-31 08:58:21.440707+00:00) 2020-03-31 08:58:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:58:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:58:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:58:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:58:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:58:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:31 UTC)" executed successfully 2020-03-31 08:58:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:58:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:52.113221+00:00 (in 29.999240 seconds) 2020-03-31 08:58:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:52 UTC)" (scheduled at 2020-03-31 08:58:22.113221+00:00) securityworker stdout | 2020-03-31 08:58:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:52.113221+00:00 (in 29.999240 seconds) 2020-03-31 08:58:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:52 UTC)" (scheduled at 2020-03-31 08:58:22.113221+00:00) 2020-03-31 08:58:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:58:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:58:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:58:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:58:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:58:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:58:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:58:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:58:52 UTC)" executed successfully 2020-03-31 08:58:24,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:58:24,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:24,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:34 UTC)" (scheduled at 2020-03-31 08:58:24.323352+00:00) 2020-03-31 08:58:24,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:58:24,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:34 UTC)" (scheduled at 2020-03-31 08:58:24.323352+00:00) 2020-03-31 08:58:24,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:34.323352+00:00 (in 9.996697 seconds) notificationworker stdout | 2020-03-31 08:58:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:34.323352+00:00 (in 9.996697 seconds) 2020-03-31 08:58:24,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 24, 327524), True, datetime.datetime(2020, 3, 31, 8, 58, 24, 327524), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:58:24,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 24, 327524), True, datetime.datetime(2020, 3, 31, 8, 58, 24, 327524), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:58:24,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:58:24,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:24,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:58:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:24,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:34 UTC)" executed successfully 2020-03-31 08:58:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:58:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:33.330112+00:00 (in 8.918257 seconds) 2020-03-31 08:58:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:24 UTC)" (scheduled at 2020-03-31 08:58:24.410927+00:00) 2020-03-31 08:58:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:58:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:58:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:33.330112+00:00 (in 8.918257 seconds) 2020-03-31 08:58:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:24 UTC)" (scheduled at 2020-03-31 08:58:24.410927+00:00) 2020-03-31 08:58:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:58:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:24 UTC)" executed successfully 2020-03-31 08:58:24,446 [527] [DEBUG] [app] Starting request: urn:request:9b7165bc-e391-4995-9468-ba5b16f21ae8 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:24,446 [527] [DEBUG] [app] Starting request: urn:request:9b7165bc-e391-4995-9468-ba5b16f21ae8 (/health/instance) 2020-03-31 08:58:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:24,453 [548] [DEBUG] [app] Starting request: urn:request:8198a47c-797b-45f6-8e31-8551d96e06ae (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:24,453 [548] [DEBUG] [app] Starting request: urn:request:8198a47c-797b-45f6-8e31-8551d96e06ae (/v1/_internal_ping) 2020-03-31 08:58:24,454 [548] [DEBUG] [app] Ending request: urn:request:8198a47c-797b-45f6-8e31-8551d96e06ae (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:24,454 [548] [DEBUG] [app] Ending request: urn:request:8198a47c-797b-45f6-8e31-8551d96e06ae (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:24,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:24,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:58:24,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:24,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:24,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:24,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:24,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:24,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:24,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:24,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:24,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:24,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:24,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:24,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:24,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:24,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:24,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:24,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:24,486 [527] [DEBUG] [app] Starting request: urn:request:9541e6cd-21c7-4aa9-a2a5-6102023921cd (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:24,486 [527] [DEBUG] [app] Starting request: urn:request:9541e6cd-21c7-4aa9-a2a5-6102023921cd (/_internal_ping) 2020-03-31 08:58:24,487 [527] [DEBUG] [app] Ending request: urn:request:9541e6cd-21c7-4aa9-a2a5-6102023921cd (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:58:24,487 [527] [DEBUG] [app] Ending request: urn:request:9541e6cd-21c7-4aa9-a2a5-6102023921cd (/_internal_ping) 2020-03-31 08:58:24,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:24,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:58:24,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:24,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:24,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:58:24,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:58:24,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:24,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:24,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:58:24,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:58:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:24,503 [536] [DEBUG] [app] Starting request: urn:request:63aaa81b-9c74-43c2-80ca-e33c971a2b33 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:24,503 [536] [DEBUG] [app] Starting request: urn:request:63aaa81b-9c74-43c2-80ca-e33c971a2b33 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:24,504 [536] [DEBUG] [app] Ending request: urn:request:63aaa81b-9c74-43c2-80ca-e33c971a2b33 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:24,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:58:24,504 [536] [DEBUG] [app] Ending request: urn:request:63aaa81b-9c74-43c2-80ca-e33c971a2b33 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:24,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:24,507 [527] [DEBUG] [app] Ending request: urn:request:9b7165bc-e391-4995-9468-ba5b16f21ae8 (/health/instance) 2020-03-31 08:58:24,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:24,507 [527] [DEBUG] [app] Ending request: urn:request:9b7165bc-e391-4995-9468-ba5b16f21ae8 (/health/instance) 2020-03-31 08:58:24,507 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:58:24,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:58:24,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:58:24,505 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:58:24,505 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:36.924497+00:00 (in 9.997777 seconds) 2020-03-31 08:58:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:36 UTC)" (scheduled at 2020-03-31 08:58:26.924497+00:00) 2020-03-31 08:58:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 26, 927596), True, datetime.datetime(2020, 3, 31, 8, 58, 26, 927596), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:58:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:36.924497+00:00 (in 9.997777 seconds) 2020-03-31 08:58:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:36 UTC)" (scheduled at 2020-03-31 08:58:26.924497+00:00) 2020-03-31 08:58:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 26, 927596), True, datetime.datetime(2020, 3, 31, 8, 58, 26, 927596), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:58:26,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:58:26,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:58:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:36 UTC)" executed successfully 2020-03-31 08:58:28,273 [518] [DEBUG] [app] Starting request: urn:request:1c1f150c-4a3f-49b0-802d-bbf77355d837 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:28,273 [518] [DEBUG] [app] Starting request: urn:request:1c1f150c-4a3f-49b0-802d-bbf77355d837 (/health/instance) 2020-03-31 08:58:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:28,282 [538] [DEBUG] [app] Starting request: urn:request:8531ec77-2c6c-49b1-83f1-d59830d4e441 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:28,282 [538] [DEBUG] [app] Starting request: urn:request:8531ec77-2c6c-49b1-83f1-d59830d4e441 (/v1/_internal_ping) 2020-03-31 08:58:28,283 [538] [DEBUG] [app] Ending request: urn:request:8531ec77-2c6c-49b1-83f1-d59830d4e441 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:58:28,283 [538] [DEBUG] [app] Ending request: urn:request:8531ec77-2c6c-49b1-83f1-d59830d4e441 (/v1/_internal_ping) 2020-03-31 08:58:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:28,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:28,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:58:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:28,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:58:28,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:58:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:28,291 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:28,291 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:28,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:28,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:28,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:28,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:28,323 [527] [DEBUG] [app] Starting request: urn:request:42f62a28-2fdd-438f-be1c-8518147fc3f4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:28,323 [527] [DEBUG] [app] Starting request: urn:request:42f62a28-2fdd-438f-be1c-8518147fc3f4 (/_internal_ping) 2020-03-31 08:58:28,324 [527] [DEBUG] [app] Ending request: urn:request:42f62a28-2fdd-438f-be1c-8518147fc3f4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:28,324 [527] [DEBUG] [app] Ending request: urn:request:42f62a28-2fdd-438f-be1c-8518147fc3f4 (/_internal_ping) 2020-03-31 08:58:28,325 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 08:58:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:28,325 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 08:58:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:58:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:58:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:28,337 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:58:28,337 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:28,342 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:28,344 [534] [DEBUG] [app] Starting request: urn:request:dbb1a1e3-e617-41c0-a686-4ccd230eaa74 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:28,342 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 08:58:28,344 [534] [DEBUG] [app] Starting request: urn:request:dbb1a1e3-e617-41c0-a686-4ccd230eaa74 (/c1/_internal_ping) 2020-03-31 08:58:28,347 [534] [DEBUG] [app] Ending request: urn:request:dbb1a1e3-e617-41c0-a686-4ccd230eaa74 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:28,347 [534] [DEBUG] [app] Ending request: urn:request:dbb1a1e3-e617-41c0-a686-4ccd230eaa74 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) 2020-03-31 08:58:28,350 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) 2020-03-31 08:58:28,350 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:58:28,350 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:28,350 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:28,352 [518] [DEBUG] [app] Ending request: urn:request:1c1f150c-4a3f-49b0-802d-bbf77355d837 (/health/instance) 2020-03-31 08:58:28,352 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:28,352 [518] [DEBUG] [app] Ending request: urn:request:1c1f150c-4a3f-49b0-802d-bbf77355d837 (/health/instance) 2020-03-31 08:58:28,352 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:28,353 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:58:28,353 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.082 49 0.082) 2020-03-31 08:58:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:58:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:00.337699+00:00 (in 29.999259 seconds) 2020-03-31 08:58:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:00 UTC)" (scheduled at 2020-03-31 08:58:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:58:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:00.337699+00:00 (in 29.999259 seconds) 2020-03-31 08:58:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:00 UTC)" (scheduled at 2020-03-31 08:58:30.337699+00:00) 2020-03-31 08:58:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 58, 30, 339382), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:58:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 58, 30, 339382), False, 50, 1, 0]) 2020-03-31 08:58:30,387 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:58:30,387 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:58:30,387 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:30,388 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:58:30,387 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:30,388 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:00 UTC)" executed successfully 2020-03-31 08:58:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:58:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:41 UTC)" (scheduled at 2020-03-31 08:58:31.440707+00:00) 2020-03-31 08:58:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:41.440707+00:00 (in 9.998998 seconds) repositoryactioncounter stdout | 2020-03-31 08:58:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:41 UTC)" (scheduled at 2020-03-31 08:58:31.440707+00:00) 2020-03-31 08:58:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:41.440707+00:00 (in 9.998998 seconds) 2020-03-31 08:58:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:58:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:58:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:58:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:58:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:58:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:41 UTC)" executed successfully 2020-03-31 08:58:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:58:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:03 UTC)" (scheduled at 2020-03-31 08:58:33.330112+00:00) 2020-03-31 08:58:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:58:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:03 UTC)" (scheduled at 2020-03-31 08:58:33.330112+00:00) 2020-03-31 08:58:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:03.330112+00:00 (in 29.999240 seconds) security_notification_worker stdout | 2020-03-31 08:58:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:03.330112+00:00 (in 29.999240 seconds) 2020-03-31 08:58:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 33, 331708), True, datetime.datetime(2020, 3, 31, 8, 58, 33, 331708), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:58:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 33, 331708), True, datetime.datetime(2020, 3, 31, 8, 58, 33, 331708), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:58:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:58:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:58:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:03 UTC)" executed successfully 2020-03-31 08:58:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:58:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:43.539999+00:00 (in 10.171955 seconds) exportactionlogsworker stdout | 2020-03-31 08:58:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:43.539999+00:00 (in 10.171955 seconds) 2020-03-31 08:58:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:33 UTC)" (scheduled at 2020-03-31 08:58:33.366791+00:00) 2020-03-31 08:58:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:58:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:33 UTC)" (scheduled at 2020-03-31 08:58:33.366791+00:00) 2020-03-31 08:58:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 33, 368972), True, datetime.datetime(2020, 3, 31, 8, 58, 33, 368972), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:58:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 33, 368972), True, datetime.datetime(2020, 3, 31, 8, 58, 33, 368972), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:58:33,382 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:58:33,382 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:58:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:33 UTC)" executed successfully 2020-03-31 08:58:34,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:58:34,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:34,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:44.323352+00:00 (in 9.997731 seconds) 2020-03-31 08:58:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:44 UTC)" (scheduled at 2020-03-31 08:58:34.323352+00:00) 2020-03-31 08:58:34,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:58:34,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:44.323352+00:00 (in 9.997731 seconds) 2020-03-31 08:58:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:44 UTC)" (scheduled at 2020-03-31 08:58:34.323352+00:00) 2020-03-31 08:58:34,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 34, 326519), True, datetime.datetime(2020, 3, 31, 8, 58, 34, 326519), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:58:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 34, 326519), True, datetime.datetime(2020, 3, 31, 8, 58, 34, 326519), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:58:34,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:58:34,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:34,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:58:34,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:44 UTC)" executed successfully 2020-03-31 08:58:34,446 [526] [DEBUG] [app] Starting request: urn:request:9c6a9f7a-7e56-4a28-b13b-fde6f41350f8 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:34,446 [526] [DEBUG] [app] Starting request: urn:request:9c6a9f7a-7e56-4a28-b13b-fde6f41350f8 (/health/instance) 2020-03-31 08:58:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:34,454 [547] [DEBUG] [app] Starting request: urn:request:86580307-96b3-4167-a427-0cdbbbf54296 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:34,454 [547] [DEBUG] [app] Starting request: urn:request:86580307-96b3-4167-a427-0cdbbbf54296 (/v1/_internal_ping) 2020-03-31 08:58:34,455 [547] [DEBUG] [app] Ending request: urn:request:86580307-96b3-4167-a427-0cdbbbf54296 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:34,455 [547] [DEBUG] [app] Ending request: urn:request:86580307-96b3-4167-a427-0cdbbbf54296 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:34,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:58:34,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:34,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:34,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:34,460 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:34,460 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:34,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:34,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:34,473 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:34,473 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:34,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:34,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:34,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:34,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:34,492 [521] [DEBUG] [app] Starting request: urn:request:569833b2-3bb8-4e39-8a05-cd5987b2b0d1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:34,492 [521] [DEBUG] [app] Starting request: urn:request:569833b2-3bb8-4e39-8a05-cd5987b2b0d1 (/_internal_ping) 2020-03-31 08:58:34,493 [521] [DEBUG] [app] Ending request: urn:request:569833b2-3bb8-4e39-8a05-cd5987b2b0d1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:34,493 [521] [DEBUG] [app] Ending request: urn:request:569833b2-3bb8-4e39-8a05-cd5987b2b0d1 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:58:34,494 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:34,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:34,494 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:34,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:34,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:34,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:34,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:34,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:34,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:34,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:34,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:34,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:34,508 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:34,508 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:34,509 [536] [DEBUG] [app] Starting request: urn:request:f58fb008-8e4f-4088-b5fb-6751e65df31c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:34,509 [536] [DEBUG] [app] Starting request: urn:request:f58fb008-8e4f-4088-b5fb-6751e65df31c (/c1/_internal_ping) 2020-03-31 08:58:34,510 [536] [DEBUG] [app] Ending request: urn:request:f58fb008-8e4f-4088-b5fb-6751e65df31c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:34,510 [536] [DEBUG] [app] Ending request: urn:request:f58fb008-8e4f-4088-b5fb-6751e65df31c (/c1/_internal_ping) 2020-03-31 08:58:34,511 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:34,511 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:34,511 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:34,513 [526] [DEBUG] [app] Ending request: urn:request:9c6a9f7a-7e56-4a28-b13b-fde6f41350f8 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:58:34,511 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:34,513 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:34,513 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 08:58:34,513 [526] [DEBUG] [app] Ending request: urn:request:9c6a9f7a-7e56-4a28-b13b-fde6f41350f8 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:34,513 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:34,513 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 08:58:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:58:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:38.010934+00:00 (in 1.085685 seconds) 2020-03-31 08:58:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:46 UTC)" (scheduled at 2020-03-31 08:58:36.924497+00:00) 2020-03-31 08:58:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:58:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:38.010934+00:00 (in 1.085685 seconds) 2020-03-31 08:58:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:46 UTC)" (scheduled at 2020-03-31 08:58:36.924497+00:00) 2020-03-31 08:58:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 36, 925929), True, datetime.datetime(2020, 3, 31, 8, 58, 36, 925929), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:58:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 36, 925929), True, datetime.datetime(2020, 3, 31, 8, 58, 36, 925929), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:58:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:36,944 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 08:58:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:58:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:46 UTC)" executed successfully 2020-03-31 08:58:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:58:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:46.924497+00:00 (in 8.912876 seconds) 2020-03-31 08:58:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:38 UTC)" (scheduled at 2020-03-31 08:58:38.010934+00:00) 2020-03-31 08:58:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:58:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:58:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:46.924497+00:00 (in 8.912876 seconds) 2020-03-31 08:58:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:38 UTC)" (scheduled at 2020-03-31 08:58:38.010934+00:00) 2020-03-31 08:58:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:58:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:38 UTC)" executed successfully 2020-03-31 08:58:38,273 [527] [DEBUG] [app] Starting request: urn:request:18de2c6c-27d1-4d56-883a-d3643647f226 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:38,273 [527] [DEBUG] [app] Starting request: urn:request:18de2c6c-27d1-4d56-883a-d3643647f226 (/health/instance) 2020-03-31 08:58:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:38,281 [555] [DEBUG] [app] Starting request: urn:request:d087f3d4-5458-4070-a3bc-72a722892098 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:38,281 [555] [DEBUG] [app] Starting request: urn:request:d087f3d4-5458-4070-a3bc-72a722892098 (/v1/_internal_ping) 2020-03-31 08:58:38,282 [555] [DEBUG] [app] Ending request: urn:request:d087f3d4-5458-4070-a3bc-72a722892098 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:38,282 [555] [DEBUG] [app] Ending request: urn:request:d087f3d4-5458-4070-a3bc-72a722892098 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:38,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:58:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:38,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:38,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:38,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:38,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:38,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:38,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:38,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:38,313 [527] [DEBUG] [app] Starting request: urn:request:51f9640a-4db1-436f-9932-6e0fc0d48d33 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:38,313 [527] [DEBUG] [app] Starting request: urn:request:51f9640a-4db1-436f-9932-6e0fc0d48d33 (/_internal_ping) 2020-03-31 08:58:38,314 [527] [DEBUG] [app] Ending request: urn:request:51f9640a-4db1-436f-9932-6e0fc0d48d33 (/_internal_ping) 2020-03-31 08:58:38,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:38,314 [527] [DEBUG] [app] Ending request: urn:request:51f9640a-4db1-436f-9932-6e0fc0d48d33 (/_internal_ping) 2020-03-31 08:58:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:38,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:58:38,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:38,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:38,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:38,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:38,327 [541] [DEBUG] [app] Starting request: urn:request:e633d41d-8340-41f5-9296-9f275dc0891c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:38,327 [541] [DEBUG] [app] Starting request: urn:request:e633d41d-8340-41f5-9296-9f275dc0891c (/c1/_internal_ping) 2020-03-31 08:58:38,328 [541] [DEBUG] [app] Ending request: urn:request:e633d41d-8340-41f5-9296-9f275dc0891c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:38,328 [541] [DEBUG] [app] Ending request: urn:request:e633d41d-8340-41f5-9296-9f275dc0891c (/c1/_internal_ping) 2020-03-31 08:58:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:38,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:38,330 [527] [DEBUG] [app] Ending request: urn:request:18de2c6c-27d1-4d56-883a-d3643647f226 (/health/instance) 2020-03-31 08:58:38,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:58:38,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 08:58:38,330 [527] [DEBUG] [app] Ending request: urn:request:18de2c6c-27d1-4d56-883a-d3643647f226 (/health/instance) 2020-03-31 08:58:38,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 08:58:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:58:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:51.440707+00:00 (in 9.998280 seconds) 2020-03-31 08:58:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:51 UTC)" (scheduled at 2020-03-31 08:58:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:58:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:51.440707+00:00 (in 9.998280 seconds) 2020-03-31 08:58:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:51 UTC)" (scheduled at 2020-03-31 08:58:41.440707+00:00) 2020-03-31 08:58:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:58:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:58:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:58:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:58:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:58:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:51 UTC)" executed successfully 2020-03-31 08:58:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:58:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:33.366791+00:00 (in 49.825563 seconds) 2020-03-31 08:58:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:43 UTC)" (scheduled at 2020-03-31 08:58:43.539999+00:00) 2020-03-31 08:58:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 08:58:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:33.366791+00:00 (in 49.825563 seconds) 2020-03-31 08:58:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:43 UTC)" (scheduled at 2020-03-31 08:58:43.539999+00:00) 2020-03-31 08:58:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:58:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:58:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:43 UTC)" executed successfully 2020-03-31 08:58:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:58:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:49.498966+00:00 (in 5.174502 seconds) 2020-03-31 08:58:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:54 UTC)" (scheduled at 2020-03-31 08:58:44.323352+00:00) 2020-03-31 08:58:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:58:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:49.498966+00:00 (in 5.174502 seconds) 2020-03-31 08:58:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:54 UTC)" (scheduled at 2020-03-31 08:58:44.323352+00:00) 2020-03-31 08:58:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 44, 325548), True, datetime.datetime(2020, 3, 31, 8, 58, 44, 325548), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:58:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 44, 325548), True, datetime.datetime(2020, 3, 31, 8, 58, 44, 325548), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:58:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:58:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:58:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:54 UTC)" executed successfully 2020-03-31 08:58:44,445 [518] [DEBUG] [app] Starting request: urn:request:fcb78f2c-949d-4081-a26b-a182f20470a9 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:44,445 [518] [DEBUG] [app] Starting request: urn:request:fcb78f2c-949d-4081-a26b-a182f20470a9 (/health/instance) 2020-03-31 08:58:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:44,454 [556] [DEBUG] [app] Starting request: urn:request:4ba65f6d-44f8-429e-8ec6-063dc34e1320 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:44,454 [556] [DEBUG] [app] Starting request: urn:request:4ba65f6d-44f8-429e-8ec6-063dc34e1320 (/v1/_internal_ping) 2020-03-31 08:58:44,455 [556] [DEBUG] [app] Ending request: urn:request:4ba65f6d-44f8-429e-8ec6-063dc34e1320 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:44,455 [556] [DEBUG] [app] Ending request: urn:request:4ba65f6d-44f8-429e-8ec6-063dc34e1320 (/v1/_internal_ping) 2020-03-31 08:58:44,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:58:44,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:44,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:58:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:58:44,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:44,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:44,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:44,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:44,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:44,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:44,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:44,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:44,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:44,491 [518] [DEBUG] [app] Starting request: urn:request:477eb7f1-3f13-40ac-b47d-0e72c8790b7e (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:44,491 [518] [DEBUG] [app] Starting request: urn:request:477eb7f1-3f13-40ac-b47d-0e72c8790b7e (/_internal_ping) 2020-03-31 08:58:44,492 [518] [DEBUG] [app] Ending request: urn:request:477eb7f1-3f13-40ac-b47d-0e72c8790b7e (/_internal_ping) 2020-03-31 08:58:44,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:44,492 [518] [DEBUG] [app] Ending request: urn:request:477eb7f1-3f13-40ac-b47d-0e72c8790b7e (/_internal_ping) 2020-03-31 08:58:44,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:44,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:44,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:44,505 [534] [DEBUG] [app] Starting request: urn:request:5d1b9736-d913-4e1d-9991-18fc805d6724 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:44,505 [534] [DEBUG] [app] Starting request: urn:request:5d1b9736-d913-4e1d-9991-18fc805d6724 (/c1/_internal_ping) 2020-03-31 08:58:44,505 [534] [DEBUG] [app] Ending request: urn:request:5d1b9736-d913-4e1d-9991-18fc805d6724 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:44,505 [534] [DEBUG] [app] Ending request: urn:request:5d1b9736-d913-4e1d-9991-18fc805d6724 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:44,506 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:44,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:58:44,506 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:44,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:44,508 [518] [DEBUG] [app] Ending request: urn:request:fcb78f2c-949d-4081-a26b-a182f20470a9 (/health/instance) 2020-03-31 08:58:44,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:44,508 [518] [DEBUG] [app] Ending request: urn:request:fcb78f2c-949d-4081-a26b-a182f20470a9 (/health/instance) 2020-03-31 08:58:44,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:44,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:58:44,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:58:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:58:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:15.064514+00:00 (in 29.998701 seconds) 2020-03-31 08:58:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:15 UTC)" (scheduled at 2020-03-31 08:58:45.064514+00:00) gcworker stdout | 2020-03-31 08:58:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:15.064514+00:00 (in 29.998701 seconds) 2020-03-31 08:58:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:15 UTC)" (scheduled at 2020-03-31 08:58:45.064514+00:00) 2020-03-31 08:58:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435525, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:58:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435525, 1209600, 500, 1, 0]) 2020-03-31 08:58:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:58:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:58:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:58:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:15 UTC)" executed successfully 2020-03-31 08:58:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:58:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:56.924497+00:00 (in 9.998980 seconds) 2020-03-31 08:58:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:56 UTC)" (scheduled at 2020-03-31 08:58:46.924497+00:00) 2020-03-31 08:58:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:58:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:56.924497+00:00 (in 9.998980 seconds) 2020-03-31 08:58:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:56 UTC)" (scheduled at 2020-03-31 08:58:46.924497+00:00) 2020-03-31 08:58:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 46, 926474), True, datetime.datetime(2020, 3, 31, 8, 58, 46, 926474), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:58:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 46, 926474), True, datetime.datetime(2020, 3, 31, 8, 58, 46, 926474), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:58:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:58:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:58:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:58:56 UTC)" executed successfully 2020-03-31 08:58:48,273 [518] [DEBUG] [app] Starting request: urn:request:3d53ef24-10b4-430f-82e8-b15e93b892c1 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:48,273 [518] [DEBUG] [app] Starting request: urn:request:3d53ef24-10b4-430f-82e8-b15e93b892c1 (/health/instance) 2020-03-31 08:58:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:48,279 [545] [DEBUG] [app] Starting request: urn:request:c3111a5d-d02e-408c-9d6b-b63beceb16f4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:48,279 [545] [DEBUG] [app] Starting request: urn:request:c3111a5d-d02e-408c-9d6b-b63beceb16f4 (/v1/_internal_ping) 2020-03-31 08:58:48,280 [545] [DEBUG] [app] Ending request: urn:request:c3111a5d-d02e-408c-9d6b-b63beceb16f4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:48,280 [545] [DEBUG] [app] Ending request: urn:request:c3111a5d-d02e-408c-9d6b-b63beceb16f4 (/v1/_internal_ping) 2020-03-31 08:58:48,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 08:58:48,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 08:58:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:48,284 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:48,284 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:48,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:48,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:48,286 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:48,286 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:48,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:48,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:48,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:48,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:48,315 [518] [DEBUG] [app] Starting request: urn:request:0e44dda9-bb58-4c59-b569-c107a70ec4a8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:48,315 [518] [DEBUG] [app] Starting request: urn:request:0e44dda9-bb58-4c59-b569-c107a70ec4a8 (/_internal_ping) 2020-03-31 08:58:48,316 [518] [DEBUG] [app] Ending request: urn:request:0e44dda9-bb58-4c59-b569-c107a70ec4a8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:48,316 [518] [DEBUG] [app] Ending request: urn:request:0e44dda9-bb58-4c59-b569-c107a70ec4a8 (/_internal_ping) 2020-03-31 08:58:48,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:58:48,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:58:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:58:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:48,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:48,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:48,331 [534] [DEBUG] [app] Starting request: urn:request:64b64f4a-2ef2-438a-9bee-0c8957a2e9b9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:48,331 [534] [DEBUG] [app] Starting request: urn:request:64b64f4a-2ef2-438a-9bee-0c8957a2e9b9 (/c1/_internal_ping) 2020-03-31 08:58:48,332 [534] [DEBUG] [app] Ending request: urn:request:64b64f4a-2ef2-438a-9bee-0c8957a2e9b9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:48,332 [534] [DEBUG] [app] Ending request: urn:request:64b64f4a-2ef2-438a-9bee-0c8957a2e9b9 (/c1/_internal_ping) 2020-03-31 08:58:48,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:48,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:48,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:48,334 [518] [DEBUG] [app] Ending request: urn:request:3d53ef24-10b4-430f-82e8-b15e93b892c1 (/health/instance) 2020-03-31 08:58:48,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:58:48,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:48,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:58:48,334 [518] [DEBUG] [app] Ending request: urn:request:3d53ef24-10b4-430f-82e8-b15e93b892c1 (/health/instance) 2020-03-31 08:58:48,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:48,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:58:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:58:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:54.323352+00:00 (in 4.823293 seconds) 2020-03-31 08:58:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:49 UTC)" (scheduled at 2020-03-31 08:58:49.498966+00:00) 2020-03-31 08:58:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 08:58:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:54.323352+00:00 (in 4.823293 seconds) 2020-03-31 08:58:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:49 UTC)" (scheduled at 2020-03-31 08:58:49.498966+00:00) 2020-03-31 08:58:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:58:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:58:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:49 UTC)" executed successfully 2020-03-31 08:58:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:58:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:59.560326+00:00 (in 8.448643 seconds) 2020-03-31 08:58:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:51 UTC)" (scheduled at 2020-03-31 08:58:51.110610+00:00) 2020-03-31 08:58:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:58:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:58:59.560326+00:00 (in 8.448643 seconds) 2020-03-31 08:58:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:51 UTC)" (scheduled at 2020-03-31 08:58:51.110610+00:00) 2020-03-31 08:58:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 51, 112552), True, datetime.datetime(2020, 3, 31, 8, 58, 51, 112552), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:58:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 51, 112552), True, datetime.datetime(2020, 3, 31, 8, 58, 51, 112552), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:58:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:58:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:58:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:51 UTC)" executed successfully 2020-03-31 08:58:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:58:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:01.440707+00:00 (in 9.998567 seconds) 2020-03-31 08:58:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:01 UTC)" (scheduled at 2020-03-31 08:58:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:58:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:01.440707+00:00 (in 9.998567 seconds) 2020-03-31 08:58:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:01 UTC)" (scheduled at 2020-03-31 08:58:51.440707+00:00) 2020-03-31 08:58:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:58:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:58:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:58:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:58:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:58:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:01 UTC)" executed successfully 2020-03-31 08:58:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:58:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:22.113221+00:00 (in 29.997769 seconds) 2020-03-31 08:58:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:22 UTC)" (scheduled at 2020-03-31 08:58:52.113221+00:00) securityworker stdout | 2020-03-31 08:58:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:22.113221+00:00 (in 29.997769 seconds) 2020-03-31 08:58:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:22 UTC)" (scheduled at 2020-03-31 08:58:52.113221+00:00) 2020-03-31 08:58:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:58:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:58:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:58:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:58:52,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:58:52,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:58:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:58:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:22 UTC)" executed successfully 2020-03-31 08:58:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:58:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:04.323352+00:00 (in 9.999048 seconds) 2020-03-31 08:58:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:04 UTC)" (scheduled at 2020-03-31 08:58:54.323352+00:00) 2020-03-31 08:58:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:58:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:04.323352+00:00 (in 9.999048 seconds) 2020-03-31 08:58:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:04 UTC)" (scheduled at 2020-03-31 08:58:54.323352+00:00) 2020-03-31 08:58:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 54, 325272), True, datetime.datetime(2020, 3, 31, 8, 58, 54, 325272), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:58:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 54, 325272), True, datetime.datetime(2020, 3, 31, 8, 58, 54, 325272), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:58:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:58:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:58:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:04 UTC)" executed successfully 2020-03-31 08:58:54,446 [526] [DEBUG] [app] Starting request: urn:request:f74cb13e-1964-4ae7-8b43-f0c472258eaf (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:54,446 [526] [DEBUG] [app] Starting request: urn:request:f74cb13e-1964-4ae7-8b43-f0c472258eaf (/health/instance) 2020-03-31 08:58:54,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:54,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:54,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:54,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:54,454 [552] [DEBUG] [app] Starting request: urn:request:6730dfa5-6df0-4b34-a456-414956285bf4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:54,454 [552] [DEBUG] [app] Starting request: urn:request:6730dfa5-6df0-4b34-a456-414956285bf4 (/v1/_internal_ping) 2020-03-31 08:58:54,455 [552] [DEBUG] [app] Ending request: urn:request:6730dfa5-6df0-4b34-a456-414956285bf4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:54,455 [552] [DEBUG] [app] Ending request: urn:request:6730dfa5-6df0-4b34-a456-414956285bf4 (/v1/_internal_ping) 2020-03-31 08:58:54,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:58:54,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:54,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:54,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:54,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:58:54,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:54,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:54,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:54,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:54,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:54,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:54,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:54,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:54,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:54,487 [518] [DEBUG] [app] Starting request: urn:request:87612bfc-a01c-456d-bc1f-b0cdfa60f667 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:54,487 [518] [DEBUG] [app] Starting request: urn:request:87612bfc-a01c-456d-bc1f-b0cdfa60f667 (/_internal_ping) 2020-03-31 08:58:54,488 [518] [DEBUG] [app] Ending request: urn:request:87612bfc-a01c-456d-bc1f-b0cdfa60f667 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:54,488 [518] [DEBUG] [app] Ending request: urn:request:87612bfc-a01c-456d-bc1f-b0cdfa60f667 (/_internal_ping) 2020-03-31 08:58:54,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:54,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:54,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:54,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:54,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:54,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:54,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:58:54,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:54,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:54,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:54,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:58:54,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:54,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:54,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:54,503 [536] [DEBUG] [app] Starting request: urn:request:d757a74b-30ad-41c2-8f2d-350f19d7f7ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:54,503 [536] [DEBUG] [app] Starting request: urn:request:d757a74b-30ad-41c2-8f2d-350f19d7f7ec (/c1/_internal_ping) 2020-03-31 08:58:54,504 [536] [DEBUG] [app] Ending request: urn:request:d757a74b-30ad-41c2-8f2d-350f19d7f7ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:54,504 [536] [DEBUG] [app] Ending request: urn:request:d757a74b-30ad-41c2-8f2d-350f19d7f7ec (/c1/_internal_ping) 2020-03-31 08:58:54,505 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:54,504 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:54,505 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:58:54,504 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:54,507 [526] [DEBUG] [app] Ending request: urn:request:f74cb13e-1964-4ae7-8b43-f0c472258eaf (/health/instance) 2020-03-31 08:58:54,507 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:54,507 [526] [DEBUG] [app] Ending request: urn:request:f74cb13e-1964-4ae7-8b43-f0c472258eaf (/health/instance) 2020-03-31 08:58:54,507 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:54,508 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 08:58:54,508 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 08:58:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:58:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:06.924497+00:00 (in 9.999143 seconds) 2020-03-31 08:58:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:06 UTC)" (scheduled at 2020-03-31 08:58:56.924497+00:00) 2020-03-31 08:58:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:58:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:06.924497+00:00 (in 9.999143 seconds) 2020-03-31 08:58:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:06 UTC)" (scheduled at 2020-03-31 08:58:56.924497+00:00) 2020-03-31 08:58:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:58:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 56, 925990), True, datetime.datetime(2020, 3, 31, 8, 58, 56, 925990), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:58:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 58, 56, 925990), True, datetime.datetime(2020, 3, 31, 8, 58, 56, 925990), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:58:56,941 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:58:56,941 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:58:56,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:56,941 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:58:56,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:58:56,941 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:06 UTC)" executed successfully 2020-03-31 08:58:58,273 [518] [DEBUG] [app] Starting request: urn:request:0bc2b52a-928e-45b5-bd37-210ebeedec06 (/health/instance) gunicorn-web stdout | 2020-03-31 08:58:58,273 [518] [DEBUG] [app] Starting request: urn:request:0bc2b52a-928e-45b5-bd37-210ebeedec06 (/health/instance) 2020-03-31 08:58:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:58,281 [555] [DEBUG] [app] Starting request: urn:request:233d39df-af3d-46b9-a02b-d01f9f7d7c46 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:58,281 [555] [DEBUG] [app] Starting request: urn:request:233d39df-af3d-46b9-a02b-d01f9f7d7c46 (/v1/_internal_ping) 2020-03-31 08:58:58,282 [555] [DEBUG] [app] Ending request: urn:request:233d39df-af3d-46b9-a02b-d01f9f7d7c46 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:58:58,282 [555] [DEBUG] [app] Ending request: urn:request:233d39df-af3d-46b9-a02b-d01f9f7d7c46 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:58:58,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:58:58,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:58:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:58:58,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:58:58,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:58:58,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:58:58,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:58:58,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:58:58,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:58:58,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:58:58,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:58:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:58:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:58:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:58,313 [518] [DEBUG] [app] Starting request: urn:request:0dd21323-fa32-48f0-bbe8-2160c020c05f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:58,313 [518] [DEBUG] [app] Starting request: urn:request:0dd21323-fa32-48f0-bbe8-2160c020c05f (/_internal_ping) 2020-03-31 08:58:58,314 [518] [DEBUG] [app] Ending request: urn:request:0dd21323-fa32-48f0-bbe8-2160c020c05f (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:58:58,314 [518] [DEBUG] [app] Ending request: urn:request:0dd21323-fa32-48f0-bbe8-2160c020c05f (/_internal_ping) 2020-03-31 08:58:58,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:58,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:58:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:58:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:58:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:58:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:58:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:58:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:58:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:58:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:58:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:58:58,328 [541] [DEBUG] [app] Starting request: urn:request:a98958c6-7112-428e-a517-666ff05a30ab (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:58,328 [541] [DEBUG] [app] Starting request: urn:request:a98958c6-7112-428e-a517-666ff05a30ab (/c1/_internal_ping) 2020-03-31 08:58:58,329 [541] [DEBUG] [app] Ending request: urn:request:a98958c6-7112-428e-a517-666ff05a30ab (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:58:58,329 [541] [DEBUG] [app] Ending request: urn:request:a98958c6-7112-428e-a517-666ff05a30ab (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:58,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:58:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:58:58,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:58:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:58:58,331 [518] [DEBUG] [app] Ending request: urn:request:0bc2b52a-928e-45b5-bd37-210ebeedec06 (/health/instance) 2020-03-31 08:58:58,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:58:58,331 [518] [DEBUG] [app] Ending request: urn:request:0bc2b52a-928e-45b5-bd37-210ebeedec06 (/health/instance) 2020-03-31 08:58:58,331 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:58:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:58:58,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:58:58,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:58:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:58:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:58:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:58:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:51.110610+00:00 (in 51.549450 seconds) 2020-03-31 08:58:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:59 UTC)" (scheduled at 2020-03-31 08:58:59.560326+00:00) 2020-03-31 08:58:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 08:58:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:51.110610+00:00 (in 51.549450 seconds) 2020-03-31 08:58:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:59 UTC)" (scheduled at 2020-03-31 08:58:59.560326+00:00) 2020-03-31 08:58:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:58:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:58:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 08:59:59 UTC)" executed successfully 2020-03-31 08:59:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:59:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:30.337699+00:00 (in 29.998924 seconds) 2020-03-31 08:59:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:30 UTC)" (scheduled at 2020-03-31 08:59:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:59:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:30.337699+00:00 (in 29.998924 seconds) 2020-03-31 08:59:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:30 UTC)" (scheduled at 2020-03-31 08:59:00.337699+00:00) 2020-03-31 08:59:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 59, 0, 339412), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:59:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 59, 0, 339412), False, 50, 1, 0]) 2020-03-31 08:59:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:59:00,357 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 08:59:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:59:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:59:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:30 UTC)" executed successfully 2020-03-31 08:59:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:59:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:11.440707+00:00 (in 9.998938 seconds) 2020-03-31 08:59:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:11 UTC)" (scheduled at 2020-03-31 08:59:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:59:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:11.440707+00:00 (in 9.998938 seconds) 2020-03-31 08:59:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:11 UTC)" (scheduled at 2020-03-31 08:59:01.440707+00:00) 2020-03-31 08:59:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:59:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:59:01,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:59:01,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:59:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:59:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:11 UTC)" executed successfully 2020-03-31 08:59:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:59:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:23.410371+00:00 (in 20.078338 seconds) 2020-03-31 08:59:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:33 UTC)" (scheduled at 2020-03-31 08:59:03.330112+00:00) 2020-03-31 08:59:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:59:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:23.410371+00:00 (in 20.078338 seconds) 2020-03-31 08:59:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:33 UTC)" (scheduled at 2020-03-31 08:59:03.330112+00:00) 2020-03-31 08:59:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 3, 333050), True, datetime.datetime(2020, 3, 31, 8, 59, 3, 333050), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:59:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 3, 333050), True, datetime.datetime(2020, 3, 31, 8, 59, 3, 333050), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:59:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:03,350 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 08:59:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:59:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:33 UTC)" executed successfully 2020-03-31 08:59:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:59:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:14.323352+00:00 (in 9.998988 seconds) 2020-03-31 08:59:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:14 UTC)" (scheduled at 2020-03-31 08:59:04.323352+00:00) 2020-03-31 08:59:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:59:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:14.323352+00:00 (in 9.998988 seconds) 2020-03-31 08:59:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:14 UTC)" (scheduled at 2020-03-31 08:59:04.323352+00:00) 2020-03-31 08:59:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 4, 325332), True, datetime.datetime(2020, 3, 31, 8, 59, 4, 325332), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:59:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 4, 325332), True, datetime.datetime(2020, 3, 31, 8, 59, 4, 325332), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:59:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:59:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:59:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:14 UTC)" executed successfully 2020-03-31 08:59:04,446 [518] [DEBUG] [app] Starting request: urn:request:d79d5af4-01e1-445d-8c3f-222aa8f73b06 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:04,446 [518] [DEBUG] [app] Starting request: urn:request:d79d5af4-01e1-445d-8c3f-222aa8f73b06 (/health/instance) 2020-03-31 08:59:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:04,454 [544] [DEBUG] [app] Starting request: urn:request:62f5ae2e-4141-465e-8991-4b7fbd1d1ba8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:04,454 [544] [DEBUG] [app] Starting request: urn:request:62f5ae2e-4141-465e-8991-4b7fbd1d1ba8 (/v1/_internal_ping) 2020-03-31 08:59:04,455 [544] [DEBUG] [app] Ending request: urn:request:62f5ae2e-4141-465e-8991-4b7fbd1d1ba8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:04,455 [544] [DEBUG] [app] Ending request: urn:request:62f5ae2e-4141-465e-8991-4b7fbd1d1ba8 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:04,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:04,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:59:04,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:59:04,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:04,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:04,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:04,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:04,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:04,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:04,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:04,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:04,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:04,488 [525] [DEBUG] [app] Starting request: urn:request:63f06a9f-8704-4c95-967a-10e58fabc71a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:04,488 [525] [DEBUG] [app] Starting request: urn:request:63f06a9f-8704-4c95-967a-10e58fabc71a (/_internal_ping) 2020-03-31 08:59:04,488 [525] [DEBUG] [app] Ending request: urn:request:63f06a9f-8704-4c95-967a-10e58fabc71a (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:04,488 [525] [DEBUG] [app] Ending request: urn:request:63f06a9f-8704-4c95-967a-10e58fabc71a (/_internal_ping) 2020-03-31 08:59:04,489 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:04,489 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:59:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:59:04,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:04,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:04,503 [534] [DEBUG] [app] Starting request: urn:request:3ec9c6f2-8249-4a18-8d3b-d76c73c749c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:04,503 [534] [DEBUG] [app] Starting request: urn:request:3ec9c6f2-8249-4a18-8d3b-d76c73c749c2 (/c1/_internal_ping) 2020-03-31 08:59:04,504 [534] [DEBUG] [app] Ending request: urn:request:3ec9c6f2-8249-4a18-8d3b-d76c73c749c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:04,504 [534] [DEBUG] [app] Ending request: urn:request:3ec9c6f2-8249-4a18-8d3b-d76c73c749c2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:04,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:04,505 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:04,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:59:04,505 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:04,506 [518] [DEBUG] [app] Ending request: urn:request:d79d5af4-01e1-445d-8c3f-222aa8f73b06 (/health/instance) 2020-03-31 08:59:04,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:04,506 [518] [DEBUG] [app] Ending request: urn:request:d79d5af4-01e1-445d-8c3f-222aa8f73b06 (/health/instance) 2020-03-31 08:59:04,506 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:04,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:59:04,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 08:59:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:16.924497+00:00 (in 9.999262 seconds) 2020-03-31 08:59:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:16 UTC)" (scheduled at 2020-03-31 08:59:06.924497+00:00) 2020-03-31 08:59:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 6, 925999), True, datetime.datetime(2020, 3, 31, 8, 59, 6, 925999), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:59:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:16.924497+00:00 (in 9.999262 seconds) 2020-03-31 08:59:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:16 UTC)" (scheduled at 2020-03-31 08:59:06.924497+00:00) 2020-03-31 08:59:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 6, 925999), True, datetime.datetime(2020, 3, 31, 8, 59, 6, 925999), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:59:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:59:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:59:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:16 UTC)" executed successfully 2020-03-31 08:59:08,273 [518] [DEBUG] [app] Starting request: urn:request:693d3323-ed60-47ba-81f1-7b1b991487ef (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:08,273 [518] [DEBUG] [app] Starting request: urn:request:693d3323-ed60-47ba-81f1-7b1b991487ef (/health/instance) 2020-03-31 08:59:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:08,279 [554] [DEBUG] [app] Starting request: urn:request:b23980c2-8da8-4cb4-a0a8-0f9235ecf042 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:08,279 [554] [DEBUG] [app] Starting request: urn:request:b23980c2-8da8-4cb4-a0a8-0f9235ecf042 (/v1/_internal_ping) 2020-03-31 08:59:08,280 [554] [DEBUG] [app] Ending request: urn:request:b23980c2-8da8-4cb4-a0a8-0f9235ecf042 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:08,280 [554] [DEBUG] [app] Ending request: urn:request:b23980c2-8da8-4cb4-a0a8-0f9235ecf042 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:08,281 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:08,281 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:08,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:59:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:59:08,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:08,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:08,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:08,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:08,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:08,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:08,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:08,312 [518] [DEBUG] [app] Starting request: urn:request:eae22ed2-d8ae-48c0-9d79-a98c4b818e9c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:08,312 [518] [DEBUG] [app] Starting request: urn:request:eae22ed2-d8ae-48c0-9d79-a98c4b818e9c (/_internal_ping) 2020-03-31 08:59:08,313 [518] [DEBUG] [app] Ending request: urn:request:eae22ed2-d8ae-48c0-9d79-a98c4b818e9c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:08,313 [518] [DEBUG] [app] Ending request: urn:request:eae22ed2-d8ae-48c0-9d79-a98c4b818e9c (/_internal_ping) 2020-03-31 08:59:08,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:08,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:59:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:59:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:08,328 [534] [DEBUG] [app] Starting request: urn:request:49c8ab70-08ed-48ba-8672-d7a3c376ca02 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:08,328 [534] [DEBUG] [app] Starting request: urn:request:49c8ab70-08ed-48ba-8672-d7a3c376ca02 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:08,329 [534] [DEBUG] [app] Ending request: urn:request:49c8ab70-08ed-48ba-8672-d7a3c376ca02 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:08,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:59:08,329 [534] [DEBUG] [app] Ending request: urn:request:49c8ab70-08ed-48ba-8672-d7a3c376ca02 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:08,331 [518] [DEBUG] [app] Ending request: urn:request:693d3323-ed60-47ba-81f1-7b1b991487ef (/health/instance) 2020-03-31 08:59:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:59:08,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:59:08,331 [518] [DEBUG] [app] Ending request: urn:request:693d3323-ed60-47ba-81f1-7b1b991487ef (/health/instance) 2020-03-31 08:59:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:59:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:59:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:21.440707+00:00 (in 9.995888 seconds) 2020-03-31 08:59:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:21 UTC)" (scheduled at 2020-03-31 08:59:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:59:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:21.440707+00:00 (in 9.995888 seconds) 2020-03-31 08:59:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:21 UTC)" (scheduled at 2020-03-31 08:59:11.440707+00:00) 2020-03-31 08:59:11,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:59:11,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:59:11,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:59:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:59:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:59:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:21 UTC)" executed successfully 2020-03-31 08:59:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:59:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:24.323352+00:00 (in 9.999024 seconds) 2020-03-31 08:59:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:24 UTC)" (scheduled at 2020-03-31 08:59:14.323352+00:00) 2020-03-31 08:59:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:59:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:24.323352+00:00 (in 9.999024 seconds) 2020-03-31 08:59:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:24 UTC)" (scheduled at 2020-03-31 08:59:14.323352+00:00) 2020-03-31 08:59:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 14, 325290), True, datetime.datetime(2020, 3, 31, 8, 59, 14, 325290), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:59:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 14, 325290), True, datetime.datetime(2020, 3, 31, 8, 59, 14, 325290), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:59:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:14,344 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 08:59:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:59:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:24 UTC)" executed successfully 2020-03-31 08:59:14,446 [518] [DEBUG] [app] Starting request: urn:request:2005be8f-5c91-44a2-895c-9e5936bcaee6 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:14,446 [518] [DEBUG] [app] Starting request: urn:request:2005be8f-5c91-44a2-895c-9e5936bcaee6 (/health/instance) 2020-03-31 08:59:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:14,454 [555] [DEBUG] [app] Starting request: urn:request:4d1a7521-2640-4c14-88d1-5a50b697dd31 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:14,454 [555] [DEBUG] [app] Starting request: urn:request:4d1a7521-2640-4c14-88d1-5a50b697dd31 (/v1/_internal_ping) 2020-03-31 08:59:14,456 [555] [DEBUG] [app] Ending request: urn:request:4d1a7521-2640-4c14-88d1-5a50b697dd31 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:14,456 [555] [DEBUG] [app] Ending request: urn:request:4d1a7521-2640-4c14-88d1-5a50b697dd31 (/v1/_internal_ping) 2020-03-31 08:59:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:59:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:59:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:59:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:14,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:14,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:14,488 [518] [DEBUG] [app] Starting request: urn:request:76cd0ccd-b2ab-44df-b538-ec092bf85792 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:14,488 [518] [DEBUG] [app] Starting request: urn:request:76cd0ccd-b2ab-44df-b538-ec092bf85792 (/_internal_ping) 2020-03-31 08:59:14,489 [518] [DEBUG] [app] Ending request: urn:request:76cd0ccd-b2ab-44df-b538-ec092bf85792 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:14,489 [518] [DEBUG] [app] Ending request: urn:request:76cd0ccd-b2ab-44df-b538-ec092bf85792 (/_internal_ping) 2020-03-31 08:59:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:59:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:14,501 [534] [DEBUG] [app] Starting request: urn:request:b0dc1134-efc0-47e2-8322-64cfe6b8ae6b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:14,501 [534] [DEBUG] [app] Starting request: urn:request:b0dc1134-efc0-47e2-8322-64cfe6b8ae6b (/c1/_internal_ping) 2020-03-31 08:59:14,503 [534] [DEBUG] [app] Ending request: urn:request:b0dc1134-efc0-47e2-8322-64cfe6b8ae6b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:14,503 [534] [DEBUG] [app] Ending request: urn:request:b0dc1134-efc0-47e2-8322-64cfe6b8ae6b (/c1/_internal_ping) 2020-03-31 08:59:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:14,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:14,505 [518] [DEBUG] [app] Ending request: urn:request:2005be8f-5c91-44a2-895c-9e5936bcaee6 (/health/instance) 2020-03-31 08:59:14,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:59:14,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:59:14,505 [518] [DEBUG] [app] Ending request: urn:request:2005be8f-5c91-44a2-895c-9e5936bcaee6 (/health/instance) 2020-03-31 08:59:14,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:14,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 08:59:14,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:59:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:59:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:45.064514+00:00 (in 29.998986 seconds) 2020-03-31 08:59:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:45 UTC)" (scheduled at 2020-03-31 08:59:15.064514+00:00) gcworker stdout | 2020-03-31 08:59:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:45.064514+00:00 (in 29.998986 seconds) 2020-03-31 08:59:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:45 UTC)" (scheduled at 2020-03-31 08:59:15.064514+00:00) 2020-03-31 08:59:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435555, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:59:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435555, 1209600, 500, 1, 0]) 2020-03-31 08:59:15,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:59:15,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:59:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:45 UTC)" executed successfully gcworker stdout | 2020-03-31 08:59:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:45 UTC)" executed successfully 2020-03-31 08:59:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:59:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:26.924497+00:00 (in 9.999233 seconds) 2020-03-31 08:59:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:26 UTC)" (scheduled at 2020-03-31 08:59:16.924497+00:00) 2020-03-31 08:59:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:59:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:26.924497+00:00 (in 9.999233 seconds) 2020-03-31 08:59:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:26 UTC)" (scheduled at 2020-03-31 08:59:16.924497+00:00) 2020-03-31 08:59:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 16, 926219), True, datetime.datetime(2020, 3, 31, 8, 59, 16, 926219), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:59:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 16, 926219), True, datetime.datetime(2020, 3, 31, 8, 59, 16, 926219), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:59:16,952 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:16,952 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:16,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:59:16,952 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:16,952 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:16,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:26 UTC)" executed successfully 2020-03-31 08:59:18,273 [518] [DEBUG] [app] Starting request: urn:request:24372319-0995-417f-bdba-e9c25f1570e9 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:18,273 [518] [DEBUG] [app] Starting request: urn:request:24372319-0995-417f-bdba-e9c25f1570e9 (/health/instance) 2020-03-31 08:59:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:18,279 [538] [DEBUG] [app] Starting request: urn:request:97c04749-e379-4bb9-a30a-ccfefdde91bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:18,279 [538] [DEBUG] [app] Starting request: urn:request:97c04749-e379-4bb9-a30a-ccfefdde91bd (/v1/_internal_ping) 2020-03-31 08:59:18,280 [538] [DEBUG] [app] Ending request: urn:request:97c04749-e379-4bb9-a30a-ccfefdde91bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:18,280 [538] [DEBUG] [app] Ending request: urn:request:97c04749-e379-4bb9-a30a-ccfefdde91bd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:18,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:18,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:18,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:59:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:59:18,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:18,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:18,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:18,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:18,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:18,316 [527] [DEBUG] [app] Starting request: urn:request:a7047056-3812-4a75-9820-ed332d2eb255 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:18,316 [527] [DEBUG] [app] Starting request: urn:request:a7047056-3812-4a75-9820-ed332d2eb255 (/_internal_ping) 2020-03-31 08:59:18,317 [527] [DEBUG] [app] Ending request: urn:request:a7047056-3812-4a75-9820-ed332d2eb255 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:18,317 [527] [DEBUG] [app] Ending request: urn:request:a7047056-3812-4a75-9820-ed332d2eb255 (/_internal_ping) 2020-03-31 08:59:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:59:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:18,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:18,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 08:59:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:59:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:18,332 [536] [DEBUG] [app] Starting request: urn:request:1911b0c1-0a8d-46c8-859b-272544f9acff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:18,332 [536] [DEBUG] [app] Starting request: urn:request:1911b0c1-0a8d-46c8-859b-272544f9acff (/c1/_internal_ping) 2020-03-31 08:59:18,333 [536] [DEBUG] [app] Ending request: urn:request:1911b0c1-0a8d-46c8-859b-272544f9acff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:18,333 [536] [DEBUG] [app] Ending request: urn:request:1911b0c1-0a8d-46c8-859b-272544f9acff (/c1/_internal_ping) 2020-03-31 08:59:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:18,334 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 08:59:18,334 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:18,335 [518] [DEBUG] [app] Ending request: urn:request:24372319-0995-417f-bdba-e9c25f1570e9 (/health/instance) 2020-03-31 08:59:18,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:18,335 [518] [DEBUG] [app] Ending request: urn:request:24372319-0995-417f-bdba-e9c25f1570e9 (/health/instance) 2020-03-31 08:59:18,336 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:59:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 08:59:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:59:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:31.440707+00:00 (in 9.999053 seconds) 2020-03-31 08:59:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:31 UTC)" (scheduled at 2020-03-31 08:59:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:59:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:31.440707+00:00 (in 9.999053 seconds) 2020-03-31 08:59:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:31 UTC)" (scheduled at 2020-03-31 08:59:21.440707+00:00) 2020-03-31 08:59:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:59:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:59:21,457 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:59:21,457 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:59:21,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:21,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:59:21,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:21,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:31 UTC)" executed successfully 2020-03-31 08:59:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:59:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:52.113221+00:00 (in 29.999242 seconds) 2020-03-31 08:59:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:52 UTC)" (scheduled at 2020-03-31 08:59:22.113221+00:00) securityworker stdout | 2020-03-31 08:59:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:52.113221+00:00 (in 29.999242 seconds) 2020-03-31 08:59:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:52 UTC)" (scheduled at 2020-03-31 08:59:22.113221+00:00) 2020-03-31 08:59:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:59:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:59:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:59:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:59:22,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:59:22,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:59:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:52 UTC)" executed successfully securityworker stdout | 2020-03-31 08:59:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 08:59:52 UTC)" executed successfully 2020-03-31 08:59:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:59:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:24.410927+00:00 (in 0.999355 seconds) 2020-03-31 08:59:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:23 UTC)" (scheduled at 2020-03-31 08:59:23.410371+00:00) security_notification_worker stdout | 2020-03-31 08:59:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:24.410927+00:00 (in 0.999355 seconds) 2020-03-31 08:59:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:23 UTC)" (scheduled at 2020-03-31 08:59:23.410371+00:00) 2020-03-31 08:59:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 08:59:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), u'security_notification/%']) 2020-03-31 08:59:23,427 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), True, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 08:59:23,427 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), True, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), 0, u'security_notification/%']) 2020-03-31 08:59:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), True, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 08:59:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), True, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 8, 59, 23, 412137), u'security_notification/%']) 2020-03-31 08:59:23,437 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 08:59:23,437 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:23,437 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:59:23,437 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:23 UTC)" executed successfully 2020-03-31 08:59:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 08:59:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 08:59:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 08:59:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 08:59:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:59:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:34.323352+00:00 (in 9.998788 seconds) 2020-03-31 08:59:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:34 UTC)" (scheduled at 2020-03-31 08:59:24.323352+00:00) 2020-03-31 08:59:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:59:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:34.323352+00:00 (in 9.998788 seconds) 2020-03-31 08:59:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:34 UTC)" (scheduled at 2020-03-31 08:59:24.323352+00:00) 2020-03-31 08:59:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 24, 325650), True, datetime.datetime(2020, 3, 31, 8, 59, 24, 325650), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:59:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 24, 325650), True, datetime.datetime(2020, 3, 31, 8, 59, 24, 325650), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:59:24,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:59:24,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:59:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:34 UTC)" executed successfully 2020-03-31 08:59:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:59:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:33.330112+00:00 (in 8.918455 seconds) 2020-03-31 08:59:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:24 UTC)" (scheduled at 2020-03-31 08:59:24.410927+00:00) 2020-03-31 08:59:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:59:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:33.330112+00:00 (in 8.918455 seconds) 2020-03-31 08:59:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:24 UTC)" (scheduled at 2020-03-31 08:59:24.410927+00:00) 2020-03-31 08:59:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:24 UTC)" executed successfully 2020-03-31 08:59:24,445 [518] [DEBUG] [app] Starting request: urn:request:6e295ca4-4542-479d-89f4-f24493e0e77f (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:24,445 [518] [DEBUG] [app] Starting request: urn:request:6e295ca4-4542-479d-89f4-f24493e0e77f (/health/instance) 2020-03-31 08:59:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:24,454 [552] [DEBUG] [app] Starting request: urn:request:5d5dc511-ee66-4d63-adbb-0c2e19c13163 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:24,454 [552] [DEBUG] [app] Starting request: urn:request:5d5dc511-ee66-4d63-adbb-0c2e19c13163 (/v1/_internal_ping) 2020-03-31 08:59:24,455 [552] [DEBUG] [app] Ending request: urn:request:5d5dc511-ee66-4d63-adbb-0c2e19c13163 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:24,455 [552] [DEBUG] [app] Ending request: urn:request:5d5dc511-ee66-4d63-adbb-0c2e19c13163 (/v1/_internal_ping) 2020-03-31 08:59:24,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 08:59:24,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:24,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:59:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:59:24,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:24,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:59:24,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:24,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:24,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:24,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:24,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:24,488 [518] [DEBUG] [app] Starting request: urn:request:162d2a93-a902-465c-bea3-2f9ce3cb55e5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:24,488 [518] [DEBUG] [app] Starting request: urn:request:162d2a93-a902-465c-bea3-2f9ce3cb55e5 (/_internal_ping) 2020-03-31 08:59:24,489 [518] [DEBUG] [app] Ending request: urn:request:162d2a93-a902-465c-bea3-2f9ce3cb55e5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:24,489 [518] [DEBUG] [app] Ending request: urn:request:162d2a93-a902-465c-bea3-2f9ce3cb55e5 (/_internal_ping) 2020-03-31 08:59:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:59:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:59:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:24,504 [534] [DEBUG] [app] Starting request: urn:request:472eaecd-4a5b-418e-898a-78a2793e4161 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:24,504 [534] [DEBUG] [app] Starting request: urn:request:472eaecd-4a5b-418e-898a-78a2793e4161 (/c1/_internal_ping) 2020-03-31 08:59:24,505 [534] [DEBUG] [app] Ending request: urn:request:472eaecd-4a5b-418e-898a-78a2793e4161 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:24,505 [534] [DEBUG] [app] Ending request: urn:request:472eaecd-4a5b-418e-898a-78a2793e4161 (/c1/_internal_ping) 2020-03-31 08:59:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:24,506 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:24,507 [518] [DEBUG] [app] Ending request: urn:request:6e295ca4-4542-479d-89f4-f24493e0e77f (/health/instance) 2020-03-31 08:59:24,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:59:24,506 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:59:24,507 [518] [DEBUG] [app] Ending request: urn:request:6e295ca4-4542-479d-89f4-f24493e0e77f (/health/instance) 2020-03-31 08:59:24,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 08:59:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:59:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:59:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:36.924497+00:00 (in 9.999197 seconds) 2020-03-31 08:59:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:36 UTC)" (scheduled at 2020-03-31 08:59:26.924497+00:00) 2020-03-31 08:59:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:59:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:36.924497+00:00 (in 9.999197 seconds) 2020-03-31 08:59:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:36 UTC)" (scheduled at 2020-03-31 08:59:26.924497+00:00) 2020-03-31 08:59:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 26, 926017), True, datetime.datetime(2020, 3, 31, 8, 59, 26, 926017), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:59:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 26, 926017), True, datetime.datetime(2020, 3, 31, 8, 59, 26, 926017), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:59:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:59:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:59:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:36 UTC)" executed successfully 2020-03-31 08:59:28,273 [527] [DEBUG] [app] Starting request: urn:request:11151092-a006-4da5-9fc7-23dd8be56164 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:28,273 [527] [DEBUG] [app] Starting request: urn:request:11151092-a006-4da5-9fc7-23dd8be56164 (/health/instance) 2020-03-31 08:59:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:28,282 [538] [DEBUG] [app] Starting request: urn:request:d66884b6-e3b4-485c-afc8-f51758e4cf89 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:28,282 [538] [DEBUG] [app] Starting request: urn:request:d66884b6-e3b4-485c-afc8-f51758e4cf89 (/v1/_internal_ping) 2020-03-31 08:59:28,283 [538] [DEBUG] [app] Ending request: urn:request:d66884b6-e3b4-485c-afc8-f51758e4cf89 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:28,283 [538] [DEBUG] [app] Ending request: urn:request:d66884b6-e3b4-485c-afc8-f51758e4cf89 (/v1/_internal_ping) 2020-03-31 08:59:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:59:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:28,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:28,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:28,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:28,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:59:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:59:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:28,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:28,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:28,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:28,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:28,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:28,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:28,319 [527] [DEBUG] [app] Starting request: urn:request:7082fa6b-8fb3-4da6-9eef-7cd3d7d1be8c (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:28,319 [527] [DEBUG] [app] Starting request: urn:request:7082fa6b-8fb3-4da6-9eef-7cd3d7d1be8c (/_internal_ping) 2020-03-31 08:59:28,320 [527] [DEBUG] [app] Ending request: urn:request:7082fa6b-8fb3-4da6-9eef-7cd3d7d1be8c (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:59:28,321 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:28,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:28,320 [527] [DEBUG] [app] Ending request: urn:request:7082fa6b-8fb3-4da6-9eef-7cd3d7d1be8c (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:28,321 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:28,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:59:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:59:28,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:28,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:28,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:28,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:28,336 [534] [DEBUG] [app] Starting request: urn:request:6f9ef5ce-20f5-44a0-8597-40ef0cada9c6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:28,336 [534] [DEBUG] [app] Starting request: urn:request:6f9ef5ce-20f5-44a0-8597-40ef0cada9c6 (/c1/_internal_ping) 2020-03-31 08:59:28,338 [534] [DEBUG] [app] Ending request: urn:request:6f9ef5ce-20f5-44a0-8597-40ef0cada9c6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:28,338 [534] [DEBUG] [app] Ending request: urn:request:6f9ef5ce-20f5-44a0-8597-40ef0cada9c6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:59:28,338 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 08:59:28,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:59:28,338 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:28,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:28,341 [527] [DEBUG] [app] Ending request: urn:request:11151092-a006-4da5-9fc7-23dd8be56164 (/health/instance) 2020-03-31 08:59:28,341 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:28,341 [527] [DEBUG] [app] Ending request: urn:request:11151092-a006-4da5-9fc7-23dd8be56164 (/health/instance) 2020-03-31 08:59:28,341 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:28,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 08:59:28,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 08:59:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 08:59:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:00.337699+00:00 (in 29.999256 seconds) 2020-03-31 08:59:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:00 UTC)" (scheduled at 2020-03-31 08:59:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 08:59:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:00.337699+00:00 (in 29.999256 seconds) 2020-03-31 08:59:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:00 UTC)" (scheduled at 2020-03-31 08:59:30.337699+00:00) 2020-03-31 08:59:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 59, 30, 339297), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 08:59:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 8, 59, 30, 339297), False, 50, 1, 0]) 2020-03-31 08:59:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 08:59:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 08:59:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 08:59:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:00 UTC)" executed successfully 2020-03-31 08:59:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:59:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:41.440707+00:00 (in 9.998865 seconds) 2020-03-31 08:59:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:41 UTC)" (scheduled at 2020-03-31 08:59:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:59:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:41.440707+00:00 (in 9.998865 seconds) 2020-03-31 08:59:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:41 UTC)" (scheduled at 2020-03-31 08:59:31.440707+00:00) 2020-03-31 08:59:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:59:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:59:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:59:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:59:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:59:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:41 UTC)" executed successfully 2020-03-31 08:59:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 08:59:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:03.330112+00:00 (in 29.999106 seconds) 2020-03-31 08:59:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:03 UTC)" (scheduled at 2020-03-31 08:59:33.330112+00:00) 2020-03-31 08:59:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 08:59:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:03.330112+00:00 (in 29.999106 seconds) 2020-03-31 08:59:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:03 UTC)" (scheduled at 2020-03-31 08:59:33.330112+00:00) 2020-03-31 08:59:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 33, 331829), True, datetime.datetime(2020, 3, 31, 8, 59, 33, 331829), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 08:59:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 33, 331829), True, datetime.datetime(2020, 3, 31, 8, 59, 33, 331829), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 08:59:33,348 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 08:59:33,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:33,348 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 08:59:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:33,348 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:03 UTC)" executed successfully 2020-03-31 08:59:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:59:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:43.539999+00:00 (in 10.172260 seconds) 2020-03-31 08:59:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:33 UTC)" (scheduled at 2020-03-31 08:59:33.366791+00:00) 2020-03-31 08:59:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 08:59:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:43.539999+00:00 (in 10.172260 seconds) 2020-03-31 08:59:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:33 UTC)" (scheduled at 2020-03-31 08:59:33.366791+00:00) 2020-03-31 08:59:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 33, 368607), True, datetime.datetime(2020, 3, 31, 8, 59, 33, 368607), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 08:59:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 33, 368607), True, datetime.datetime(2020, 3, 31, 8, 59, 33, 368607), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 08:59:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 08:59:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:59:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:33 UTC)" executed successfully 2020-03-31 08:59:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:59:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:44.323352+00:00 (in 9.999025 seconds) 2020-03-31 08:59:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:44 UTC)" (scheduled at 2020-03-31 08:59:34.323352+00:00) 2020-03-31 08:59:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:59:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:44.323352+00:00 (in 9.999025 seconds) 2020-03-31 08:59:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:44 UTC)" (scheduled at 2020-03-31 08:59:34.323352+00:00) 2020-03-31 08:59:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 34, 325370), True, datetime.datetime(2020, 3, 31, 8, 59, 34, 325370), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:59:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 34, 325370), True, datetime.datetime(2020, 3, 31, 8, 59, 34, 325370), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:59:34,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:59:34,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:59:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:44 UTC)" executed successfully 2020-03-31 08:59:34,446 [518] [DEBUG] [app] Starting request: urn:request:c6d6f2cb-0be8-409d-ac68-7ab450fb9a1e (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:34,446 [518] [DEBUG] [app] Starting request: urn:request:c6d6f2cb-0be8-409d-ac68-7ab450fb9a1e (/health/instance) 2020-03-31 08:59:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:34,454 [547] [DEBUG] [app] Starting request: urn:request:3a281da3-3ef2-455d-8bc8-331eb6a1d625 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:34,454 [547] [DEBUG] [app] Starting request: urn:request:3a281da3-3ef2-455d-8bc8-331eb6a1d625 (/v1/_internal_ping) 2020-03-31 08:59:34,455 [547] [DEBUG] [app] Ending request: urn:request:3a281da3-3ef2-455d-8bc8-331eb6a1d625 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:34,455 [547] [DEBUG] [app] Ending request: urn:request:3a281da3-3ef2-455d-8bc8-331eb6a1d625 (/v1/_internal_ping) 2020-03-31 08:59:34,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 08:59:34,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:34,459 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:59:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:59:34,459 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:34,461 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:34,461 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:34,461 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:34,461 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:34,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:34,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:34,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:34,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:34,486 [518] [DEBUG] [app] Starting request: urn:request:4ad6a80a-64c2-4d78-8d95-1c5dfa3efad7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:34,486 [518] [DEBUG] [app] Starting request: urn:request:4ad6a80a-64c2-4d78-8d95-1c5dfa3efad7 (/_internal_ping) 2020-03-31 08:59:34,487 [518] [DEBUG] [app] Ending request: urn:request:4ad6a80a-64c2-4d78-8d95-1c5dfa3efad7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:34,487 [518] [DEBUG] [app] Ending request: urn:request:4ad6a80a-64c2-4d78-8d95-1c5dfa3efad7 (/_internal_ping) 2020-03-31 08:59:34,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:34,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:34,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:34,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:59:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:34,500 [534] [DEBUG] [app] Starting request: urn:request:f3e042a2-6440-4376-a1e3-ff88df4ef5bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:34,500 [534] [DEBUG] [app] Starting request: urn:request:f3e042a2-6440-4376-a1e3-ff88df4ef5bd (/c1/_internal_ping) 2020-03-31 08:59:34,501 [534] [DEBUG] [app] Ending request: urn:request:f3e042a2-6440-4376-a1e3-ff88df4ef5bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:34,501 [534] [DEBUG] [app] Ending request: urn:request:f3e042a2-6440-4376-a1e3-ff88df4ef5bd (/c1/_internal_ping) 2020-03-31 08:59:34,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:34,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:34,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 08:59:34,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:34,503 [518] [DEBUG] [app] Ending request: urn:request:c6d6f2cb-0be8-409d-ac68-7ab450fb9a1e (/health/instance) 2020-03-31 08:59:34,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:34,503 [518] [DEBUG] [app] Ending request: urn:request:c6d6f2cb-0be8-409d-ac68-7ab450fb9a1e (/health/instance) 2020-03-31 08:59:34,503 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 08:59:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 08:59:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:59:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:59:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:38.010934+00:00 (in 1.085323 seconds) 2020-03-31 08:59:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:46 UTC)" (scheduled at 2020-03-31 08:59:36.924497+00:00) 2020-03-31 08:59:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:59:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:38.010934+00:00 (in 1.085323 seconds) 2020-03-31 08:59:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:46 UTC)" (scheduled at 2020-03-31 08:59:36.924497+00:00) 2020-03-31 08:59:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 36, 926324), True, datetime.datetime(2020, 3, 31, 8, 59, 36, 926324), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:59:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 36, 926324), True, datetime.datetime(2020, 3, 31, 8, 59, 36, 926324), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:59:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:59:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:59:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:46 UTC)" executed successfully 2020-03-31 08:59:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:59:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:46.924497+00:00 (in 8.912852 seconds) 2020-03-31 08:59:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:38 UTC)" (scheduled at 2020-03-31 08:59:38.010934+00:00) 2020-03-31 08:59:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:59:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:46.924497+00:00 (in 8.912852 seconds) 2020-03-31 08:59:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:38 UTC)" (scheduled at 2020-03-31 08:59:38.010934+00:00) 2020-03-31 08:59:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:38 UTC)" executed successfully 2020-03-31 08:59:38,273 [518] [DEBUG] [app] Starting request: urn:request:840d7b10-da6b-4b87-8bef-153b45041290 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:38,273 [518] [DEBUG] [app] Starting request: urn:request:840d7b10-da6b-4b87-8bef-153b45041290 (/health/instance) 2020-03-31 08:59:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:38,277 [551] [DEBUG] [app] Starting request: urn:request:33d5fe8d-9a9d-4548-b6f9-21309000b66c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:38,277 [551] [DEBUG] [app] Starting request: urn:request:33d5fe8d-9a9d-4548-b6f9-21309000b66c (/v1/_internal_ping) 2020-03-31 08:59:38,278 [551] [DEBUG] [app] Ending request: urn:request:33d5fe8d-9a9d-4548-b6f9-21309000b66c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:38,278 [551] [DEBUG] [app] Ending request: urn:request:33d5fe8d-9a9d-4548-b6f9-21309000b66c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:38,279 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:38,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:38,279 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:38,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:38,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:38,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:59:38,282 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:59:38,282 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:38,283 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:38,283 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:38,284 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:38,284 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:38,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:38,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:38,302 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:38,302 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:38,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:38,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:38,308 [527] [DEBUG] [app] Starting request: urn:request:00b965d6-e60a-44f2-8943-a6ef18878d79 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:38,308 [527] [DEBUG] [app] Starting request: urn:request:00b965d6-e60a-44f2-8943-a6ef18878d79 (/_internal_ping) 2020-03-31 08:59:38,309 [527] [DEBUG] [app] Ending request: urn:request:00b965d6-e60a-44f2-8943-a6ef18878d79 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:38,309 [527] [DEBUG] [app] Ending request: urn:request:00b965d6-e60a-44f2-8943-a6ef18878d79 (/_internal_ping) 2020-03-31 08:59:38,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:38,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:59:38,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:38,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:59:38,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:38,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:59:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:38,324 [537] [DEBUG] [app] Starting request: urn:request:79e4374f-9858-4e26-85a1-c8f287bc5045 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:38,324 [537] [DEBUG] [app] Starting request: urn:request:79e4374f-9858-4e26-85a1-c8f287bc5045 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:38,325 [537] [DEBUG] [app] Ending request: urn:request:79e4374f-9858-4e26-85a1-c8f287bc5045 (/c1/_internal_ping) 2020-03-31 08:59:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:38,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:38,327 [518] [DEBUG] [app] Ending request: urn:request:840d7b10-da6b-4b87-8bef-153b45041290 (/health/instance) 2020-03-31 08:59:38,327 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 08:59:38,325 [537] [DEBUG] [app] Ending request: urn:request:79e4374f-9858-4e26-85a1-c8f287bc5045 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:59:38,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:38,327 [518] [DEBUG] [app] Ending request: urn:request:840d7b10-da6b-4b87-8bef-153b45041290 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:38,327 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 08:59:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:59:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:59:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:51.440707+00:00 (in 9.999182 seconds) 2020-03-31 08:59:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:51 UTC)" (scheduled at 2020-03-31 08:59:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:59:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:51.440707+00:00 (in 9.999182 seconds) 2020-03-31 08:59:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:51 UTC)" (scheduled at 2020-03-31 08:59:41.440707+00:00) 2020-03-31 08:59:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:59:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:59:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:59:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:59:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:59:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:51 UTC)" executed successfully 2020-03-31 08:59:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 08:59:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:14.539408+00:00 (in 30.998562 seconds) 2020-03-31 08:59:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" (scheduled at 2020-03-31 08:59:43.539999+00:00) exportactionlogsworker stdout | 2020-03-31 08:59:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:14.539408+00:00 (in 30.998562 seconds) 2020-03-31 08:59:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" (scheduled at 2020-03-31 08:59:43.539999+00:00) 2020-03-31 08:59:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 08:59:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" executed successfully 2020-03-31 08:59:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:59:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:49.498966+00:00 (in 5.174136 seconds) 2020-03-31 08:59:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:54 UTC)" (scheduled at 2020-03-31 08:59:44.323352+00:00) 2020-03-31 08:59:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:59:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:49.498966+00:00 (in 5.174136 seconds) 2020-03-31 08:59:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:54 UTC)" (scheduled at 2020-03-31 08:59:44.323352+00:00) 2020-03-31 08:59:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 44, 325577), True, datetime.datetime(2020, 3, 31, 8, 59, 44, 325577), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:59:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 44, 325577), True, datetime.datetime(2020, 3, 31, 8, 59, 44, 325577), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:59:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:59:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:59:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:54 UTC)" executed successfully 2020-03-31 08:59:44,446 [527] [DEBUG] [app] Starting request: urn:request:a84274c6-07be-4002-ae51-240218c2fe72 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:44,446 [527] [DEBUG] [app] Starting request: urn:request:a84274c6-07be-4002-ae51-240218c2fe72 (/health/instance) 2020-03-31 08:59:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:44,453 [556] [DEBUG] [app] Starting request: urn:request:130593c9-2040-4c54-ab06-1d9e11e625a3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:44,453 [556] [DEBUG] [app] Starting request: urn:request:130593c9-2040-4c54-ab06-1d9e11e625a3 (/v1/_internal_ping) 2020-03-31 08:59:44,454 [556] [DEBUG] [app] Ending request: urn:request:130593c9-2040-4c54-ab06-1d9e11e625a3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:44,454 [556] [DEBUG] [app] Ending request: urn:request:130593c9-2040-4c54-ab06-1d9e11e625a3 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:44,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:44,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:59:44,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:59:44,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:44,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:44,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:44,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:44,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:44,486 [527] [DEBUG] [app] Starting request: urn:request:cc7ddafe-4daf-43ac-ab7f-618d930471f9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:44,486 [527] [DEBUG] [app] Starting request: urn:request:cc7ddafe-4daf-43ac-ab7f-618d930471f9 (/_internal_ping) 2020-03-31 08:59:44,487 [527] [DEBUG] [app] Ending request: urn:request:cc7ddafe-4daf-43ac-ab7f-618d930471f9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:44,487 [527] [DEBUG] [app] Ending request: urn:request:cc7ddafe-4daf-43ac-ab7f-618d930471f9 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:59:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:59:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:59:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:59:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:44,501 [534] [DEBUG] [app] Starting request: urn:request:8e933396-a68d-4161-b262-945b92a20ec6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:44,501 [534] [DEBUG] [app] Starting request: urn:request:8e933396-a68d-4161-b262-945b92a20ec6 (/c1/_internal_ping) 2020-03-31 08:59:44,502 [534] [DEBUG] [app] Ending request: urn:request:8e933396-a68d-4161-b262-945b92a20ec6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:44,502 [534] [DEBUG] [app] Ending request: urn:request:8e933396-a68d-4161-b262-945b92a20ec6 (/c1/_internal_ping) 2020-03-31 08:59:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:44,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:44,504 [527] [DEBUG] [app] Ending request: urn:request:a84274c6-07be-4002-ae51-240218c2fe72 (/health/instance) 2020-03-31 08:59:44,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 08:59:44,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:59:44,504 [527] [DEBUG] [app] Ending request: urn:request:a84274c6-07be-4002-ae51-240218c2fe72 (/health/instance) 2020-03-31 08:59:44,505 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:59:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 08:59:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 08:59:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:15.064514+00:00 (in 29.999107 seconds) 2020-03-31 08:59:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:15 UTC)" (scheduled at 2020-03-31 08:59:45.064514+00:00) gcworker stdout | 2020-03-31 08:59:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:15.064514+00:00 (in 29.999107 seconds) 2020-03-31 08:59:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:15 UTC)" (scheduled at 2020-03-31 08:59:45.064514+00:00) 2020-03-31 08:59:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435585, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 08:59:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435585, 1209600, 500, 1, 0]) 2020-03-31 08:59:45,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 08:59:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 08:59:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:15 UTC)" executed successfully gcworker stdout | 2020-03-31 08:59:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:15 UTC)" executed successfully 2020-03-31 08:59:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:59:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:56.924497+00:00 (in 9.998907 seconds) 2020-03-31 08:59:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:56 UTC)" (scheduled at 2020-03-31 08:59:46.924497+00:00) 2020-03-31 08:59:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:59:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:56.924497+00:00 (in 9.998907 seconds) 2020-03-31 08:59:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:56 UTC)" (scheduled at 2020-03-31 08:59:46.924497+00:00) 2020-03-31 08:59:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 46, 926216), True, datetime.datetime(2020, 3, 31, 8, 59, 46, 926216), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:59:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 46, 926216), True, datetime.datetime(2020, 3, 31, 8, 59, 46, 926216), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:59:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:59:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:59:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 08:59:56 UTC)" executed successfully 2020-03-31 08:59:48,273 [518] [DEBUG] [app] Starting request: urn:request:54437e9b-91e6-4c27-bd95-1a6dd8ea5675 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:48,273 [518] [DEBUG] [app] Starting request: urn:request:54437e9b-91e6-4c27-bd95-1a6dd8ea5675 (/health/instance) 2020-03-31 08:59:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:48,282 [555] [DEBUG] [app] Starting request: urn:request:b20ed9c0-903c-4a4d-8564-39c56e515a6f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:48,282 [555] [DEBUG] [app] Starting request: urn:request:b20ed9c0-903c-4a4d-8564-39c56e515a6f (/v1/_internal_ping) 2020-03-31 08:59:48,283 [555] [DEBUG] [app] Ending request: urn:request:b20ed9c0-903c-4a4d-8564-39c56e515a6f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:48,283 [555] [DEBUG] [app] Ending request: urn:request:b20ed9c0-903c-4a4d-8564-39c56e515a6f (/v1/_internal_ping) 2020-03-31 08:59:48,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:48,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:48,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:48,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:48,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 08:59:48,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:48,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 08:59:48,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 08:59:48,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:48,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:48,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:48,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:48,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:48,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:48,314 [518] [DEBUG] [app] Starting request: urn:request:279babfa-1f03-4f36-a822-7e17ce02f029 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:48,314 [518] [DEBUG] [app] Starting request: urn:request:279babfa-1f03-4f36-a822-7e17ce02f029 (/_internal_ping) 2020-03-31 08:59:48,315 [518] [DEBUG] [app] Ending request: urn:request:279babfa-1f03-4f36-a822-7e17ce02f029 (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:48,315 [518] [DEBUG] [app] Ending request: urn:request:279babfa-1f03-4f36-a822-7e17ce02f029 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 08:59:48,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:48,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 08:59:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 08:59:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 08:59:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:59:48,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:48,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:48,328 [537] [DEBUG] [app] Starting request: urn:request:12c91539-c658-4282-a6db-ae1903d88081 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:48,328 [537] [DEBUG] [app] Starting request: urn:request:12c91539-c658-4282-a6db-ae1903d88081 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:48,329 [537] [DEBUG] [app] Ending request: urn:request:12c91539-c658-4282-a6db-ae1903d88081 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:48,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:48,330 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 08:59:48,329 [537] [DEBUG] [app] Ending request: urn:request:12c91539-c658-4282-a6db-ae1903d88081 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:48,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:48,331 [518] [DEBUG] [app] Ending request: urn:request:54437e9b-91e6-4c27-bd95-1a6dd8ea5675 (/health/instance) gunicorn-verbs stdout | 2020-03-31 08:59:48,330 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:48,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:48,331 [518] [DEBUG] [app] Ending request: urn:request:54437e9b-91e6-4c27-bd95-1a6dd8ea5675 (/health/instance) 2020-03-31 08:59:48,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 08:59:48,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:48,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 08:59:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:59:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:54.323352+00:00 (in 4.823577 seconds) 2020-03-31 08:59:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:49 UTC)" (scheduled at 2020-03-31 08:59:49.498966+00:00) 2020-03-31 08:59:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:59:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:54.323352+00:00 (in 4.823577 seconds) 2020-03-31 08:59:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:49 UTC)" (scheduled at 2020-03-31 08:59:49.498966+00:00) 2020-03-31 08:59:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:49 UTC)" executed successfully 2020-03-31 08:59:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:59:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:51 UTC)" (scheduled at 2020-03-31 08:59:51.110610+00:00) 2020-03-31 08:59:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:53.556282+00:00 (in 2.444656 seconds) 2020-03-31 08:59:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 08:59:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:51 UTC)" (scheduled at 2020-03-31 08:59:51.110610+00:00) 2020-03-31 08:59:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:53.556282+00:00 (in 2.444656 seconds) 2020-03-31 08:59:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 51, 112848), True, datetime.datetime(2020, 3, 31, 8, 59, 51, 112848), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 08:59:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 51, 112848), True, datetime.datetime(2020, 3, 31, 8, 59, 51, 112848), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 08:59:51,394 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 08:59:51,394 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:51,394 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:51,394 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:59:51,394 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:51,394 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:51 UTC)" executed successfully 2020-03-31 08:59:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 08:59:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:01.440707+00:00 (in 9.999169 seconds) 2020-03-31 08:59:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:01 UTC)" (scheduled at 2020-03-31 08:59:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 08:59:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:01.440707+00:00 (in 9.999169 seconds) 2020-03-31 08:59:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:01 UTC)" (scheduled at 2020-03-31 08:59:51.440707+00:00) 2020-03-31 08:59:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 08:59:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 08:59:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 08:59:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 08:59:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 08:59:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:01 UTC)" executed successfully 2020-03-31 08:59:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 08:59:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:22.113221+00:00 (in 29.999261 seconds) 2020-03-31 08:59:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:22 UTC)" (scheduled at 2020-03-31 08:59:52.113221+00:00) securityworker stdout | 2020-03-31 08:59:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:22.113221+00:00 (in 29.999261 seconds) 2020-03-31 08:59:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:22 UTC)" (scheduled at 2020-03-31 08:59:52.113221+00:00) 2020-03-31 08:59:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 08:59:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 08:59:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 08:59:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 08:59:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 08:59:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 08:59:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:52,129 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:22 UTC)" executed successfully securityworker stdout | 2020-03-31 08:59:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:52,129 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:22 UTC)" executed successfully 2020-03-31 08:59:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:59:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:53,558 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:59.560326+00:00 (in 6.002522 seconds) 2020-03-31 08:59:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:53 UTC)" (scheduled at 2020-03-31 08:59:53.556282+00:00) namespacegcworker stdout | 2020-03-31 08:59:53,558 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 08:59:59.560326+00:00 (in 6.002522 seconds) 2020-03-31 08:59:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:53 UTC)" (scheduled at 2020-03-31 08:59:53.556282+00:00) 2020-03-31 08:59:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 08:59:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), u'namespacegc/%']) 2020-03-31 08:59:53,575 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), True, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 08:59:53,575 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), True, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), 0, u'namespacegc/%']) 2020-03-31 08:59:53,580 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), True, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 08:59:53,580 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), True, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 8, 59, 53, 558623), u'namespacegc/%']) 2020-03-31 08:59:53,586 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 08:59:53,586 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:53,586 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:59:53,586 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:04:53 UTC)" executed successfully 2020-03-31 08:59:53,587 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 08:59:53,587 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:53,588 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 08:59:53,588 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:53,589 [192] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics namespacegcworker stdout | 2020-03-31 08:59:53,589 [192] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 08:59:53,590 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 08:59:53,590 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:53,591 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 08:59:53,591 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:53,592 [192] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics namespacegcworker stdout | 2020-03-31 08:59:53,592 [192] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 08:59:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:04.323352+00:00 (in 9.999087 seconds) 2020-03-31 08:59:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:04 UTC)" (scheduled at 2020-03-31 08:59:54.323352+00:00) 2020-03-31 08:59:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 08:59:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:04.323352+00:00 (in 9.999087 seconds) 2020-03-31 08:59:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:04 UTC)" (scheduled at 2020-03-31 08:59:54.323352+00:00) 2020-03-31 08:59:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 54, 325135), True, datetime.datetime(2020, 3, 31, 8, 59, 54, 325135), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 08:59:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 54, 325135), True, datetime.datetime(2020, 3, 31, 8, 59, 54, 325135), 0, u'notification/%', 50, 1, 0]) 2020-03-31 08:59:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 08:59:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 08:59:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:04 UTC)" executed successfully 2020-03-31 08:59:54,446 [518] [DEBUG] [app] Starting request: urn:request:20b8111e-3e11-481e-9d08-0944193b26bd (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:54,446 [518] [DEBUG] [app] Starting request: urn:request:20b8111e-3e11-481e-9d08-0944193b26bd (/health/instance) 2020-03-31 08:59:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:54,454 [552] [DEBUG] [app] Starting request: urn:request:28344ed6-552a-402b-80f3-bd34006a4bd4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:54,454 [552] [DEBUG] [app] Starting request: urn:request:28344ed6-552a-402b-80f3-bd34006a4bd4 (/v1/_internal_ping) 2020-03-31 08:59:54,455 [552] [DEBUG] [app] Ending request: urn:request:28344ed6-552a-402b-80f3-bd34006a4bd4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:54,455 [552] [DEBUG] [app] Ending request: urn:request:28344ed6-552a-402b-80f3-bd34006a4bd4 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:54,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:54,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:59:54,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:59:54,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:54,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:54,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:54,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:54,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:54,491 [518] [DEBUG] [app] Starting request: urn:request:df7883f2-e689-4806-ad3f-2412fc7572db (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:54,491 [518] [DEBUG] [app] Starting request: urn:request:df7883f2-e689-4806-ad3f-2412fc7572db (/_internal_ping) 2020-03-31 08:59:54,492 [518] [DEBUG] [app] Ending request: urn:request:df7883f2-e689-4806-ad3f-2412fc7572db (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:54,492 [518] [DEBUG] [app] Ending request: urn:request:df7883f2-e689-4806-ad3f-2412fc7572db (/_internal_ping) 2020-03-31 08:59:54,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:59:54,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 08:59:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 08:59:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:54,506 [537] [DEBUG] [app] Starting request: urn:request:48905c71-088c-4dfa-9053-5001a8e1c254 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:54,506 [537] [DEBUG] [app] Starting request: urn:request:48905c71-088c-4dfa-9053-5001a8e1c254 (/c1/_internal_ping) 2020-03-31 08:59:54,507 [537] [DEBUG] [app] Ending request: urn:request:48905c71-088c-4dfa-9053-5001a8e1c254 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:54,507 [537] [DEBUG] [app] Ending request: urn:request:48905c71-088c-4dfa-9053-5001a8e1c254 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:54,507 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:59:54,507 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:54,509 [518] [DEBUG] [app] Ending request: urn:request:20b8111e-3e11-481e-9d08-0944193b26bd (/health/instance) 2020-03-31 08:59:54,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 08:59:54,509 [518] [DEBUG] [app] Ending request: urn:request:20b8111e-3e11-481e-9d08-0944193b26bd (/health/instance) 2020-03-31 08:59:54,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 08:59:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 08:59:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:06.924497+00:00 (in 9.999345 seconds) 2020-03-31 08:59:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:06 UTC)" (scheduled at 2020-03-31 08:59:56.924497+00:00) 2020-03-31 08:59:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 08:59:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:06.924497+00:00 (in 9.999345 seconds) 2020-03-31 08:59:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:06 UTC)" (scheduled at 2020-03-31 08:59:56.924497+00:00) 2020-03-31 08:59:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 08:59:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 56, 925942), True, datetime.datetime(2020, 3, 31, 8, 59, 56, 925942), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 08:59:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 8, 59, 56, 925942), True, datetime.datetime(2020, 3, 31, 8, 59, 56, 925942), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 08:59:56,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 08:59:56,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 08:59:56,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:56,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 08:59:56,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:56,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:06 UTC)" executed successfully 2020-03-31 08:59:58,273 [518] [DEBUG] [app] Starting request: urn:request:e4ae6ec9-31a7-4dfa-b523-cafc817dc5d7 (/health/instance) gunicorn-web stdout | 2020-03-31 08:59:58,273 [518] [DEBUG] [app] Starting request: urn:request:e4ae6ec9-31a7-4dfa-b523-cafc817dc5d7 (/health/instance) 2020-03-31 08:59:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:58,278 [552] [DEBUG] [app] Starting request: urn:request:32fb6c65-9e47-47a7-aa56-5482043943e3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:58,278 [552] [DEBUG] [app] Starting request: urn:request:32fb6c65-9e47-47a7-aa56-5482043943e3 (/v1/_internal_ping) 2020-03-31 08:59:58,279 [552] [DEBUG] [app] Ending request: urn:request:32fb6c65-9e47-47a7-aa56-5482043943e3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 08:59:58,279 [552] [DEBUG] [app] Ending request: urn:request:32fb6c65-9e47-47a7-aa56-5482043943e3 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 08:59:58,279 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 08:59:58,279 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 08:59:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 08:59:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 08:59:58,282 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 08:59:58,282 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 08:59:58,283 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 08:59:58,283 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 08:59:58,284 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 08:59:58,284 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 08:59:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 08:59:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 08:59:58,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 08:59:58,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 08:59:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:58,315 [527] [DEBUG] [app] Starting request: urn:request:87b18e92-e3c7-4f15-8107-de58b33650bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:58,315 [527] [DEBUG] [app] Starting request: urn:request:87b18e92-e3c7-4f15-8107-de58b33650bc (/_internal_ping) 2020-03-31 08:59:58,316 [527] [DEBUG] [app] Ending request: urn:request:87b18e92-e3c7-4f15-8107-de58b33650bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 08:59:58,316 [527] [DEBUG] [app] Ending request: urn:request:87b18e92-e3c7-4f15-8107-de58b33650bc (/_internal_ping) 2020-03-31 08:59:58,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 08:59:58,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 08:59:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 08:59:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 08:59:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 08:59:58,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 08:59:58,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 08:59:58,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 08:59:58,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 08:59:58,331 [534] [DEBUG] [app] Starting request: urn:request:310f6736-451c-422d-a788-71add977cf76 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:58,331 [534] [DEBUG] [app] Starting request: urn:request:310f6736-451c-422d-a788-71add977cf76 (/c1/_internal_ping) 2020-03-31 08:59:58,332 [534] [DEBUG] [app] Ending request: urn:request:310f6736-451c-422d-a788-71add977cf76 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 08:59:58,332 [534] [DEBUG] [app] Ending request: urn:request:310f6736-451c-422d-a788-71add977cf76 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 08:59:58,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 08:59:58,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 08:59:58,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 08:59:58,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 08:59:58,334 [518] [DEBUG] [app] Ending request: urn:request:e4ae6ec9-31a7-4dfa-b523-cafc817dc5d7 (/health/instance) 2020-03-31 08:59:58,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 08:59:58,334 [518] [DEBUG] [app] Ending request: urn:request:e4ae6ec9-31a7-4dfa-b523-cafc817dc5d7 (/health/instance) 2020-03-31 08:59:58,334 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 08:59:58,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 08:59:58,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:08:59:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:08:59:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 08:59:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 08:59:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 08:59:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:51.110610+00:00 (in 51.549311 seconds) 2020-03-31 08:59:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:59 UTC)" (scheduled at 2020-03-31 08:59:59.560326+00:00) 2020-03-31 08:59:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 08:59:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:51.110610+00:00 (in 51.549311 seconds) 2020-03-31 08:59:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:59 UTC)" (scheduled at 2020-03-31 08:59:59.560326+00:00) 2020-03-31 08:59:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 08:59:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:59 UTC)" executed successfully 2020-03-31 09:00:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:00:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:30.337699+00:00 (in 29.998902 seconds) 2020-03-31 09:00:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:30 UTC)" (scheduled at 2020-03-31 09:00:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:00:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:30.337699+00:00 (in 29.998902 seconds) 2020-03-31 09:00:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:30 UTC)" (scheduled at 2020-03-31 09:00:00.337699+00:00) 2020-03-31 09:00:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 0, 0, 339784), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:00:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 0, 0, 339784), False, 50, 1, 0]) 2020-03-31 09:00:00,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:00:00,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:00:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:00,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:00:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:00,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:30 UTC)" executed successfully 2020-03-31 09:00:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:00:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:11.440707+00:00 (in 9.999215 seconds) 2020-03-31 09:00:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:11 UTC)" (scheduled at 2020-03-31 09:00:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:00:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:11.440707+00:00 (in 9.999215 seconds) 2020-03-31 09:00:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:11 UTC)" (scheduled at 2020-03-31 09:00:01.440707+00:00) 2020-03-31 09:00:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:00:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:00:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:00:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:00:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:00:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:11 UTC)" executed successfully 2020-03-31 09:00:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:00:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:24.410927+00:00 (in 21.079861 seconds) 2020-03-31 09:00:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:33 UTC)" (scheduled at 2020-03-31 09:00:03.330112+00:00) 2020-03-31 09:00:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:00:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:24.410927+00:00 (in 21.079861 seconds) 2020-03-31 09:00:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:33 UTC)" (scheduled at 2020-03-31 09:00:03.330112+00:00) 2020-03-31 09:00:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 3, 332117), True, datetime.datetime(2020, 3, 31, 9, 0, 3, 332117), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:00:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 3, 332117), True, datetime.datetime(2020, 3, 31, 9, 0, 3, 332117), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:00:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:03,350 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:00:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:00:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:33 UTC)" executed successfully 2020-03-31 09:00:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:00:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:14 UTC)" (scheduled at 2020-03-31 09:00:04.323352+00:00) 2020-03-31 09:00:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:00:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:14 UTC)" (scheduled at 2020-03-31 09:00:04.323352+00:00) 2020-03-31 09:00:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:04,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:14.323352+00:00 (in 9.998912 seconds) notificationworker stdout | 2020-03-31 09:00:04,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:14.323352+00:00 (in 9.998912 seconds) 2020-03-31 09:00:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 4, 325164), True, datetime.datetime(2020, 3, 31, 9, 0, 4, 325164), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:00:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 4, 325164), True, datetime.datetime(2020, 3, 31, 9, 0, 4, 325164), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:00:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:00:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:00:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:14 UTC)" executed successfully 2020-03-31 09:00:04,446 [518] [DEBUG] [app] Starting request: urn:request:9e47a879-0618-4838-96e5-53db5387525b (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:04,446 [518] [DEBUG] [app] Starting request: urn:request:9e47a879-0618-4838-96e5-53db5387525b (/health/instance) 2020-03-31 09:00:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:04,454 [556] [DEBUG] [app] Starting request: urn:request:f2a5ede5-c118-4f1c-9bd4-2ddae984fb80 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:04,454 [556] [DEBUG] [app] Starting request: urn:request:f2a5ede5-c118-4f1c-9bd4-2ddae984fb80 (/v1/_internal_ping) 2020-03-31 09:00:04,455 [556] [DEBUG] [app] Ending request: urn:request:f2a5ede5-c118-4f1c-9bd4-2ddae984fb80 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:04,455 [556] [DEBUG] [app] Ending request: urn:request:f2a5ede5-c118-4f1c-9bd4-2ddae984fb80 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:04,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:00:04,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:04,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:00:04,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:04,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:04,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:04,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:04,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:04,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:04,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:04,492 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:04,492 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:04,498 [518] [DEBUG] [app] Starting request: urn:request:a4ae3469-af68-4271-92ee-081041713e08 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:04,498 [518] [DEBUG] [app] Starting request: urn:request:a4ae3469-af68-4271-92ee-081041713e08 (/_internal_ping) 2020-03-31 09:00:04,498 [518] [DEBUG] [app] Ending request: urn:request:a4ae3469-af68-4271-92ee-081041713e08 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:04,498 [518] [DEBUG] [app] Ending request: urn:request:a4ae3469-af68-4271-92ee-081041713e08 (/_internal_ping) 2020-03-31 09:00:04,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:00:04,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:00:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:00:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:04,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:04,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:04,510 [537] [DEBUG] [app] Starting request: urn:request:607edbd1-80c3-40d3-b679-31a901297066 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:04,510 [537] [DEBUG] [app] Starting request: urn:request:607edbd1-80c3-40d3-b679-31a901297066 (/c1/_internal_ping) 2020-03-31 09:00:04,511 [537] [DEBUG] [app] Ending request: urn:request:607edbd1-80c3-40d3-b679-31a901297066 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:04,511 [537] [DEBUG] [app] Ending request: urn:request:607edbd1-80c3-40d3-b679-31a901297066 (/c1/_internal_ping) 2020-03-31 09:00:04,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:04,511 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:04,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:04,513 [518] [DEBUG] [app] Ending request: urn:request:9e47a879-0618-4838-96e5-53db5387525b (/health/instance) 2020-03-31 09:00:04,514 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:00:04,511 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:04,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:00:04,513 [518] [DEBUG] [app] Ending request: urn:request:9e47a879-0618-4838-96e5-53db5387525b (/health/instance) 2020-03-31 09:00:04,514 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:04,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:00:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:00:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:16.924497+00:00 (in 9.999241 seconds) 2020-03-31 09:00:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:16 UTC)" (scheduled at 2020-03-31 09:00:06.924497+00:00) 2020-03-31 09:00:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:00:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:16.924497+00:00 (in 9.999241 seconds) 2020-03-31 09:00:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:16 UTC)" (scheduled at 2020-03-31 09:00:06.924497+00:00) 2020-03-31 09:00:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 6, 925937), True, datetime.datetime(2020, 3, 31, 9, 0, 6, 925937), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:00:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 6, 925937), True, datetime.datetime(2020, 3, 31, 9, 0, 6, 925937), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:00:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:00:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:00:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:16 UTC)" executed successfully 2020-03-31 09:00:08,273 [526] [DEBUG] [app] Starting request: urn:request:47cf27c9-fe45-4eb9-bf03-4f46de038637 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:08,273 [526] [DEBUG] [app] Starting request: urn:request:47cf27c9-fe45-4eb9-bf03-4f46de038637 (/health/instance) 2020-03-31 09:00:08,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:08,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:08,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:08,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:08,281 [545] [DEBUG] [app] Starting request: urn:request:1b9df865-72e0-4e58-a514-c65c52d1bdd6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:08,281 [545] [DEBUG] [app] Starting request: urn:request:1b9df865-72e0-4e58-a514-c65c52d1bdd6 (/v1/_internal_ping) 2020-03-31 09:00:08,282 [545] [DEBUG] [app] Ending request: urn:request:1b9df865-72e0-4e58-a514-c65c52d1bdd6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:08,282 [545] [DEBUG] [app] Ending request: urn:request:1b9df865-72e0-4e58-a514-c65c52d1bdd6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:08,283 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:08,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:00:08,283 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:08,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:08,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:08,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:08,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:00:08,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:08,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:08,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:08,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:08,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:08,295 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:08,295 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:08,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:08,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:08,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:08,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:08,314 [526] [DEBUG] [app] Starting request: urn:request:4a54ca53-8bb2-4461-9c84-b5167c08cd94 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:08,314 [526] [DEBUG] [app] Starting request: urn:request:4a54ca53-8bb2-4461-9c84-b5167c08cd94 (/_internal_ping) 2020-03-31 09:00:08,315 [526] [DEBUG] [app] Ending request: urn:request:4a54ca53-8bb2-4461-9c84-b5167c08cd94 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:08,315 [526] [DEBUG] [app] Ending request: urn:request:4a54ca53-8bb2-4461-9c84-b5167c08cd94 (/_internal_ping) 2020-03-31 09:00:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:00:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:08,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:00:08,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:08,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:08,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:08,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:08,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:08,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:08,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:08,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:08,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:08,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:08,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:08,331 [541] [DEBUG] [app] Starting request: urn:request:bbb132b2-1c23-491c-89cf-add594de80b7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:08,331 [541] [DEBUG] [app] Starting request: urn:request:bbb132b2-1c23-491c-89cf-add594de80b7 (/c1/_internal_ping) 2020-03-31 09:00:08,332 [541] [DEBUG] [app] Ending request: urn:request:bbb132b2-1c23-491c-89cf-add594de80b7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:08,332 [541] [DEBUG] [app] Ending request: urn:request:bbb132b2-1c23-491c-89cf-add594de80b7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:08,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:08,333 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:08,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:00:08,333 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:08,334 [526] [DEBUG] [app] Ending request: urn:request:47cf27c9-fe45-4eb9-bf03-4f46de038637 (/health/instance) 2020-03-31 09:00:08,334 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:08,334 [526] [DEBUG] [app] Ending request: urn:request:47cf27c9-fe45-4eb9-bf03-4f46de038637 (/health/instance) 2020-03-31 09:00:08,334 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:08,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:00:08,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:00:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:00:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:21.440707+00:00 (in 9.999093 seconds) 2020-03-31 09:00:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:21 UTC)" (scheduled at 2020-03-31 09:00:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:00:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:21.440707+00:00 (in 9.999093 seconds) 2020-03-31 09:00:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:21 UTC)" (scheduled at 2020-03-31 09:00:11.440707+00:00) 2020-03-31 09:00:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:00:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:00:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:00:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:00:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:00:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:21 UTC)" executed successfully 2020-03-31 09:00:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:00:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:14,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:24.323352+00:00 (in 9.997367 seconds) 2020-03-31 09:00:14,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:24 UTC)" (scheduled at 2020-03-31 09:00:14.323352+00:00) 2020-03-31 09:00:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:00:14,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:24.323352+00:00 (in 9.997367 seconds) 2020-03-31 09:00:14,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:24 UTC)" (scheduled at 2020-03-31 09:00:14.323352+00:00) 2020-03-31 09:00:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:14,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 14, 326966), True, datetime.datetime(2020, 3, 31, 9, 0, 14, 326966), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:00:14,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 14, 326966), True, datetime.datetime(2020, 3, 31, 9, 0, 14, 326966), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:00:14,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:00:14,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:14,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:14,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:00:14,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:14,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:24 UTC)" executed successfully 2020-03-31 09:00:14,446 [527] [DEBUG] [app] Starting request: urn:request:f5d9b743-2c6a-4f0d-ba02-90e6c8cecc7d (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:14,446 [527] [DEBUG] [app] Starting request: urn:request:f5d9b743-2c6a-4f0d-ba02-90e6c8cecc7d (/health/instance) 2020-03-31 09:00:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:14,453 [550] [DEBUG] [app] Starting request: urn:request:bb6e0968-2314-4927-893e-cdd42a582781 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:14,453 [550] [DEBUG] [app] Starting request: urn:request:bb6e0968-2314-4927-893e-cdd42a582781 (/v1/_internal_ping) 2020-03-31 09:00:14,454 [550] [DEBUG] [app] Ending request: urn:request:bb6e0968-2314-4927-893e-cdd42a582781 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:14,454 [550] [DEBUG] [app] Ending request: urn:request:bb6e0968-2314-4927-893e-cdd42a582781 (/v1/_internal_ping) 2020-03-31 09:00:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:00:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:00:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:00:14,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:00:14,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:14,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:14,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:14,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:14,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:14,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:14,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:14,487 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:14,487 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:14,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:14,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:14,493 [518] [DEBUG] [app] Starting request: urn:request:2a38cda3-aba7-4530-9f93-abb48665bede (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:14,493 [518] [DEBUG] [app] Starting request: urn:request:2a38cda3-aba7-4530-9f93-abb48665bede (/_internal_ping) 2020-03-31 09:00:14,494 [518] [DEBUG] [app] Ending request: urn:request:2a38cda3-aba7-4530-9f93-abb48665bede (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:14,494 [518] [DEBUG] [app] Ending request: urn:request:2a38cda3-aba7-4530-9f93-abb48665bede (/_internal_ping) 2020-03-31 09:00:14,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:00:14,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:00:14,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:14,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:14,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:00:14,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:00:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:00:14,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:14,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:14,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:14,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:14,507 [541] [DEBUG] [app] Starting request: urn:request:fcea330b-b382-47d0-87a7-9c4b112dbd65 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:14,507 [541] [DEBUG] [app] Starting request: urn:request:fcea330b-b382-47d0-87a7-9c4b112dbd65 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:14,508 [541] [DEBUG] [app] Ending request: urn:request:fcea330b-b382-47d0-87a7-9c4b112dbd65 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:14,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:14,508 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:00:14,508 [541] [DEBUG] [app] Ending request: urn:request:fcea330b-b382-47d0-87a7-9c4b112dbd65 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:14,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:14,510 [527] [DEBUG] [app] Ending request: urn:request:f5d9b743-2c6a-4f0d-ba02-90e6c8cecc7d (/health/instance) 2020-03-31 09:00:14,510 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:00:14,508 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:14,510 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:00:14,510 [527] [DEBUG] [app] Ending request: urn:request:f5d9b743-2c6a-4f0d-ba02-90e6c8cecc7d (/health/instance) 2020-03-31 09:00:14,510 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:14,510 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:00:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:00:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:33.366791+00:00 (in 18.825675 seconds) 2020-03-31 09:00:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:05:14 UTC)" (scheduled at 2020-03-31 09:00:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:00:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:33.366791+00:00 (in 18.825675 seconds) 2020-03-31 09:00:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:05:14 UTC)" (scheduled at 2020-03-31 09:00:14.539408+00:00) 2020-03-31 09:00:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:00:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), u'exportactionlogs/%']) 2020-03-31 09:00:14,555 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), True, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:00:14,555 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), True, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), 0, u'exportactionlogs/%']) 2020-03-31 09:00:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), True, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:00:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), True, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 0, 14, 541996), u'exportactionlogs/%']) 2020-03-31 09:00:14,564 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:00:14,564 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:14,565 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:05:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:00:14,565 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:05:14 UTC)" executed successfully 2020-03-31 09:00:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:00:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:00:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:14,568 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:00:14,568 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:00:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:00:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:45.064514+00:00 (in 29.998966 seconds) 2020-03-31 09:00:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:45 UTC)" (scheduled at 2020-03-31 09:00:15.064514+00:00) gcworker stdout | 2020-03-31 09:00:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:45.064514+00:00 (in 29.998966 seconds) 2020-03-31 09:00:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:45 UTC)" (scheduled at 2020-03-31 09:00:15.064514+00:00) 2020-03-31 09:00:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435615, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:00:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435615, 1209600, 500, 1, 0]) 2020-03-31 09:00:15,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:00:15,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:00:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:00:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:45 UTC)" executed successfully 2020-03-31 09:00:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:00:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:26.924497+00:00 (in 9.999267 seconds) 2020-03-31 09:00:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:26 UTC)" (scheduled at 2020-03-31 09:00:16.924497+00:00) 2020-03-31 09:00:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:00:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:26.924497+00:00 (in 9.999267 seconds) 2020-03-31 09:00:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:26 UTC)" (scheduled at 2020-03-31 09:00:16.924497+00:00) 2020-03-31 09:00:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 16, 925914), True, datetime.datetime(2020, 3, 31, 9, 0, 16, 925914), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:00:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 16, 925914), True, datetime.datetime(2020, 3, 31, 9, 0, 16, 925914), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:00:16,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:00:16,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:00:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:26 UTC)" executed successfully 2020-03-31 09:00:18,272 [527] [DEBUG] [app] Starting request: urn:request:6ae08b72-5a82-4a8e-bbb2-1cc239f0abf5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:18,272 [527] [DEBUG] [app] Starting request: urn:request:6ae08b72-5a82-4a8e-bbb2-1cc239f0abf5 (/health/instance) 2020-03-31 09:00:18,275 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:18,275 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:18,277 [554] [DEBUG] [app] Starting request: urn:request:4161337a-ce81-47a7-b91d-4c5eb640a324 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:18,277 [554] [DEBUG] [app] Starting request: urn:request:4161337a-ce81-47a7-b91d-4c5eb640a324 (/v1/_internal_ping) 2020-03-31 09:00:18,278 [554] [DEBUG] [app] Ending request: urn:request:4161337a-ce81-47a7-b91d-4c5eb640a324 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:18,278 [554] [DEBUG] [app] Ending request: urn:request:4161337a-ce81-47a7-b91d-4c5eb640a324 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:18,279 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:18,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:00:18,279 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:18,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:18,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:18,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:00:18,282 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:00:18,282 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:18,283 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:18,283 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:18,284 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:18,284 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:18,301 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:18,301 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:18,315 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:18,315 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:18,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:18,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:18,320 [518] [DEBUG] [app] Starting request: urn:request:e849efe6-4b53-4937-9ad8-1a30f6332285 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:18,320 [518] [DEBUG] [app] Starting request: urn:request:e849efe6-4b53-4937-9ad8-1a30f6332285 (/_internal_ping) 2020-03-31 09:00:18,321 [518] [DEBUG] [app] Ending request: urn:request:e849efe6-4b53-4937-9ad8-1a30f6332285 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:18,321 [518] [DEBUG] [app] Ending request: urn:request:e849efe6-4b53-4937-9ad8-1a30f6332285 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:00:18,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:00:18,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:18,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:18,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:18,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:18,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:18,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:18,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:18,334 [537] [DEBUG] [app] Starting request: urn:request:ddbb113c-acdf-40b8-a200-a8c24d1d8a90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:18,334 [537] [DEBUG] [app] Starting request: urn:request:ddbb113c-acdf-40b8-a200-a8c24d1d8a90 (/c1/_internal_ping) 2020-03-31 09:00:18,335 [537] [DEBUG] [app] Ending request: urn:request:ddbb113c-acdf-40b8-a200-a8c24d1d8a90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:18,335 [537] [DEBUG] [app] Ending request: urn:request:ddbb113c-acdf-40b8-a200-a8c24d1d8a90 (/c1/_internal_ping) 2020-03-31 09:00:18,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:18,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:18,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:00:18,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:18,337 [527] [DEBUG] [app] Ending request: urn:request:6ae08b72-5a82-4a8e-bbb2-1cc239f0abf5 (/health/instance) 2020-03-31 09:00:18,337 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:18,337 [527] [DEBUG] [app] Ending request: urn:request:6ae08b72-5a82-4a8e-bbb2-1cc239f0abf5 (/health/instance) 2020-03-31 09:00:18,337 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:18,338 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:00:18,338 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:00:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:00:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:31.440707+00:00 (in 9.999146 seconds) 2020-03-31 09:00:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:31 UTC)" (scheduled at 2020-03-31 09:00:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:00:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:31.440707+00:00 (in 9.999146 seconds) 2020-03-31 09:00:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:31 UTC)" (scheduled at 2020-03-31 09:00:21.440707+00:00) 2020-03-31 09:00:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:00:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:00:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:00:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:00:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:00:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:31 UTC)" executed successfully 2020-03-31 09:00:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:00:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:52.113221+00:00 (in 29.999006 seconds) 2020-03-31 09:00:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:52 UTC)" (scheduled at 2020-03-31 09:00:22.113221+00:00) securityworker stdout | 2020-03-31 09:00:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:52.113221+00:00 (in 29.999006 seconds) 2020-03-31 09:00:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:52 UTC)" (scheduled at 2020-03-31 09:00:22.113221+00:00) 2020-03-31 09:00:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:00:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:00:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:00:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:00:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:00:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:00:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:00:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:00:52 UTC)" executed successfully 2020-03-31 09:00:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:00:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:34.323352+00:00 (in 9.999045 seconds) 2020-03-31 09:00:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:34 UTC)" (scheduled at 2020-03-31 09:00:24.323352+00:00) 2020-03-31 09:00:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:00:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:34.323352+00:00 (in 9.999045 seconds) 2020-03-31 09:00:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:34 UTC)" (scheduled at 2020-03-31 09:00:24.323352+00:00) 2020-03-31 09:00:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 24, 325028), True, datetime.datetime(2020, 3, 31, 9, 0, 24, 325028), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:00:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 24, 325028), True, datetime.datetime(2020, 3, 31, 9, 0, 24, 325028), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:00:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:00:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:00:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:34 UTC)" executed successfully 2020-03-31 09:00:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:00:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:33.330112+00:00 (in 8.918026 seconds) 2020-03-31 09:00:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:24 UTC)" (scheduled at 2020-03-31 09:00:24.410927+00:00) 2020-03-31 09:00:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:00:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:33.330112+00:00 (in 8.918026 seconds) 2020-03-31 09:00:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:24 UTC)" (scheduled at 2020-03-31 09:00:24.410927+00:00) 2020-03-31 09:00:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:24 UTC)" executed successfully 2020-03-31 09:00:24,446 [526] [DEBUG] [app] Starting request: urn:request:aff94c5e-d632-49bf-9b5e-d15df8b47c03 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:24,446 [526] [DEBUG] [app] Starting request: urn:request:aff94c5e-d632-49bf-9b5e-d15df8b47c03 (/health/instance) 2020-03-31 09:00:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:24,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:24,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:24,454 [550] [DEBUG] [app] Starting request: urn:request:f7119c6b-7928-452d-91ed-3d88665c0522 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:24,454 [550] [DEBUG] [app] Starting request: urn:request:f7119c6b-7928-452d-91ed-3d88665c0522 (/v1/_internal_ping) 2020-03-31 09:00:24,455 [550] [DEBUG] [app] Ending request: urn:request:f7119c6b-7928-452d-91ed-3d88665c0522 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:24,455 [550] [DEBUG] [app] Ending request: urn:request:f7119c6b-7928-452d-91ed-3d88665c0522 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:24,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:24,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:00:24,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:24,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:24,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:24,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:24,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:00:24,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:24,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:24,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:24,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:24,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:24,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:24,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:24,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:24,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:24,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:24,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:24,488 [526] [DEBUG] [app] Starting request: urn:request:81596e25-d35e-4979-be2f-40ab17db6acb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:24,488 [526] [DEBUG] [app] Starting request: urn:request:81596e25-d35e-4979-be2f-40ab17db6acb (/_internal_ping) 2020-03-31 09:00:24,488 [526] [DEBUG] [app] Ending request: urn:request:81596e25-d35e-4979-be2f-40ab17db6acb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:24,488 [526] [DEBUG] [app] Ending request: urn:request:81596e25-d35e-4979-be2f-40ab17db6acb (/_internal_ping) 2020-03-31 09:00:24,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:24,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:00:24,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:24,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:00:24,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:24,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:24,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:00:24,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:00:24,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:24,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:24,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:24,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:24,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:24,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:24,502 [541] [DEBUG] [app] Starting request: urn:request:4f232a1d-1c6b-4e8c-b460-cd647aaf5f83 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:24,502 [541] [DEBUG] [app] Starting request: urn:request:4f232a1d-1c6b-4e8c-b460-cd647aaf5f83 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:24,503 [541] [DEBUG] [app] Ending request: urn:request:4f232a1d-1c6b-4e8c-b460-cd647aaf5f83 (/c1/_internal_ping) 2020-03-31 09:00:24,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:24,503 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:24,505 [526] [DEBUG] [app] Ending request: urn:request:aff94c5e-d632-49bf-9b5e-d15df8b47c03 (/health/instance) 2020-03-31 09:00:24,505 [526] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:24,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-verbs stdout | 2020-03-31 09:00:24,503 [541] [DEBUG] [app] Ending request: urn:request:4f232a1d-1c6b-4e8c-b460-cd647aaf5f83 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:24,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:00:24,503 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:24,505 [526] [DEBUG] [app] Ending request: urn:request:aff94c5e-d632-49bf-9b5e-d15df8b47c03 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:24,505 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:24,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:00:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:00:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:36 UTC)" (scheduled at 2020-03-31 09:00:26.924497+00:00) 2020-03-31 09:00:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:00:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:36 UTC)" (scheduled at 2020-03-31 09:00:26.924497+00:00) 2020-03-31 09:00:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 26, 925867), True, datetime.datetime(2020, 3, 31, 9, 0, 26, 925867), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:00:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 26, 925867), True, datetime.datetime(2020, 3, 31, 9, 0, 26, 925867), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:00:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:36.924497+00:00 (in 9.999207 seconds) storagereplication stdout | 2020-03-31 09:00:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:36.924497+00:00 (in 9.999207 seconds) 2020-03-31 09:00:26,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:00:26,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:00:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:36 UTC)" executed successfully 2020-03-31 09:00:28,272 [527] [DEBUG] [app] Starting request: urn:request:d159bd11-71d4-4a2a-aa06-45e944907bef (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:28,272 [527] [DEBUG] [app] Starting request: urn:request:d159bd11-71d4-4a2a-aa06-45e944907bef (/health/instance) 2020-03-31 09:00:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:28,280 [538] [DEBUG] [app] Starting request: urn:request:720cbac8-a695-4637-9492-7161982d16e7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:28,280 [538] [DEBUG] [app] Starting request: urn:request:720cbac8-a695-4637-9492-7161982d16e7 (/v1/_internal_ping) 2020-03-31 09:00:28,281 [538] [DEBUG] [app] Ending request: urn:request:720cbac8-a695-4637-9492-7161982d16e7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:28,281 [538] [DEBUG] [app] Ending request: urn:request:720cbac8-a695-4637-9492-7161982d16e7 (/v1/_internal_ping) 2020-03-31 09:00:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:00:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:28,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:00:28,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:00:28,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:28,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:28,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:28,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:28,318 [527] [DEBUG] [app] Starting request: urn:request:7af733e4-3e03-43e0-890a-6f9e8fa558e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:28,318 [527] [DEBUG] [app] Starting request: urn:request:7af733e4-3e03-43e0-890a-6f9e8fa558e3 (/_internal_ping) 2020-03-31 09:00:28,318 [527] [DEBUG] [app] Ending request: urn:request:7af733e4-3e03-43e0-890a-6f9e8fa558e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:28,318 [527] [DEBUG] [app] Ending request: urn:request:7af733e4-3e03-43e0-890a-6f9e8fa558e3 (/_internal_ping) 2020-03-31 09:00:28,319 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:28,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:00:28,319 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:28,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:00:28,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:28,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:28,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:28,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:00:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:28,330 [541] [DEBUG] [app] Starting request: urn:request:05769e32-39f4-4398-986e-cfe59c12aad6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:28,330 [541] [DEBUG] [app] Starting request: urn:request:05769e32-39f4-4398-986e-cfe59c12aad6 (/c1/_internal_ping) 2020-03-31 09:00:28,331 [541] [DEBUG] [app] Ending request: urn:request:05769e32-39f4-4398-986e-cfe59c12aad6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:28,331 [541] [DEBUG] [app] Ending request: urn:request:05769e32-39f4-4398-986e-cfe59c12aad6 (/c1/_internal_ping) 2020-03-31 09:00:28,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:28,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:28,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:28,334 [527] [DEBUG] [app] Ending request: urn:request:d159bd11-71d4-4a2a-aa06-45e944907bef (/health/instance) 2020-03-31 09:00:28,334 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:00:28,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:28,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:00:28,334 [527] [DEBUG] [app] Ending request: urn:request:d159bd11-71d4-4a2a-aa06-45e944907bef (/health/instance) 2020-03-31 09:00:28,334 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:28,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:00:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:00:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:00.337699+00:00 (in 29.999150 seconds) 2020-03-31 09:00:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:00 UTC)" (scheduled at 2020-03-31 09:00:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:00:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:00.337699+00:00 (in 29.999150 seconds) 2020-03-31 09:00:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:00 UTC)" (scheduled at 2020-03-31 09:00:30.337699+00:00) 2020-03-31 09:00:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 0, 30, 339407), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:00:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 0, 30, 339407), False, 50, 1, 0]) 2020-03-31 09:00:30,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:00:30,354 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:00:30,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:00:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:00:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:00 UTC)" executed successfully 2020-03-31 09:00:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:00:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:41.440707+00:00 (in 9.998769 seconds) 2020-03-31 09:00:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:41 UTC)" (scheduled at 2020-03-31 09:00:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:00:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:41.440707+00:00 (in 9.998769 seconds) 2020-03-31 09:00:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:41 UTC)" (scheduled at 2020-03-31 09:00:31.440707+00:00) 2020-03-31 09:00:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:00:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:00:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:00:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:00:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:00:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:41 UTC)" executed successfully 2020-03-31 09:00:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:00:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:03.330112+00:00 (in 29.999069 seconds) 2020-03-31 09:00:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:03 UTC)" (scheduled at 2020-03-31 09:00:33.330112+00:00) 2020-03-31 09:00:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:00:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:03.330112+00:00 (in 29.999069 seconds) 2020-03-31 09:00:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:03 UTC)" (scheduled at 2020-03-31 09:00:33.330112+00:00) 2020-03-31 09:00:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 33, 331908), True, datetime.datetime(2020, 3, 31, 9, 0, 33, 331908), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:00:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 33, 331908), True, datetime.datetime(2020, 3, 31, 9, 0, 33, 331908), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:00:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:00:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:00:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:03 UTC)" executed successfully 2020-03-31 09:00:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:00:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:33 UTC)" (scheduled at 2020-03-31 09:00:33.366791+00:00) 2020-03-31 09:00:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:00:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:33 UTC)" (scheduled at 2020-03-31 09:00:33.366791+00:00) 2020-03-31 09:00:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:33,369 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:43.539999+00:00 (in 10.172099 seconds) exportactionlogsworker stdout | 2020-03-31 09:00:33,369 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:43.539999+00:00 (in 10.172099 seconds) 2020-03-31 09:00:33,372 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 33, 369230), True, datetime.datetime(2020, 3, 31, 9, 0, 33, 369230), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:00:33,372 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 33, 369230), True, datetime.datetime(2020, 3, 31, 9, 0, 33, 369230), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:00:33,387 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:00:33,387 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:00:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:33 UTC)" executed successfully 2020-03-31 09:00:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:00:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:44.323352+00:00 (in 9.999116 seconds) 2020-03-31 09:00:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:44 UTC)" (scheduled at 2020-03-31 09:00:34.323352+00:00) 2020-03-31 09:00:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:00:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:44.323352+00:00 (in 9.999116 seconds) 2020-03-31 09:00:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:44 UTC)" (scheduled at 2020-03-31 09:00:34.323352+00:00) 2020-03-31 09:00:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 34, 325180), True, datetime.datetime(2020, 3, 31, 9, 0, 34, 325180), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:00:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 34, 325180), True, datetime.datetime(2020, 3, 31, 9, 0, 34, 325180), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:00:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:00:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:00:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:44 UTC)" executed successfully 2020-03-31 09:00:34,447 [527] [DEBUG] [app] Starting request: urn:request:e7e88c23-a477-48af-b320-89156a250f78 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:34,447 [527] [DEBUG] [app] Starting request: urn:request:e7e88c23-a477-48af-b320-89156a250f78 (/health/instance) 2020-03-31 09:00:34,451 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:34,451 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:34,455 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:34,455 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:34,457 [555] [DEBUG] [app] Starting request: urn:request:3c964d59-082a-4702-a4ba-4d7f1177df76 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:34,457 [555] [DEBUG] [app] Starting request: urn:request:3c964d59-082a-4702-a4ba-4d7f1177df76 (/v1/_internal_ping) 2020-03-31 09:00:34,458 [555] [DEBUG] [app] Ending request: urn:request:3c964d59-082a-4702-a4ba-4d7f1177df76 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:34,458 [555] [DEBUG] [app] Ending request: urn:request:3c964d59-082a-4702-a4ba-4d7f1177df76 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:34,459 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:34,459 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:00:34,459 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:34,460 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:34,459 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:34,461 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:34,460 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:00:34,461 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:34,462 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:34,462 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:34,463 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:34,463 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:34,475 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:34,475 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:34,488 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:34,488 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:34,493 [526] [DEBUG] [app] Starting request: urn:request:ba524bfb-1299-4756-bd5c-585fec116b32 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:34,493 [526] [DEBUG] [app] Starting request: urn:request:ba524bfb-1299-4756-bd5c-585fec116b32 (/_internal_ping) 2020-03-31 09:00:34,494 [526] [DEBUG] [app] Ending request: urn:request:ba524bfb-1299-4756-bd5c-585fec116b32 (/_internal_ping) 2020-03-31 09:00:34,494 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:00:34,494 [526] [DEBUG] [app] Ending request: urn:request:ba524bfb-1299-4756-bd5c-585fec116b32 (/_internal_ping) 2020-03-31 09:00:34,494 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:34,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:00:34,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:34,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:34,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:34,509 [537] [DEBUG] [app] Starting request: urn:request:f908451f-f479-4023-9187-62f14440244a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:34,509 [537] [DEBUG] [app] Starting request: urn:request:f908451f-f479-4023-9187-62f14440244a (/c1/_internal_ping) 2020-03-31 09:00:34,509 [537] [DEBUG] [app] Ending request: urn:request:f908451f-f479-4023-9187-62f14440244a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:34,509 [537] [DEBUG] [app] Ending request: urn:request:f908451f-f479-4023-9187-62f14440244a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:34,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:34,510 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:34,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:00:34,510 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:34,512 [527] [DEBUG] [app] Ending request: urn:request:e7e88c23-a477-48af-b320-89156a250f78 (/health/instance) 2020-03-31 09:00:34,512 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:34,512 [527] [DEBUG] [app] Ending request: urn:request:e7e88c23-a477-48af-b320-89156a250f78 (/health/instance) 2020-03-31 09:00:34,512 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:34,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:00:34,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:00:36,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:00:36,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:36,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:38.010934+00:00 (in 1.083050 seconds) 2020-03-31 09:00:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:46 UTC)" (scheduled at 2020-03-31 09:00:36.924497+00:00) 2020-03-31 09:00:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:00:36,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:38.010934+00:00 (in 1.083050 seconds) 2020-03-31 09:00:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:46 UTC)" (scheduled at 2020-03-31 09:00:36.924497+00:00) 2020-03-31 09:00:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:36,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 36, 928773), True, datetime.datetime(2020, 3, 31, 9, 0, 36, 928773), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:00:36,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 36, 928773), True, datetime.datetime(2020, 3, 31, 9, 0, 36, 928773), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:00:36,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:00:36,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:36,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:00:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:36,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:46 UTC)" executed successfully 2020-03-31 09:00:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:00:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:38 UTC)" (scheduled at 2020-03-31 09:00:38.010934+00:00) 2020-03-31 09:00:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:38 UTC)" executed successfully 2020-03-31 09:00:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:46.924497+00:00 (in 8.912630 seconds) storagereplication stdout | 2020-03-31 09:00:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:38 UTC)" (scheduled at 2020-03-31 09:00:38.010934+00:00) 2020-03-31 09:00:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:38 UTC)" executed successfully 2020-03-31 09:00:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:46.924497+00:00 (in 8.912630 seconds) 2020-03-31 09:00:38,273 [527] [DEBUG] [app] Starting request: urn:request:25822b2b-ca6d-46ca-be7e-fc7e6737d6d0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:38,273 [527] [DEBUG] [app] Starting request: urn:request:25822b2b-ca6d-46ca-be7e-fc7e6737d6d0 (/health/instance) 2020-03-31 09:00:38,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:38,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:38,278 [548] [DEBUG] [app] Starting request: urn:request:b3b3e1a9-60a9-4ee7-aa92-dfafaeee6644 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:38,278 [548] [DEBUG] [app] Starting request: urn:request:b3b3e1a9-60a9-4ee7-aa92-dfafaeee6644 (/v1/_internal_ping) 2020-03-31 09:00:38,279 [548] [DEBUG] [app] Ending request: urn:request:b3b3e1a9-60a9-4ee7-aa92-dfafaeee6644 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:38,279 [548] [DEBUG] [app] Ending request: urn:request:b3b3e1a9-60a9-4ee7-aa92-dfafaeee6644 (/v1/_internal_ping) 2020-03-31 09:00:38,280 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:00:38,280 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:00:38,283 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:00:38,283 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:38,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:38,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:38,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:38,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:38,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:38,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:38,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:38,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:38,310 [527] [DEBUG] [app] Starting request: urn:request:580d9c86-0a30-4267-8804-e7e51b2e9b63 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:38,310 [527] [DEBUG] [app] Starting request: urn:request:580d9c86-0a30-4267-8804-e7e51b2e9b63 (/_internal_ping) 2020-03-31 09:00:38,311 [527] [DEBUG] [app] Ending request: urn:request:580d9c86-0a30-4267-8804-e7e51b2e9b63 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:38,311 [527] [DEBUG] [app] Ending request: urn:request:580d9c86-0a30-4267-8804-e7e51b2e9b63 (/_internal_ping) 2020-03-31 09:00:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:00:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:38,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:00:38,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:38,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:00:38,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:00:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:38,325 [541] [DEBUG] [app] Starting request: urn:request:3a01233e-07e9-4b62-aa6d-fce17268a38c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:38,325 [541] [DEBUG] [app] Starting request: urn:request:3a01233e-07e9-4b62-aa6d-fce17268a38c (/c1/_internal_ping) 2020-03-31 09:00:38,326 [541] [DEBUG] [app] Ending request: urn:request:3a01233e-07e9-4b62-aa6d-fce17268a38c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:38,326 [541] [DEBUG] [app] Ending request: urn:request:3a01233e-07e9-4b62-aa6d-fce17268a38c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:38,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:38,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:00:38,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:38,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:38,328 [527] [DEBUG] [app] Ending request: urn:request:25822b2b-ca6d-46ca-be7e-fc7e6737d6d0 (/health/instance) 2020-03-31 09:00:38,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:38,328 [527] [DEBUG] [app] Ending request: urn:request:25822b2b-ca6d-46ca-be7e-fc7e6737d6d0 (/health/instance) 2020-03-31 09:00:38,328 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:00:38,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:00:38,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:00:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:00:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:51.440707+00:00 (in 9.999265 seconds) 2020-03-31 09:00:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:51 UTC)" (scheduled at 2020-03-31 09:00:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:00:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:51.440707+00:00 (in 9.999265 seconds) 2020-03-31 09:00:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:51 UTC)" (scheduled at 2020-03-31 09:00:41.440707+00:00) 2020-03-31 09:00:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:00:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:00:41,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:00:41,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:00:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:00:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:51 UTC)" executed successfully 2020-03-31 09:00:43,544 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:00:43,544 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:43,545 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" (scheduled at 2020-03-31 09:00:43.539999+00:00) 2020-03-31 09:00:43,545 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:43,545 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:00:43,545 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" (scheduled at 2020-03-31 09:00:43.539999+00:00) 2020-03-31 09:00:43,545 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:43,545 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:00:43 UTC)" executed successfully 2020-03-31 09:00:43,546 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:33.366791+00:00 (in 49.821801 seconds) exportactionlogsworker stdout | 2020-03-31 09:00:43,546 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:33.366791+00:00 (in 49.821801 seconds) 2020-03-31 09:00:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:00:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:54 UTC)" (scheduled at 2020-03-31 09:00:44.323352+00:00) 2020-03-31 09:00:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:00:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:54 UTC)" (scheduled at 2020-03-31 09:00:44.323352+00:00) 2020-03-31 09:00:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 44, 325004), True, datetime.datetime(2020, 3, 31, 9, 0, 44, 325004), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:00:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 44, 325004), True, datetime.datetime(2020, 3, 31, 9, 0, 44, 325004), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:00:44,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:49.498966+00:00 (in 5.174738 seconds) notificationworker stdout | 2020-03-31 09:00:44,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:49.498966+00:00 (in 5.174738 seconds) 2020-03-31 09:00:44,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:00:44,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:00:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:54 UTC)" executed successfully 2020-03-31 09:00:44,445 [527] [DEBUG] [app] Starting request: urn:request:a60efc17-ad0c-4017-9e88-ec577d922a67 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:44,445 [527] [DEBUG] [app] Starting request: urn:request:a60efc17-ad0c-4017-9e88-ec577d922a67 (/health/instance) 2020-03-31 09:00:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:44,454 [546] [DEBUG] [app] Starting request: urn:request:2e566bc6-37a1-42db-8fed-816e7e89eae4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:44,454 [546] [DEBUG] [app] Starting request: urn:request:2e566bc6-37a1-42db-8fed-816e7e89eae4 (/v1/_internal_ping) 2020-03-31 09:00:44,455 [546] [DEBUG] [app] Ending request: urn:request:2e566bc6-37a1-42db-8fed-816e7e89eae4 (/v1/_internal_ping) 2020-03-31 09:00:44,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:00:44,455 [546] [DEBUG] [app] Ending request: urn:request:2e566bc6-37a1-42db-8fed-816e7e89eae4 (/v1/_internal_ping) 2020-03-31 09:00:44,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:44,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:00:44,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:44,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:44,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:44,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:44,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:44,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:44,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:44,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:44,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:44,485 [518] [DEBUG] [app] Starting request: urn:request:133c6baf-595a-41d5-9f16-481fd0e3cff4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:44,485 [518] [DEBUG] [app] Starting request: urn:request:133c6baf-595a-41d5-9f16-481fd0e3cff4 (/_internal_ping) 2020-03-31 09:00:44,486 [518] [DEBUG] [app] Ending request: urn:request:133c6baf-595a-41d5-9f16-481fd0e3cff4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:44,486 [518] [DEBUG] [app] Ending request: urn:request:133c6baf-595a-41d5-9f16-481fd0e3cff4 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:00:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:44,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:00:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:44,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:44,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:44,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:00:44,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:00:44,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:00:44,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:44,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:44,500 [536] [DEBUG] [app] Starting request: urn:request:7c40e9f5-f8b5-47ca-b108-c8996e97fc40 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:44,500 [536] [DEBUG] [app] Starting request: urn:request:7c40e9f5-f8b5-47ca-b108-c8996e97fc40 (/c1/_internal_ping) 2020-03-31 09:00:44,501 [536] [DEBUG] [app] Ending request: urn:request:7c40e9f5-f8b5-47ca-b108-c8996e97fc40 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:44,501 [536] [DEBUG] [app] Ending request: urn:request:7c40e9f5-f8b5-47ca-b108-c8996e97fc40 (/c1/_internal_ping) 2020-03-31 09:00:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:44,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:00:44,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:44,504 [527] [DEBUG] [app] Ending request: urn:request:a60efc17-ad0c-4017-9e88-ec577d922a67 (/health/instance) 2020-03-31 09:00:44,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:44,504 [527] [DEBUG] [app] Ending request: urn:request:a60efc17-ad0c-4017-9e88-ec577d922a67 (/health/instance) 2020-03-31 09:00:44,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:00:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:00:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:00:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:15.064514+00:00 (in 29.999131 seconds) 2020-03-31 09:00:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:15 UTC)" (scheduled at 2020-03-31 09:00:45.064514+00:00) gcworker stdout | 2020-03-31 09:00:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:15.064514+00:00 (in 29.999131 seconds) 2020-03-31 09:00:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:15 UTC)" (scheduled at 2020-03-31 09:00:45.064514+00:00) 2020-03-31 09:00:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435645, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:00:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435645, 1209600, 500, 1, 0]) 2020-03-31 09:00:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:00:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:00:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:00:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:15 UTC)" executed successfully 2020-03-31 09:00:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:00:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:56.924497+00:00 (in 9.998124 seconds) 2020-03-31 09:00:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:56 UTC)" (scheduled at 2020-03-31 09:00:46.924497+00:00) 2020-03-31 09:00:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:00:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:56.924497+00:00 (in 9.998124 seconds) 2020-03-31 09:00:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:56 UTC)" (scheduled at 2020-03-31 09:00:46.924497+00:00) 2020-03-31 09:00:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 46, 927040), True, datetime.datetime(2020, 3, 31, 9, 0, 46, 927040), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:00:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 46, 927040), True, datetime.datetime(2020, 3, 31, 9, 0, 46, 927040), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:00:46,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:00:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:00:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:00:56 UTC)" executed successfully 2020-03-31 09:00:48,273 [526] [DEBUG] [app] Starting request: urn:request:2e839944-a6bc-4498-8674-db43e60e5bd8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:48,273 [526] [DEBUG] [app] Starting request: urn:request:2e839944-a6bc-4498-8674-db43e60e5bd8 (/health/instance) 2020-03-31 09:00:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:48,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:48,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:48,281 [538] [DEBUG] [app] Starting request: urn:request:7d2bbd11-b0a6-4387-ac09-a3c24a779b7f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:48,281 [538] [DEBUG] [app] Starting request: urn:request:7d2bbd11-b0a6-4387-ac09-a3c24a779b7f (/v1/_internal_ping) 2020-03-31 09:00:48,282 [538] [DEBUG] [app] Ending request: urn:request:7d2bbd11-b0a6-4387-ac09-a3c24a779b7f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:48,282 [538] [DEBUG] [app] Ending request: urn:request:7d2bbd11-b0a6-4387-ac09-a3c24a779b7f (/v1/_internal_ping) 2020-03-31 09:00:48,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:00:48,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:48,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:00:48,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:48,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:48,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:48,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:48,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:48,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:48,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:48,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:48,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:48,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:48,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:48,313 [526] [DEBUG] [app] Starting request: urn:request:72f4bc3a-c690-427f-acf2-90810b9828e9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:48,313 [526] [DEBUG] [app] Starting request: urn:request:72f4bc3a-c690-427f-acf2-90810b9828e9 (/_internal_ping) 2020-03-31 09:00:48,314 [526] [DEBUG] [app] Ending request: urn:request:72f4bc3a-c690-427f-acf2-90810b9828e9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:48,314 [526] [DEBUG] [app] Ending request: urn:request:72f4bc3a-c690-427f-acf2-90810b9828e9 (/_internal_ping) 2020-03-31 09:00:48,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:48,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:00:48,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:48,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:00:48,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:48,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:48,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:48,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:48,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:48,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:48,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:00:48,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:48,455 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:48,455 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:48,457 [537] [DEBUG] [app] Starting request: urn:request:6c15588b-8fa0-4805-9e7e-9997e54eca04 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:48,457 [537] [DEBUG] [app] Starting request: urn:request:6c15588b-8fa0-4805-9e7e-9997e54eca04 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:48,457 [537] [DEBUG] [app] Ending request: urn:request:6c15588b-8fa0-4805-9e7e-9997e54eca04 (/c1/_internal_ping) 2020-03-31 09:00:48,458 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:48,458 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:00:48,457 [537] [DEBUG] [app] Ending request: urn:request:6c15588b-8fa0-4805-9e7e-9997e54eca04 (/c1/_internal_ping) 2020-03-31 09:00:48,458 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:48,458 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:48,460 [526] [DEBUG] [app] Ending request: urn:request:2e839944-a6bc-4498-8674-db43e60e5bd8 (/health/instance) 2020-03-31 09:00:48,461 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:48,460 [526] [DEBUG] [app] Ending request: urn:request:2e839944-a6bc-4498-8674-db43e60e5bd8 (/health/instance) 2020-03-31 09:00:48,461 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:48,462 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.190 49 0.190) gunicorn-web stdout | 2020-03-31 09:00:48,462 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.190 49 0.190) 2020-03-31 09:00:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:00:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:54.323352+00:00 (in 4.820214 seconds) 2020-03-31 09:00:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:49 UTC)" (scheduled at 2020-03-31 09:00:49.498966+00:00) 2020-03-31 09:00:49,504 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:00:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:54.323352+00:00 (in 4.820214 seconds) 2020-03-31 09:00:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:49 UTC)" (scheduled at 2020-03-31 09:00:49.498966+00:00) 2020-03-31 09:00:49,504 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:49,504 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:00:49,504 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:49 UTC)" executed successfully 2020-03-31 09:00:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:00:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:59.560326+00:00 (in 8.447948 seconds) 2020-03-31 09:00:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:51 UTC)" (scheduled at 2020-03-31 09:00:51.110610+00:00) 2020-03-31 09:00:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:00:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:00:59.560326+00:00 (in 8.447948 seconds) 2020-03-31 09:00:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:51 UTC)" (scheduled at 2020-03-31 09:00:51.110610+00:00) 2020-03-31 09:00:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 51, 113693), True, datetime.datetime(2020, 3, 31, 9, 0, 51, 113693), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:00:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 51, 113693), True, datetime.datetime(2020, 3, 31, 9, 0, 51, 113693), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:00:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:00:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:00:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:51 UTC)" executed successfully 2020-03-31 09:00:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:00:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:01 UTC)" (scheduled at 2020-03-31 09:00:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:00:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:01 UTC)" (scheduled at 2020-03-31 09:00:51.440707+00:00) 2020-03-31 09:00:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:01.440707+00:00 (in 9.999139 seconds) repositoryactioncounter stdout | 2020-03-31 09:00:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:01.440707+00:00 (in 9.999139 seconds) 2020-03-31 09:00:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:00:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:00:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:00:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:00:51,463 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:00:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:00:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:01 UTC)" executed successfully 2020-03-31 09:00:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:00:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:22.113221+00:00 (in 29.998907 seconds) 2020-03-31 09:00:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:22 UTC)" (scheduled at 2020-03-31 09:00:52.113221+00:00) securityworker stdout | 2020-03-31 09:00:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:22.113221+00:00 (in 29.998907 seconds) 2020-03-31 09:00:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:22 UTC)" (scheduled at 2020-03-31 09:00:52.113221+00:00) 2020-03-31 09:00:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:00:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:00:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:00:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:00:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:00:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:00:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:00:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:22 UTC)" executed successfully 2020-03-31 09:00:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:00:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:04.323352+00:00 (in 9.999039 seconds) 2020-03-31 09:00:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:04 UTC)" (scheduled at 2020-03-31 09:00:54.323352+00:00) 2020-03-31 09:00:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:00:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:04.323352+00:00 (in 9.999039 seconds) 2020-03-31 09:00:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:04 UTC)" (scheduled at 2020-03-31 09:00:54.323352+00:00) 2020-03-31 09:00:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 54, 325291), True, datetime.datetime(2020, 3, 31, 9, 0, 54, 325291), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:00:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 54, 325291), True, datetime.datetime(2020, 3, 31, 9, 0, 54, 325291), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:00:54,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:00:54,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:54,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:54,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:00:54,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:54,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:04 UTC)" executed successfully 2020-03-31 09:00:54,446 [518] [DEBUG] [app] Starting request: urn:request:4e43a352-8ee4-45cd-9c5d-70b12752b4f4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:54,446 [518] [DEBUG] [app] Starting request: urn:request:4e43a352-8ee4-45cd-9c5d-70b12752b4f4 (/health/instance) 2020-03-31 09:00:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:54,454 [544] [DEBUG] [app] Starting request: urn:request:8c715ec3-0523-4263-a892-da410f3da1ae (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:54,454 [544] [DEBUG] [app] Starting request: urn:request:8c715ec3-0523-4263-a892-da410f3da1ae (/v1/_internal_ping) 2020-03-31 09:00:54,455 [544] [DEBUG] [app] Ending request: urn:request:8c715ec3-0523-4263-a892-da410f3da1ae (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:54,455 [544] [DEBUG] [app] Ending request: urn:request:8c715ec3-0523-4263-a892-da410f3da1ae (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:54,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:00:54,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:00:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:00:54,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:00:54,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:54,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:54,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:54,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:54,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:54,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:54,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:54,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:54,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:54,489 [518] [DEBUG] [app] Starting request: urn:request:f7183311-af68-4f14-9c16-9e92a8fd2ae7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:54,489 [518] [DEBUG] [app] Starting request: urn:request:f7183311-af68-4f14-9c16-9e92a8fd2ae7 (/_internal_ping) 2020-03-31 09:00:54,490 [518] [DEBUG] [app] Ending request: urn:request:f7183311-af68-4f14-9c16-9e92a8fd2ae7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:54,490 [518] [DEBUG] [app] Ending request: urn:request:f7183311-af68-4f14-9c16-9e92a8fd2ae7 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:00:54,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:00:54,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:00:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:00:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:00:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:00:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:54,504 [537] [DEBUG] [app] Starting request: urn:request:33af8675-3bae-4185-b5a5-ce582da7979a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:54,504 [537] [DEBUG] [app] Starting request: urn:request:33af8675-3bae-4185-b5a5-ce582da7979a (/c1/_internal_ping) 2020-03-31 09:00:54,505 [537] [DEBUG] [app] Ending request: urn:request:33af8675-3bae-4185-b5a5-ce582da7979a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:54,505 [537] [DEBUG] [app] Ending request: urn:request:33af8675-3bae-4185-b5a5-ce582da7979a (/c1/_internal_ping) 2020-03-31 09:00:54,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:54,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:54,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:00:54,507 [518] [DEBUG] [app] Ending request: urn:request:4e43a352-8ee4-45cd-9c5d-70b12752b4f4 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:00:54,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:54,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:54,507 [518] [DEBUG] [app] Ending request: urn:request:4e43a352-8ee4-45cd-9c5d-70b12752b4f4 (/health/instance) 2020-03-31 09:00:54,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:00:54,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:54,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:00:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:00:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:06.924497+00:00 (in 9.999245 seconds) 2020-03-31 09:00:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:06 UTC)" (scheduled at 2020-03-31 09:00:56.924497+00:00) 2020-03-31 09:00:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:00:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:06.924497+00:00 (in 9.999245 seconds) 2020-03-31 09:00:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:06 UTC)" (scheduled at 2020-03-31 09:00:56.924497+00:00) 2020-03-31 09:00:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:00:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 56, 926014), True, datetime.datetime(2020, 3, 31, 9, 0, 56, 926014), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:00:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 0, 56, 926014), True, datetime.datetime(2020, 3, 31, 9, 0, 56, 926014), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:00:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:00:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:00:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:00:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:06 UTC)" executed successfully 2020-03-31 09:00:58,273 [518] [DEBUG] [app] Starting request: urn:request:de353779-1163-429e-96a5-e9571185d9ae (/health/instance) gunicorn-web stdout | 2020-03-31 09:00:58,273 [518] [DEBUG] [app] Starting request: urn:request:de353779-1163-429e-96a5-e9571185d9ae (/health/instance) 2020-03-31 09:00:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:58,278 [538] [DEBUG] [app] Starting request: urn:request:5c43abcb-c77b-4d50-abb1-995bd765dcf7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:58,278 [538] [DEBUG] [app] Starting request: urn:request:5c43abcb-c77b-4d50-abb1-995bd765dcf7 (/v1/_internal_ping) 2020-03-31 09:00:58,279 [538] [DEBUG] [app] Ending request: urn:request:5c43abcb-c77b-4d50-abb1-995bd765dcf7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:00:58,279 [538] [DEBUG] [app] Ending request: urn:request:5c43abcb-c77b-4d50-abb1-995bd765dcf7 (/v1/_internal_ping) 2020-03-31 09:00:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:00:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:00:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:58,281 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:00:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:00:58,281 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:00:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:00:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:00:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:00:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:00:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:58,310 [518] [DEBUG] [app] Starting request: urn:request:95cfd3d3-b7e1-475b-baae-3e3f0e2a9ec9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:58,310 [518] [DEBUG] [app] Starting request: urn:request:95cfd3d3-b7e1-475b-baae-3e3f0e2a9ec9 (/_internal_ping) 2020-03-31 09:00:58,311 [518] [DEBUG] [app] Ending request: urn:request:95cfd3d3-b7e1-475b-baae-3e3f0e2a9ec9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:00:58,311 [518] [DEBUG] [app] Ending request: urn:request:95cfd3d3-b7e1-475b-baae-3e3f0e2a9ec9 (/_internal_ping) 2020-03-31 09:00:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:00:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:00:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:00:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:00:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:00:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:00:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:00:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:00:58,327 [543] [DEBUG] [app] Starting request: urn:request:612ba17b-f68b-4e15-99ad-eeeb1c01e558 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:58,327 [543] [DEBUG] [app] Starting request: urn:request:612ba17b-f68b-4e15-99ad-eeeb1c01e558 (/c1/_internal_ping) 2020-03-31 09:00:58,328 [543] [DEBUG] [app] Ending request: urn:request:612ba17b-f68b-4e15-99ad-eeeb1c01e558 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:00:58,328 [543] [DEBUG] [app] Ending request: urn:request:612ba17b-f68b-4e15-99ad-eeeb1c01e558 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:00:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:00:58,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:00:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:00:58,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:00:58,330 [518] [DEBUG] [app] Ending request: urn:request:de353779-1163-429e-96a5-e9571185d9ae (/health/instance) 2020-03-31 09:00:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:00:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:00:58,330 [518] [DEBUG] [app] Ending request: urn:request:de353779-1163-429e-96a5-e9571185d9ae (/health/instance) 2020-03-31 09:00:58,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:00:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:00:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:00:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:00:59,468 [555] [DEBUG] [app] Starting request: urn:request:6cc3397d-8aa1-460d-8723-70b98b373fbb (/v2/) gunicorn-registry stdout | 2020-03-31 09:00:59,468 [555] [DEBUG] [app] Starting request: urn:request:6cc3397d-8aa1-460d-8723-70b98b373fbb (/v2/) 2020-03-31 09:00:59,469 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {} 2020-03-31 09:00:59,469 [555] [DEBUG] [auth.registry_jwt_auth] No auth header. gunicorn-registry stdout | 2020-03-31 09:00:59,469 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {} 2020-03-31 09:00:59,469 [555] [DEBUG] [auth.registry_jwt_auth] No auth header. 2020-03-31 09:00:59,470 [555] [DEBUG] [app] Ending request: urn:request:6cc3397d-8aa1-460d-8723-70b98b373fbb (/v2/) gunicorn-registry stdout | 2020-03-31 09:00:59,470 [555] [DEBUG] [app] Ending request: urn:request:6cc3397d-8aa1-460d-8723-70b98b373fbb (/v2/) 2020-03-31 09:00:59,470 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:00:59 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:00:59 +0000] "GET /v2/ HTTP/2.0" 401 4 "-" "Go-http-client/2.0" (0.003 63 0.003) gunicorn-registry stdout | 2020-03-31 09:00:59,470 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:00:59 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:00:59 +0000] "GET /v2/ HTTP/2.0" 401 4 "-" "Go-http-client/2.0" (0.003 63 0.003) 2020-03-31 09:00:59,472 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:00:59,472 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:00:59,474 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:00:59,474 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:00:59,474 [555] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:00:59,474 [555] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:00:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:00:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:00:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:51.110610+00:00 (in 51.549094 seconds) 2020-03-31 09:00:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:59 UTC)" (scheduled at 2020-03-31 09:00:59.560326+00:00) 2020-03-31 09:00:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:00:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:51.110610+00:00 (in 51.549094 seconds) 2020-03-31 09:00:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:59 UTC)" (scheduled at 2020-03-31 09:00:59.560326+00:00) 2020-03-31 09:00:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:00:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:01:59 UTC)" executed successfully 2020-03-31 09:00:59,828 [555] [DEBUG] [app] Starting request: urn:request:671a4f19-c52d-4aaf-b543-9e7260e4b995 (/v2/auth) gunicorn-registry stdout | 2020-03-31 09:00:59,828 [555] [DEBUG] [app] Starting request: urn:request:671a4f19-c52d-4aaf-b543-9e7260e4b995 (/v2/auth) 2020-03-31 09:00:59,828 [555] [DEBUG] [auth.basic] Attempt to process basic auth header gunicorn-registry stdout | 2020-03-31 09:00:59,828 [555] [DEBUG] [auth.basic] Attempt to process basic auth header 2020-03-31 09:00:59,835 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', [u'admin', u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:00:59,835 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', [u'admin', u'admin', 1, 0]) 2020-03-31 09:00:59,836 [555] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2020-03-31 09:00:59,836 [555] [DEBUG] [peewee.pool] No connection available in pool. 2020-03-31 09:00:59,847 [555] [DEBUG] [peewee.pool] Created new connection 140043679892496. gunicorn-registry stdout | 2020-03-31 09:00:59,847 [555] [DEBUG] [peewee.pool] Created new connection 140043679892496. 2020-03-31 09:01:00,188 [555] [DEBUG] [auth.credentials] Successfully validated credentials for user admin gunicorn-registry stdout | 2020-03-31 09:01:00,188 [555] [DEBUG] [auth.credentials] Successfully validated credentials for user admin 2020-03-31 09:01:00,189 [555] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [None, None, None, None, , None]) 2020-03-31 09:01:00,189 [555] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:01:00,189 [555] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 8353caae-85ec-4764-8c12-bda3feb072ac gunicorn-registry stdout | 2020-03-31 09:01:00,189 [555] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [None, None, None, None, , None]) 2020-03-31 09:01:00,189 [555] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:01:00,189 [555] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:01:00,190 [555] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com 2020-03-31 09:01:00,190 [555] [DEBUG] [endpoints.v2.v2auth] Scope request: [u'repository:testorg/redhat-operators:pull,push'] gunicorn-registry stdout | 2020-03-31 09:01:00,190 [555] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com 2020-03-31 09:01:00,190 [555] [DEBUG] [endpoints.v2.v2auth] Scope request: [u'repository:testorg/redhat-operators:pull,push'] 2020-03-31 09:01:00,191 [555] [DEBUG] [endpoints.v2.v2auth] Match: (u'testorg/redhat-operators', u'testorg/redhat-operators', u'pull,push') gunicorn-registry stdout | 2020-03-31 09:01:00,191 [555] [DEBUG] [endpoints.v2.v2auth] Match: (u'testorg/redhat-operators', u'testorg/redhat-operators', u'pull,push') 2020-03-31 09:01:00,194 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'testorg', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:01:00,194 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'testorg', 1, 0]) 2020-03-31 09:01:00,202 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'testorg', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:01:00,202 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'testorg', u'redhat-operators', 1, 0]) 2020-03-31 09:01:00,207 [555] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac gunicorn-registry stdout | 2020-03-31 09:01:00,207 [555] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:01:00,207 [555] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username=u'admin', role='admin') 2020-03-31 09:01:00,207 [555] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace=u'admin', role='admin') 2020-03-31 09:01:00,208 [555] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace=u'admin', role='admin') 2020-03-31 09:01:00,208 [555] [DEBUG] [auth.permissions] Adding superuser to user: admin gunicorn-registry stdout | 2020-03-31 09:01:00,207 [555] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username=u'admin', role='admin') 2020-03-31 09:01:00,207 [555] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace=u'admin', role='admin') 2020-03-31 09:01:00,208 [555] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace=u'admin', role='admin') 2020-03-31 09:01:00,208 [555] [DEBUG] [auth.permissions] Adding superuser to user: admin 2020-03-31 09:01:00,210 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-registry stdout | 2020-03-31 09:01:00,210 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) 2020-03-31 09:01:00,216 [555] [DEBUG] [endpoints.v2.v2auth] No permission to create repository testorg/redhat-operators gunicorn-registry stdout | 2020-03-31 09:01:00,216 [555] [DEBUG] [endpoints.v2.v2auth] No permission to create repository testorg/redhat-operators 2020-03-31 09:01:00,216 [555] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac gunicorn-registry stdout | 2020-03-31 09:01:00,216 [555] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:01:00,220 [555] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', [u'redhat-operators', u'testorg', 1, u'redhat-operators', u'testorg', 1]) gunicorn-registry stdout | 2020-03-31 09:01:00,220 [555] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', [u'redhat-operators', u'testorg', 1, u'redhat-operators', u'testorg', 1]) 2020-03-31 09:01:00,228 [555] [DEBUG] [endpoints.v2.v2auth] No permission to pull repository testorg/redhat-operators gunicorn-registry stdout | 2020-03-31 09:01:00,228 [555] [DEBUG] [endpoints.v2.v2auth] No permission to pull repository testorg/redhat-operators 2020-03-31 09:01:00,230 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:00,230 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:00,236 [555] [DEBUG] [app] Ending request: urn:request:671a4f19-c52d-4aaf-b543-9e7260e4b995 (/v2/auth) gunicorn-registry stdout | 2020-03-31 09:01:00,236 [555] [DEBUG] [app] Ending request: urn:request:671a4f19-c52d-4aaf-b543-9e7260e4b995 (/v2/auth) 2020-03-31 09:01:00,236 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:00,237 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:01:00,236 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:00,237 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:01:00,237 [555] [INFO] [gunicorn.access] 10.131.0.7 - admin [31/Mar/2020:09:01:00 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/1.1" 200 1082 "-" "Go-http-client/2.0" 10.131.0.7 () - admin [31/Mar/2020:09:01:00 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/2.0" 200 1082 "-" "Go-http-client/2.0" (0.411 134 0.411) gunicorn-registry stdout | 2020-03-31 09:01:00,237 [555] [INFO] [gunicorn.access] 10.131.0.7 - admin [31/Mar/2020:09:01:00 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/1.1" 200 1082 "-" "Go-http-client/2.0" 2020-03-31 09:01:00,239 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:00,240 [555] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics nginx stdout | 10.131.0.7 () - admin [31/Mar/2020:09:01:00 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/2.0" 200 1082 "-" "Go-http-client/2.0" (0.411 134 0.411) gunicorn-registry stdout | 2020-03-31 09:01:00,239 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:00,240 [555] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:01:00,242 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:00,242 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:00,241 [554] [DEBUG] [app] Starting request: urn:request:25e92884-d6cf-4ec3-9440-2a9b734a5fe8 (/v2/testorg/redhat-operators/blobs/uploads/) 2020-03-31 09:01:00,244 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:00,241 [554] [DEBUG] [app] Starting request: urn:request:25e92884-d6cf-4ec3-9440-2a9b734a5fe8 (/v2/testorg/redhat-operators/blobs/uploads/) 2020-03-31 09:01:00,244 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:00,245 [554] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'testorg', 'repo_name': 'redhat-operators'} 2020-03-31 09:01:00,245 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:00,245 [555] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'login', 'namespace': 'testorg', 'repository': 'redhat-operators'} 2020-03-31 09:01:00,245 [554] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjQ4ODYwLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ1MjYwLCJuYmYiOjE1ODU2NDUyNjAsInN1YiI6ImFkbWluIn0.D3bpZKtnQfvw61P6ImpX-8Lyr8wUmr1Xyx72BjKYKsuZg0vDfnDpx8EMLvEtUGd0-0iREtuTUqTXfR40t9g37Wu3MyjY7bll_0lv2dvX8jJs4ImcYwBPLk_gO93hD-AaiS-0TL0FfeliZ51VQiH40KBVb-VumI_2YwQKdIY0Z6kGTgGlk5l-qDAb6S04Oedb7hluKncXK2QQwt3uGcu-2SbfaWs_74g6nTjqmgN4mM0kCwCdnu9U--4JxYdTUNYFttmJO4VWeFVGNiIc-oaBgBWAWmF-hotJvtB_t7gXkGKA4feHvKKQ1PohXvcuDXOF8IC1C501WtnJr5CG-SprQg gunicorn-registry stdout | 2020-03-31 09:01:00,245 [554] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'testorg', 'repo_name': 'redhat-operators'} 2020-03-31 09:01:00,245 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:00,245 [555] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'login', 'namespace': 'testorg', 'repository': 'redhat-operators'} 2020-03-31 09:01:00,245 [554] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjQ4ODYwLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ1MjYwLCJuYmYiOjE1ODU2NDUyNjAsInN1YiI6ImFkbWluIn0.D3bpZKtnQfvw61P6ImpX-8Lyr8wUmr1Xyx72BjKYKsuZg0vDfnDpx8EMLvEtUGd0-0iREtuTUqTXfR40t9g37Wu3MyjY7bll_0lv2dvX8jJs4ImcYwBPLk_gO93hD-AaiS-0TL0FfeliZ51VQiH40KBVb-VumI_2YwQKdIY0Z6kGTgGlk5l-qDAb6S04Oedb7hluKncXK2QQwt3uGcu-2SbfaWs_74g6nTjqmgN4mM0kCwCdnu9U--4JxYdTUNYFttmJO4VWeFVGNiIc-oaBgBWAWmF-hotJvtB_t7gXkGKA4feHvKKQ1PohXvcuDXOF8IC1C501WtnJr5CG-SprQg 2020-03-31 09:01:00,246 [554] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjQ4ODYwLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ1MjYwLCJuYmYiOjE1ODU2NDUyNjAsInN1YiI6ImFkbWluIn0.D3bpZKtnQfvw61P6ImpX-8Lyr8wUmr1Xyx72BjKYKsuZg0vDfnDpx8EMLvEtUGd0-0iREtuTUqTXfR40t9g37Wu3MyjY7bll_0lv2dvX8jJs4ImcYwBPLk_gO93hD-AaiS-0TL0FfeliZ51VQiH40KBVb-VumI_2YwQKdIY0Z6kGTgGlk5l-qDAb6S04Oedb7hluKncXK2QQwt3uGcu-2SbfaWs_74g6nTjqmgN4mM0kCwCdnu9U--4JxYdTUNYFttmJO4VWeFVGNiIc-oaBgBWAWmF-hotJvtB_t7gXkGKA4feHvKKQ1PohXvcuDXOF8IC1C501WtnJr5CG-SprQg gunicorn-registry stdout | 2020-03-31 09:01:00,246 [554] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjQ4ODYwLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ1MjYwLCJuYmYiOjE1ODU2NDUyNjAsInN1YiI6ImFkbWluIn0.D3bpZKtnQfvw61P6ImpX-8Lyr8wUmr1Xyx72BjKYKsuZg0vDfnDpx8EMLvEtUGd0-0iREtuTUqTXfR40t9g37Wu3MyjY7bll_0lv2dvX8jJs4ImcYwBPLk_gO93hD-AaiS-0TL0FfeliZ51VQiH40KBVb-VumI_2YwQKdIY0Z6kGTgGlk5l-qDAb6S04Oedb7hluKncXK2QQwt3uGcu-2SbfaWs_74g6nTjqmgN4mM0kCwCdnu9U--4JxYdTUNYFttmJO4VWeFVGNiIc-oaBgBWAWmF-hotJvtB_t7gXkGKA4feHvKKQ1PohXvcuDXOF8IC1C501WtnJr5CG-SprQg 2020-03-31 09:01:00,249 [554] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 1, 0, 247297), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 1, 0, 247343), u'quay', None, datetime.datetime(2020, 3, 30, 9, 1, 0, 247367), datetime.datetime(2020, 3, 24, 9, 1, 0, 247385), None]) gunicorn-registry stdout | 2020-03-31 09:01:00,249 [554] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 1, 0, 247297), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 1, 0, 247343), u'quay', None, datetime.datetime(2020, 3, 30, 9, 1, 0, 247367), datetime.datetime(2020, 3, 24, 9, 1, 0, 247385), None]) 2020-03-31 09:01:00,249 [554] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2020-03-31 09:01:00,249 [554] [DEBUG] [peewee.pool] No connection available in pool. 2020-03-31 09:01:00,259 [554] [DEBUG] [peewee.pool] Created new connection 140043679892496. gunicorn-registry stdout | 2020-03-31 09:01:00,259 [554] [DEBUG] [peewee.pool] Created new connection 140043679892496. 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'testorg/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:01:00,275 [554] [DEBUG] [endpoints.v2] Checking permission for repo: testorg/redhat-operators gunicorn-registry stdout | 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:01:00,274 [554] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'testorg/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:01:00,275 [554] [DEBUG] [endpoints.v2] Checking permission for repo: testorg/redhat-operators 2020-03-31 09:01:00,275 [554] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"UNAUTHORIZED","detail":{},"message":"access to the requested resource is not authorized"}]} gunicorn-registry stdout | 2020-03-31 09:01:00,275 [554] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"UNAUTHORIZED","detail":{},"message":"access to the requested resource is not authorized"}]} 2020-03-31 09:01:00,276 [554] [DEBUG] [app] Ending request: urn:request:25e92884-d6cf-4ec3-9440-2a9b734a5fe8 (/v2/testorg/redhat-operators/blobs/uploads/) 2020-03-31 09:01:00,277 [554] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:01:00,276 [554] [DEBUG] [app] Ending request: urn:request:25e92884-d6cf-4ec3-9440-2a9b734a5fe8 (/v2/testorg/redhat-operators/blobs/uploads/) 2020-03-31 09:01:00,277 [554] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:00,277 [554] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:01:00,277 [554] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:01:00 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/1.1" 401 112 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:01:00 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/2.0" 401 112 "-" "Go-http-client/2.0" (0.038 906 0.038) gunicorn-registry stdout | 2020-03-31 09:01:00,277 [554] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:01:00,277 [554] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:01:00 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/1.1" 401 112 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:01:00 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/2.0" 401 112 "-" "Go-http-client/2.0" (0.038 906 0.038) 2020-03-31 09:01:00,279 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:00,279 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:00,280 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:00,280 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:00,281 [554] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:00,281 [554] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:01:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:01:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:30.337699+00:00 (in 29.998953 seconds) 2020-03-31 09:01:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:30 UTC)" (scheduled at 2020-03-31 09:01:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:01:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:30.337699+00:00 (in 29.998953 seconds) 2020-03-31 09:01:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:30 UTC)" (scheduled at 2020-03-31 09:01:00.337699+00:00) 2020-03-31 09:01:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 1, 0, 339639), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:01:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 1, 0, 339639), False, 50, 1, 0]) 2020-03-31 09:01:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:01:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:01:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:01:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:30 UTC)" executed successfully 2020-03-31 09:01:01,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:01:01,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:11.440707+00:00 (in 9.996850 seconds) repositoryactioncounter stdout | 2020-03-31 09:01:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:11.440707+00:00 (in 9.996850 seconds) 2020-03-31 09:01:01,447 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:11 UTC)" (scheduled at 2020-03-31 09:01:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:01:01,447 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:11 UTC)" (scheduled at 2020-03-31 09:01:01.440707+00:00) 2020-03-31 09:01:01,449 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:01:01,449 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:01:01,467 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:01:01,467 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:01:01,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:01,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:01:01,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:01,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:11 UTC)" executed successfully 2020-03-31 09:01:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:01:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:24.410927+00:00 (in 21.079815 seconds) 2020-03-31 09:01:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:33 UTC)" (scheduled at 2020-03-31 09:01:03.330112+00:00) 2020-03-31 09:01:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:01:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:24.410927+00:00 (in 21.079815 seconds) 2020-03-31 09:01:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:33 UTC)" (scheduled at 2020-03-31 09:01:03.330112+00:00) 2020-03-31 09:01:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 3, 331937), True, datetime.datetime(2020, 3, 31, 9, 1, 3, 331937), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:01:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 3, 331937), True, datetime.datetime(2020, 3, 31, 9, 1, 3, 331937), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:01:03,354 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:01:03,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:01:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:33 UTC)" executed successfully 2020-03-31 09:01:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:01:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:14.323352+00:00 (in 9.998982 seconds) 2020-03-31 09:01:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:14 UTC)" (scheduled at 2020-03-31 09:01:04.323352+00:00) 2020-03-31 09:01:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:01:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:14.323352+00:00 (in 9.998982 seconds) 2020-03-31 09:01:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:14 UTC)" (scheduled at 2020-03-31 09:01:04.323352+00:00) 2020-03-31 09:01:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 4, 325447), True, datetime.datetime(2020, 3, 31, 9, 1, 4, 325447), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:01:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 4, 325447), True, datetime.datetime(2020, 3, 31, 9, 1, 4, 325447), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:01:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:01:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:01:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:14 UTC)" executed successfully 2020-03-31 09:01:04,445 [527] [DEBUG] [app] Starting request: urn:request:e7fcb72d-03b0-4d9f-ae99-017949103039 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:04,445 [527] [DEBUG] [app] Starting request: urn:request:e7fcb72d-03b0-4d9f-ae99-017949103039 (/health/instance) 2020-03-31 09:01:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:04,454 [544] [DEBUG] [app] Starting request: urn:request:fc88fa6c-b08a-4e14-92da-56815c87f97b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:04,454 [544] [DEBUG] [app] Starting request: urn:request:fc88fa6c-b08a-4e14-92da-56815c87f97b (/v1/_internal_ping) 2020-03-31 09:01:04,455 [544] [DEBUG] [app] Ending request: urn:request:fc88fa6c-b08a-4e14-92da-56815c87f97b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:04,455 [544] [DEBUG] [app] Ending request: urn:request:fc88fa6c-b08a-4e14-92da-56815c87f97b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:04,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:01:04,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:01:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:01:04,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:04,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:04,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:04,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:04,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:04,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:04,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:04,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:04,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:04,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:04,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:04,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:04,490 [527] [DEBUG] [app] Starting request: urn:request:6717ecf6-c111-4447-9c87-c2f3db9a29bd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:04,490 [527] [DEBUG] [app] Starting request: urn:request:6717ecf6-c111-4447-9c87-c2f3db9a29bd (/_internal_ping) 2020-03-31 09:01:04,491 [527] [DEBUG] [app] Ending request: urn:request:6717ecf6-c111-4447-9c87-c2f3db9a29bd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:04,491 [527] [DEBUG] [app] Ending request: urn:request:6717ecf6-c111-4447-9c87-c2f3db9a29bd (/_internal_ping) 2020-03-31 09:01:04,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:04,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:04,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:04,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:04,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:04,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:04,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:04,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:04,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:04,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:04,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:04,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:04,505 [541] [DEBUG] [app] Starting request: urn:request:59db2a93-ca70-4171-bc12-364e01b4c839 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:04,505 [541] [DEBUG] [app] Starting request: urn:request:59db2a93-ca70-4171-bc12-364e01b4c839 (/c1/_internal_ping) 2020-03-31 09:01:04,506 [541] [DEBUG] [app] Ending request: urn:request:59db2a93-ca70-4171-bc12-364e01b4c839 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:04,506 [541] [DEBUG] [app] Ending request: urn:request:59db2a93-ca70-4171-bc12-364e01b4c839 (/c1/_internal_ping) 2020-03-31 09:01:04,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:04,507 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:04,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:01:04,507 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:04,508 [527] [DEBUG] [app] Ending request: urn:request:e7fcb72d-03b0-4d9f-ae99-017949103039 (/health/instance) 2020-03-31 09:01:04,508 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:04,508 [527] [DEBUG] [app] Ending request: urn:request:e7fcb72d-03b0-4d9f-ae99-017949103039 (/health/instance) 2020-03-31 09:01:04,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:04,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:01:04,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:01:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:01:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:16.924497+00:00 (in 9.999102 seconds) storagereplication stdout | 2020-03-31 09:01:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:16.924497+00:00 (in 9.999102 seconds) 2020-03-31 09:01:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:16 UTC)" (scheduled at 2020-03-31 09:01:06.924497+00:00) 2020-03-31 09:01:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:01:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:16 UTC)" (scheduled at 2020-03-31 09:01:06.924497+00:00) 2020-03-31 09:01:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 6, 927699), True, datetime.datetime(2020, 3, 31, 9, 1, 6, 927699), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:01:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 6, 927699), True, datetime.datetime(2020, 3, 31, 9, 1, 6, 927699), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:01:06,949 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:01:06,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:06,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:06,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:01:06,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:06,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:16 UTC)" executed successfully 2020-03-31 09:01:08,273 [518] [DEBUG] [app] Starting request: urn:request:459f0ec1-e7fa-4bd1-a8e0-43f84fc7e1dc (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:08,273 [518] [DEBUG] [app] Starting request: urn:request:459f0ec1-e7fa-4bd1-a8e0-43f84fc7e1dc (/health/instance) 2020-03-31 09:01:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:08,281 [547] [DEBUG] [app] Starting request: urn:request:7048cde5-27ad-4144-807d-fec13775e582 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:08,281 [547] [DEBUG] [app] Starting request: urn:request:7048cde5-27ad-4144-807d-fec13775e582 (/v1/_internal_ping) 2020-03-31 09:01:08,282 [547] [DEBUG] [app] Ending request: urn:request:7048cde5-27ad-4144-807d-fec13775e582 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:08,282 [547] [DEBUG] [app] Ending request: urn:request:7048cde5-27ad-4144-807d-fec13775e582 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:08,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:01:08,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:08,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:01:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:01:08,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:08,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:08,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:08,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:08,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:08,314 [518] [DEBUG] [app] Starting request: urn:request:d92d2b30-5b9c-4454-83f1-9c495219d6ec (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:08,314 [518] [DEBUG] [app] Starting request: urn:request:d92d2b30-5b9c-4454-83f1-9c495219d6ec (/_internal_ping) 2020-03-31 09:01:08,315 [518] [DEBUG] [app] Ending request: urn:request:d92d2b30-5b9c-4454-83f1-9c495219d6ec (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:08,315 [518] [DEBUG] [app] Ending request: urn:request:d92d2b30-5b9c-4454-83f1-9c495219d6ec (/_internal_ping) 2020-03-31 09:01:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:01:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:08,327 [537] [DEBUG] [app] Starting request: urn:request:4ebbcec0-9d45-4066-b5b6-f7d6dbaf0ff4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:08,327 [537] [DEBUG] [app] Starting request: urn:request:4ebbcec0-9d45-4066-b5b6-f7d6dbaf0ff4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:08,328 [537] [DEBUG] [app] Ending request: urn:request:4ebbcec0-9d45-4066-b5b6-f7d6dbaf0ff4 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:08,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:01:08,328 [537] [DEBUG] [app] Ending request: urn:request:4ebbcec0-9d45-4066-b5b6-f7d6dbaf0ff4 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:08,330 [518] [DEBUG] [app] Ending request: urn:request:459f0ec1-e7fa-4bd1-a8e0-43f84fc7e1dc (/health/instance) 2020-03-31 09:01:08,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:01:08,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:01:08,330 [518] [DEBUG] [app] Ending request: urn:request:459f0ec1-e7fa-4bd1-a8e0-43f84fc7e1dc (/health/instance) 2020-03-31 09:01:08,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:01:11,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:01:11,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:21.440707+00:00 (in 9.997629 seconds) 2020-03-31 09:01:11,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:21 UTC)" (scheduled at 2020-03-31 09:01:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:01:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:21.440707+00:00 (in 9.997629 seconds) 2020-03-31 09:01:11,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:21 UTC)" (scheduled at 2020-03-31 09:01:11.440707+00:00) 2020-03-31 09:01:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:01:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:01:11,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:01:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:01:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:01:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:21 UTC)" executed successfully 2020-03-31 09:01:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:01:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:24.323352+00:00 (in 9.999104 seconds) 2020-03-31 09:01:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:24 UTC)" (scheduled at 2020-03-31 09:01:14.323352+00:00) 2020-03-31 09:01:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:01:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:24.323352+00:00 (in 9.999104 seconds) 2020-03-31 09:01:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:24 UTC)" (scheduled at 2020-03-31 09:01:14.323352+00:00) 2020-03-31 09:01:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 14, 325081), True, datetime.datetime(2020, 3, 31, 9, 1, 14, 325081), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:01:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 14, 325081), True, datetime.datetime(2020, 3, 31, 9, 1, 14, 325081), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:01:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:01:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:01:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:24 UTC)" executed successfully 2020-03-31 09:01:14,446 [527] [DEBUG] [app] Starting request: urn:request:d6116b2d-9d56-44a9-8265-1d08792d8d7c (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:14,446 [527] [DEBUG] [app] Starting request: urn:request:d6116b2d-9d56-44a9-8265-1d08792d8d7c (/health/instance) 2020-03-31 09:01:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:14,454 [555] [DEBUG] [app] Starting request: urn:request:8b00ed7c-417f-4d29-b7b1-c4575d46949b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:14,454 [555] [DEBUG] [app] Starting request: urn:request:8b00ed7c-417f-4d29-b7b1-c4575d46949b (/v1/_internal_ping) 2020-03-31 09:01:14,455 [555] [DEBUG] [app] Ending request: urn:request:8b00ed7c-417f-4d29-b7b1-c4575d46949b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:14,455 [555] [DEBUG] [app] Ending request: urn:request:8b00ed7c-417f-4d29-b7b1-c4575d46949b (/v1/_internal_ping) 2020-03-31 09:01:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:01:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:01:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:01:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:14,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:14,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:14,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:14,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:14,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:14,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:14,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:14,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:14,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:14,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:14,490 [527] [DEBUG] [app] Starting request: urn:request:acbbe6bf-518e-4a27-a10e-c1eec24246e4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:14,490 [527] [DEBUG] [app] Starting request: urn:request:acbbe6bf-518e-4a27-a10e-c1eec24246e4 (/_internal_ping) 2020-03-31 09:01:14,491 [527] [DEBUG] [app] Ending request: urn:request:acbbe6bf-518e-4a27-a10e-c1eec24246e4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:14,491 [527] [DEBUG] [app] Ending request: urn:request:acbbe6bf-518e-4a27-a10e-c1eec24246e4 (/_internal_ping) 2020-03-31 09:01:14,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:01:14,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:01:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:01:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:14,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:14,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:14,507 [541] [DEBUG] [app] Starting request: urn:request:30abf2ad-0cfc-4c30-8ce3-39afe289bb90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:14,507 [541] [DEBUG] [app] Starting request: urn:request:30abf2ad-0cfc-4c30-8ce3-39afe289bb90 (/c1/_internal_ping) 2020-03-31 09:01:14,508 [541] [DEBUG] [app] Ending request: urn:request:30abf2ad-0cfc-4c30-8ce3-39afe289bb90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:14,508 [541] [DEBUG] [app] Ending request: urn:request:30abf2ad-0cfc-4c30-8ce3-39afe289bb90 (/c1/_internal_ping) 2020-03-31 09:01:14,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:14,508 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:14,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:14,510 [527] [DEBUG] [app] Ending request: urn:request:d6116b2d-9d56-44a9-8265-1d08792d8d7c (/health/instance) 2020-03-31 09:01:14,511 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:01:14,508 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:14,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:01:14,510 [527] [DEBUG] [app] Ending request: urn:request:d6116b2d-9d56-44a9-8265-1d08792d8d7c (/health/instance) 2020-03-31 09:01:14,511 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:14,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:01:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:01:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:45.064514+00:00 (in 29.999039 seconds) 2020-03-31 09:01:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:45 UTC)" (scheduled at 2020-03-31 09:01:15.064514+00:00) gcworker stdout | 2020-03-31 09:01:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:45.064514+00:00 (in 29.999039 seconds) 2020-03-31 09:01:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:45 UTC)" (scheduled at 2020-03-31 09:01:15.064514+00:00) 2020-03-31 09:01:15,067 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2020-03-31 09:01:15,067 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) 2020-03-31 09:01:15,082 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435675, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:01:15,082 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435675, 1209600, 500, 1, 0]) 2020-03-31 09:01:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:01:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:01:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:01:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:45 UTC)" executed successfully 2020-03-31 09:01:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:01:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:26.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:01:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:26 UTC)" (scheduled at 2020-03-31 09:01:16.924497+00:00) 2020-03-31 09:01:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 16, 926005), True, datetime.datetime(2020, 3, 31, 9, 1, 16, 926005), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:01:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:26.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:01:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:26 UTC)" (scheduled at 2020-03-31 09:01:16.924497+00:00) 2020-03-31 09:01:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:01:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 16, 926005), True, datetime.datetime(2020, 3, 31, 9, 1, 16, 926005), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:01:16,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:01:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:01:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:26 UTC)" executed successfully 2020-03-31 09:01:18,273 [518] [DEBUG] [app] Starting request: urn:request:a82564d1-5ec6-4413-92d0-af9bc4e32358 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:18,273 [518] [DEBUG] [app] Starting request: urn:request:a82564d1-5ec6-4413-92d0-af9bc4e32358 (/health/instance) 2020-03-31 09:01:18,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:18,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:18,282 [554] [DEBUG] [app] Starting request: urn:request:97f243af-8ced-4366-a1b8-940c92cafbe5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:18,282 [554] [DEBUG] [app] Starting request: urn:request:97f243af-8ced-4366-a1b8-940c92cafbe5 (/v1/_internal_ping) 2020-03-31 09:01:18,283 [554] [DEBUG] [app] Ending request: urn:request:97f243af-8ced-4366-a1b8-940c92cafbe5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:18,283 [554] [DEBUG] [app] Ending request: urn:request:97f243af-8ced-4366-a1b8-940c92cafbe5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:18,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:01:18,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:18,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:01:18,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:01:18,286 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:18,286 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:18,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:18,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:18,288 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:18,288 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:18,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:18,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:18,316 [518] [DEBUG] [app] Starting request: urn:request:ffad0a98-3127-4b01-af27-1fc87a94bac9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:18,316 [518] [DEBUG] [app] Starting request: urn:request:ffad0a98-3127-4b01-af27-1fc87a94bac9 (/_internal_ping) 2020-03-31 09:01:18,316 [518] [DEBUG] [app] Ending request: urn:request:ffad0a98-3127-4b01-af27-1fc87a94bac9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:18,316 [518] [DEBUG] [app] Ending request: urn:request:ffad0a98-3127-4b01-af27-1fc87a94bac9 (/_internal_ping) 2020-03-31 09:01:18,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:18,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:01:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:18,328 [543] [DEBUG] [app] Starting request: urn:request:b2f2feab-5a54-42c4-a6ad-f731c1a67188 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:18,328 [543] [DEBUG] [app] Starting request: urn:request:b2f2feab-5a54-42c4-a6ad-f731c1a67188 (/c1/_internal_ping) 2020-03-31 09:01:18,329 [543] [DEBUG] [app] Ending request: urn:request:b2f2feab-5a54-42c4-a6ad-f731c1a67188 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:18,329 [543] [DEBUG] [app] Ending request: urn:request:b2f2feab-5a54-42c4-a6ad-f731c1a67188 (/c1/_internal_ping) 2020-03-31 09:01:18,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:18,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:18,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:18,332 [518] [DEBUG] [app] Ending request: urn:request:a82564d1-5ec6-4413-92d0-af9bc4e32358 (/health/instance) 2020-03-31 09:01:18,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:01:18,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:18,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:01:18,332 [518] [DEBUG] [app] Ending request: urn:request:a82564d1-5ec6-4413-92d0-af9bc4e32358 (/health/instance) 2020-03-31 09:01:18,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:18,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:01:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:01:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:31.440707+00:00 (in 9.998858 seconds) 2020-03-31 09:01:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:31 UTC)" (scheduled at 2020-03-31 09:01:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:01:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:31.440707+00:00 (in 9.998858 seconds) 2020-03-31 09:01:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:31 UTC)" (scheduled at 2020-03-31 09:01:21.440707+00:00) 2020-03-31 09:01:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:01:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:01:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:01:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:01:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:01:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:31 UTC)" executed successfully 2020-03-31 09:01:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:01:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:52 UTC)" (scheduled at 2020-03-31 09:01:22.113221+00:00) securityworker stdout | 2020-03-31 09:01:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:52 UTC)" (scheduled at 2020-03-31 09:01:22.113221+00:00) 2020-03-31 09:01:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:52.113221+00:00 (in 29.998767 seconds) 2020-03-31 09:01:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:01:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:52.113221+00:00 (in 29.998767 seconds) 2020-03-31 09:01:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:01:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:01:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:01:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:01:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:01:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:01:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:01:52 UTC)" executed successfully 2020-03-31 09:01:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:01:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:34.323352+00:00 (in 9.999251 seconds) 2020-03-31 09:01:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:34 UTC)" (scheduled at 2020-03-31 09:01:24.323352+00:00) 2020-03-31 09:01:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:01:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:34.323352+00:00 (in 9.999251 seconds) 2020-03-31 09:01:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:34 UTC)" (scheduled at 2020-03-31 09:01:24.323352+00:00) 2020-03-31 09:01:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 24, 324901), True, datetime.datetime(2020, 3, 31, 9, 1, 24, 324901), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:01:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 24, 324901), True, datetime.datetime(2020, 3, 31, 9, 1, 24, 324901), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:01:24,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:01:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:01:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:34 UTC)" executed successfully 2020-03-31 09:01:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:01:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:33.330112+00:00 (in 8.918406 seconds) 2020-03-31 09:01:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:24 UTC)" (scheduled at 2020-03-31 09:01:24.410927+00:00) 2020-03-31 09:01:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:01:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:33.330112+00:00 (in 8.918406 seconds) 2020-03-31 09:01:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:24 UTC)" (scheduled at 2020-03-31 09:01:24.410927+00:00) 2020-03-31 09:01:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:24 UTC)" executed successfully 2020-03-31 09:01:24,445 [518] [DEBUG] [app] Starting request: urn:request:c6722159-da94-4c79-bf18-c0028a0c2df7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:24,445 [518] [DEBUG] [app] Starting request: urn:request:c6722159-da94-4c79-bf18-c0028a0c2df7 (/health/instance) 2020-03-31 09:01:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:24,454 [553] [DEBUG] [app] Starting request: urn:request:beb440b8-9f0e-42ce-9db1-75cdf07cbf3b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:24,454 [553] [DEBUG] [app] Starting request: urn:request:beb440b8-9f0e-42ce-9db1-75cdf07cbf3b (/v1/_internal_ping) 2020-03-31 09:01:24,455 [553] [DEBUG] [app] Ending request: urn:request:beb440b8-9f0e-42ce-9db1-75cdf07cbf3b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:24,455 [553] [DEBUG] [app] Ending request: urn:request:beb440b8-9f0e-42ce-9db1-75cdf07cbf3b (/v1/_internal_ping) 2020-03-31 09:01:24,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:01:24,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:24,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:01:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:01:24,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:24,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:24,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:24,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:24,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:24,489 [518] [DEBUG] [app] Starting request: urn:request:b6124e1c-2d6b-4680-a9e6-cb3195b25365 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:24,489 [518] [DEBUG] [app] Starting request: urn:request:b6124e1c-2d6b-4680-a9e6-cb3195b25365 (/_internal_ping) 2020-03-31 09:01:24,490 [518] [DEBUG] [app] Ending request: urn:request:b6124e1c-2d6b-4680-a9e6-cb3195b25365 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:24,490 [518] [DEBUG] [app] Ending request: urn:request:b6124e1c-2d6b-4680-a9e6-cb3195b25365 (/_internal_ping) 2020-03-31 09:01:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:24,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:24,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:24,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:24,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:24,503 [543] [DEBUG] [app] Starting request: urn:request:8609fc73-f705-40e2-b1bc-05058736f437 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:24,503 [543] [DEBUG] [app] Starting request: urn:request:8609fc73-f705-40e2-b1bc-05058736f437 (/c1/_internal_ping) 2020-03-31 09:01:24,503 [543] [DEBUG] [app] Ending request: urn:request:8609fc73-f705-40e2-b1bc-05058736f437 (/c1/_internal_ping) 2020-03-31 09:01:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:24,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:24,506 [518] [DEBUG] [app] Ending request: urn:request:c6722159-da94-4c79-bf18-c0028a0c2df7 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:01:24,503 [543] [DEBUG] [app] Ending request: urn:request:8609fc73-f705-40e2-b1bc-05058736f437 (/c1/_internal_ping) 2020-03-31 09:01:24,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:01:24,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:24,506 [518] [DEBUG] [app] Ending request: urn:request:c6722159-da94-4c79-bf18-c0028a0c2df7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:24,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:01:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:36.924497+00:00 (in 9.999167 seconds) 2020-03-31 09:01:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:36 UTC)" (scheduled at 2020-03-31 09:01:26.924497+00:00) 2020-03-31 09:01:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 26, 926157), True, datetime.datetime(2020, 3, 31, 9, 1, 26, 926157), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:01:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:36.924497+00:00 (in 9.999167 seconds) 2020-03-31 09:01:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:36 UTC)" (scheduled at 2020-03-31 09:01:26.924497+00:00) 2020-03-31 09:01:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 26, 926157), True, datetime.datetime(2020, 3, 31, 9, 1, 26, 926157), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:01:26,949 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:01:26,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:26,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:26,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:01:26,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:26,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:36 UTC)" executed successfully 2020-03-31 09:01:28,274 [518] [DEBUG] [app] Starting request: urn:request:aedf6d15-4e6a-4510-80cf-b032c17836e7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:28,274 [518] [DEBUG] [app] Starting request: urn:request:aedf6d15-4e6a-4510-80cf-b032c17836e7 (/health/instance) 2020-03-31 09:01:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:28,280 [556] [DEBUG] [app] Starting request: urn:request:66ee81d6-fdf0-4ac7-a375-83234ceaab16 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:28,280 [556] [DEBUG] [app] Starting request: urn:request:66ee81d6-fdf0-4ac7-a375-83234ceaab16 (/v1/_internal_ping) 2020-03-31 09:01:28,281 [556] [DEBUG] [app] Ending request: urn:request:66ee81d6-fdf0-4ac7-a375-83234ceaab16 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:28,281 [556] [DEBUG] [app] Ending request: urn:request:66ee81d6-fdf0-4ac7-a375-83234ceaab16 (/v1/_internal_ping) 2020-03-31 09:01:28,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:01:28,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:01:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:01:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:28,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:28,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:28,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:28,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:28,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:28,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:28,313 [518] [DEBUG] [app] Starting request: urn:request:274af2d5-ce79-4d04-88dc-3591f5bce9b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:28,313 [518] [DEBUG] [app] Starting request: urn:request:274af2d5-ce79-4d04-88dc-3591f5bce9b2 (/_internal_ping) 2020-03-31 09:01:28,314 [518] [DEBUG] [app] Ending request: urn:request:274af2d5-ce79-4d04-88dc-3591f5bce9b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:28,314 [518] [DEBUG] [app] Ending request: urn:request:274af2d5-ce79-4d04-88dc-3591f5bce9b2 (/_internal_ping) 2020-03-31 09:01:28,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:28,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:01:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:01:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:01:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:01:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:01:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:28,329 [543] [DEBUG] [app] Starting request: urn:request:21b8ecab-6d8e-4adb-beba-c361ab813934 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:28,329 [543] [DEBUG] [app] Starting request: urn:request:21b8ecab-6d8e-4adb-beba-c361ab813934 (/c1/_internal_ping) 2020-03-31 09:01:28,330 [543] [DEBUG] [app] Ending request: urn:request:21b8ecab-6d8e-4adb-beba-c361ab813934 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:28,330 [543] [DEBUG] [app] Ending request: urn:request:21b8ecab-6d8e-4adb-beba-c361ab813934 (/c1/_internal_ping) 2020-03-31 09:01:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:28,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:01:28,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:28,332 [518] [DEBUG] [app] Ending request: urn:request:aedf6d15-4e6a-4510-80cf-b032c17836e7 (/health/instance) 2020-03-31 09:01:28,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:28,332 [518] [DEBUG] [app] Ending request: urn:request:aedf6d15-4e6a-4510-80cf-b032c17836e7 (/health/instance) 2020-03-31 09:01:28,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:28,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:01:28,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:01:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:01:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:00.337699+00:00 (in 29.998876 seconds) 2020-03-31 09:01:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:00 UTC)" (scheduled at 2020-03-31 09:01:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:01:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:00.337699+00:00 (in 29.998876 seconds) 2020-03-31 09:01:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:00 UTC)" (scheduled at 2020-03-31 09:01:30.337699+00:00) 2020-03-31 09:01:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 1, 30, 339701), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:01:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 1, 30, 339701), False, 50, 1, 0]) 2020-03-31 09:01:30,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:01:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:01:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:01:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:00 UTC)" executed successfully 2020-03-31 09:01:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:01:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:41.440707+00:00 (in 9.999136 seconds) 2020-03-31 09:01:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:41 UTC)" (scheduled at 2020-03-31 09:01:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:01:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:41.440707+00:00 (in 9.999136 seconds) 2020-03-31 09:01:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:41 UTC)" (scheduled at 2020-03-31 09:01:31.440707+00:00) 2020-03-31 09:01:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:01:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:01:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:01:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:01:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:01:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:41 UTC)" executed successfully 2020-03-31 09:01:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:01:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:03.330112+00:00 (in 29.999258 seconds) 2020-03-31 09:01:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:03 UTC)" (scheduled at 2020-03-31 09:01:33.330112+00:00) 2020-03-31 09:01:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:01:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:03.330112+00:00 (in 29.999258 seconds) 2020-03-31 09:01:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:03 UTC)" (scheduled at 2020-03-31 09:01:33.330112+00:00) 2020-03-31 09:01:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 33, 331863), True, datetime.datetime(2020, 3, 31, 9, 1, 33, 331863), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:01:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 33, 331863), True, datetime.datetime(2020, 3, 31, 9, 1, 33, 331863), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:01:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:01:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:01:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:03 UTC)" executed successfully 2020-03-31 09:01:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:01:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:43.539999+00:00 (in 10.171539 seconds) 2020-03-31 09:01:33,369 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:33 UTC)" (scheduled at 2020-03-31 09:01:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:01:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:43.539999+00:00 (in 10.171539 seconds) 2020-03-31 09:01:33,369 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:33 UTC)" (scheduled at 2020-03-31 09:01:33.366791+00:00) 2020-03-31 09:01:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:01:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 33, 369265), True, datetime.datetime(2020, 3, 31, 9, 1, 33, 369265), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:01:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 33, 369265), True, datetime.datetime(2020, 3, 31, 9, 1, 33, 369265), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:01:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:01:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:01:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:33 UTC)" executed successfully 2020-03-31 09:01:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:01:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:44.323352+00:00 (in 9.999236 seconds) 2020-03-31 09:01:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:44 UTC)" (scheduled at 2020-03-31 09:01:34.323352+00:00) 2020-03-31 09:01:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:01:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:44.323352+00:00 (in 9.999236 seconds) 2020-03-31 09:01:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:44 UTC)" (scheduled at 2020-03-31 09:01:34.323352+00:00) 2020-03-31 09:01:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 34, 325023), True, datetime.datetime(2020, 3, 31, 9, 1, 34, 325023), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:01:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 34, 325023), True, datetime.datetime(2020, 3, 31, 9, 1, 34, 325023), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:01:34,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:34,345 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:01:34,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:01:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:44 UTC)" executed successfully 2020-03-31 09:01:34,446 [527] [DEBUG] [app] Starting request: urn:request:f8864ee6-1870-4cbe-80da-eeb27d247106 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:34,446 [527] [DEBUG] [app] Starting request: urn:request:f8864ee6-1870-4cbe-80da-eeb27d247106 (/health/instance) 2020-03-31 09:01:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:34,453 [554] [DEBUG] [app] Starting request: urn:request:7f7c3539-533f-48d9-bdf0-978f6ff69f20 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:34,453 [554] [DEBUG] [app] Starting request: urn:request:7f7c3539-533f-48d9-bdf0-978f6ff69f20 (/v1/_internal_ping) 2020-03-31 09:01:34,454 [554] [DEBUG] [app] Ending request: urn:request:7f7c3539-533f-48d9-bdf0-978f6ff69f20 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:34,454 [554] [DEBUG] [app] Ending request: urn:request:7f7c3539-533f-48d9-bdf0-978f6ff69f20 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:01:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:01:34,457 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:01:34,457 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:34,459 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:34,459 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:34,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:34,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:34,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:34,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:34,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:34,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:34,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:34,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:34,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:34,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:34,484 [527] [DEBUG] [app] Starting request: urn:request:a0a9cc55-415d-451d-b050-a362212e0c7f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:34,484 [527] [DEBUG] [app] Starting request: urn:request:a0a9cc55-415d-451d-b050-a362212e0c7f (/_internal_ping) 2020-03-31 09:01:34,485 [527] [DEBUG] [app] Ending request: urn:request:a0a9cc55-415d-451d-b050-a362212e0c7f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:34,485 [527] [DEBUG] [app] Ending request: urn:request:a0a9cc55-415d-451d-b050-a362212e0c7f (/_internal_ping) 2020-03-31 09:01:34,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:01:34,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:01:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:01:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:34,500 [543] [DEBUG] [app] Starting request: urn:request:694b50e2-b675-4457-9492-a9bb6dd5bb87 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:34,500 [543] [DEBUG] [app] Starting request: urn:request:694b50e2-b675-4457-9492-a9bb6dd5bb87 (/c1/_internal_ping) 2020-03-31 09:01:34,501 [543] [DEBUG] [app] Ending request: urn:request:694b50e2-b675-4457-9492-a9bb6dd5bb87 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:34,501 [543] [DEBUG] [app] Ending request: urn:request:694b50e2-b675-4457-9492-a9bb6dd5bb87 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:34,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:01:34,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:34,503 [527] [DEBUG] [app] Ending request: urn:request:f8864ee6-1870-4cbe-80da-eeb27d247106 (/health/instance) 2020-03-31 09:01:34,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:34,503 [527] [DEBUG] [app] Ending request: urn:request:f8864ee6-1870-4cbe-80da-eeb27d247106 (/health/instance) 2020-03-31 09:01:34,503 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:01:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:01:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:38.010934+00:00 (in 1.085610 seconds) 2020-03-31 09:01:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:46 UTC)" (scheduled at 2020-03-31 09:01:36.924497+00:00) 2020-03-31 09:01:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 36, 926050), True, datetime.datetime(2020, 3, 31, 9, 1, 36, 926050), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:01:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:01:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:38.010934+00:00 (in 1.085610 seconds) 2020-03-31 09:01:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:46 UTC)" (scheduled at 2020-03-31 09:01:36.924497+00:00) 2020-03-31 09:01:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:01:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 36, 926050), True, datetime.datetime(2020, 3, 31, 9, 1, 36, 926050), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:01:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:36,947 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:01:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:01:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:46 UTC)" executed successfully 2020-03-31 09:01:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:01:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:46.924497+00:00 (in 8.912856 seconds) 2020-03-31 09:01:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:38 UTC)" (scheduled at 2020-03-31 09:01:38.010934+00:00) 2020-03-31 09:01:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:01:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:46.924497+00:00 (in 8.912856 seconds) 2020-03-31 09:01:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:38 UTC)" (scheduled at 2020-03-31 09:01:38.010934+00:00) 2020-03-31 09:01:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:38 UTC)" executed successfully 2020-03-31 09:01:38,272 [526] [DEBUG] [app] Starting request: urn:request:8628901d-12b2-45ee-8676-9440c6c3841f (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:38,272 [526] [DEBUG] [app] Starting request: urn:request:8628901d-12b2-45ee-8676-9440c6c3841f (/health/instance) 2020-03-31 09:01:38,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:38,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:38,280 [551] [DEBUG] [app] Starting request: urn:request:d0edcd71-3d61-4fd1-a39f-8be30be588da (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:38,280 [551] [DEBUG] [app] Starting request: urn:request:d0edcd71-3d61-4fd1-a39f-8be30be588da (/v1/_internal_ping) 2020-03-31 09:01:38,281 [551] [DEBUG] [app] Ending request: urn:request:d0edcd71-3d61-4fd1-a39f-8be30be588da (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:38,281 [551] [DEBUG] [app] Ending request: urn:request:d0edcd71-3d61-4fd1-a39f-8be30be588da (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:38,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:01:38,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:38,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:01:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:01:38,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:38,285 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:38,285 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:38,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:38,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:38,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:38,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:38,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:38,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:38,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:38,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:38,313 [526] [DEBUG] [app] Starting request: urn:request:806809c1-91ae-4960-91c4-029979e6ee32 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:38,313 [526] [DEBUG] [app] Starting request: urn:request:806809c1-91ae-4960-91c4-029979e6ee32 (/_internal_ping) 2020-03-31 09:01:38,314 [526] [DEBUG] [app] Ending request: urn:request:806809c1-91ae-4960-91c4-029979e6ee32 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:38,314 [526] [DEBUG] [app] Ending request: urn:request:806809c1-91ae-4960-91c4-029979e6ee32 (/_internal_ping) 2020-03-31 09:01:38,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:38,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:38,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:38,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:38,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:38,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:38,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:38,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:38,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:38,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:38,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:38,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:38,327 [541] [DEBUG] [app] Starting request: urn:request:f3d56f57-b9c9-4f2b-a3bd-148d8fba3068 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:38,327 [541] [DEBUG] [app] Starting request: urn:request:f3d56f57-b9c9-4f2b-a3bd-148d8fba3068 (/c1/_internal_ping) 2020-03-31 09:01:38,328 [541] [DEBUG] [app] Ending request: urn:request:f3d56f57-b9c9-4f2b-a3bd-148d8fba3068 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:38,328 [541] [DEBUG] [app] Ending request: urn:request:f3d56f57-b9c9-4f2b-a3bd-148d8fba3068 (/c1/_internal_ping) 2020-03-31 09:01:38,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:38,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:38,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:01:38,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:38,330 [526] [DEBUG] [app] Ending request: urn:request:8628901d-12b2-45ee-8676-9440c6c3841f (/health/instance) 2020-03-31 09:01:38,331 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:38,330 [526] [DEBUG] [app] Ending request: urn:request:8628901d-12b2-45ee-8676-9440c6c3841f (/health/instance) 2020-03-31 09:01:38,331 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:38,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:01:38,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:01:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:01:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:51.440707+00:00 (in 9.999190 seconds) 2020-03-31 09:01:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:51 UTC)" (scheduled at 2020-03-31 09:01:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:01:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:51.440707+00:00 (in 9.999190 seconds) 2020-03-31 09:01:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:51 UTC)" (scheduled at 2020-03-31 09:01:41.440707+00:00) 2020-03-31 09:01:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:01:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:01:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:01:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:01:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:01:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:51 UTC)" executed successfully 2020-03-31 09:01:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:01:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:33.366791+00:00 (in 49.826047 seconds) 2020-03-31 09:01:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:43 UTC)" (scheduled at 2020-03-31 09:01:43.539999+00:00) 2020-03-31 09:01:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:01:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:33.366791+00:00 (in 49.826047 seconds) 2020-03-31 09:01:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:43 UTC)" (scheduled at 2020-03-31 09:01:43.539999+00:00) 2020-03-31 09:01:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:43 UTC)" executed successfully 2020-03-31 09:01:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:01:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:49.498966+00:00 (in 5.174899 seconds) 2020-03-31 09:01:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:54 UTC)" (scheduled at 2020-03-31 09:01:44.323352+00:00) 2020-03-31 09:01:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:01:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:49.498966+00:00 (in 5.174899 seconds) 2020-03-31 09:01:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:54 UTC)" (scheduled at 2020-03-31 09:01:44.323352+00:00) 2020-03-31 09:01:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 44, 324871), True, datetime.datetime(2020, 3, 31, 9, 1, 44, 324871), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:01:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 44, 324871), True, datetime.datetime(2020, 3, 31, 9, 1, 44, 324871), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:01:44,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:01:44,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:01:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:54 UTC)" executed successfully 2020-03-31 09:01:44,446 [527] [DEBUG] [app] Starting request: urn:request:9d78f088-0c03-4fc9-ae8a-f8f10d1299d0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:44,446 [527] [DEBUG] [app] Starting request: urn:request:9d78f088-0c03-4fc9-ae8a-f8f10d1299d0 (/health/instance) 2020-03-31 09:01:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:44,454 [556] [DEBUG] [app] Starting request: urn:request:5c514d81-95c0-41f1-9e3f-be5bb7998988 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:44,454 [556] [DEBUG] [app] Starting request: urn:request:5c514d81-95c0-41f1-9e3f-be5bb7998988 (/v1/_internal_ping) 2020-03-31 09:01:44,455 [556] [DEBUG] [app] Ending request: urn:request:5c514d81-95c0-41f1-9e3f-be5bb7998988 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:44,455 [556] [DEBUG] [app] Ending request: urn:request:5c514d81-95c0-41f1-9e3f-be5bb7998988 (/v1/_internal_ping) 2020-03-31 09:01:44,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:01:44,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:01:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:01:44,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:44,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:44,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:44,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:44,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:44,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:44,486 [518] [DEBUG] [app] Starting request: urn:request:4c72eedb-d908-4944-afc6-5622c565d93b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:44,486 [518] [DEBUG] [app] Starting request: urn:request:4c72eedb-d908-4944-afc6-5622c565d93b (/_internal_ping) 2020-03-31 09:01:44,487 [518] [DEBUG] [app] Ending request: urn:request:4c72eedb-d908-4944-afc6-5622c565d93b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:44,487 [518] [DEBUG] [app] Ending request: urn:request:4c72eedb-d908-4944-afc6-5622c565d93b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:01:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:01:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:44,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:44,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:44,501 [534] [DEBUG] [app] Starting request: urn:request:db83d0b4-6633-4c44-a828-69eddf5220e2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:44,501 [534] [DEBUG] [app] Starting request: urn:request:db83d0b4-6633-4c44-a828-69eddf5220e2 (/c1/_internal_ping) 2020-03-31 09:01:44,502 [534] [DEBUG] [app] Ending request: urn:request:db83d0b4-6633-4c44-a828-69eddf5220e2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:44,502 [534] [DEBUG] [app] Ending request: urn:request:db83d0b4-6633-4c44-a828-69eddf5220e2 (/c1/_internal_ping) 2020-03-31 09:01:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:44,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:44,505 [527] [DEBUG] [app] Ending request: urn:request:9d78f088-0c03-4fc9-ae8a-f8f10d1299d0 (/health/instance) 2020-03-31 09:01:44,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:01:44,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:44,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:01:44,505 [527] [DEBUG] [app] Ending request: urn:request:9d78f088-0c03-4fc9-ae8a-f8f10d1299d0 (/health/instance) 2020-03-31 09:01:44,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:44,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:01:45,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:01:45,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:15.064514+00:00 (in 29.997830 seconds) 2020-03-31 09:01:45,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:15 UTC)" (scheduled at 2020-03-31 09:01:45.064514+00:00) gcworker stdout | 2020-03-31 09:01:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:15.064514+00:00 (in 29.997830 seconds) 2020-03-31 09:01:45,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:15 UTC)" (scheduled at 2020-03-31 09:01:45.064514+00:00) 2020-03-31 09:01:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435705, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:01:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435705, 1209600, 500, 1, 0]) 2020-03-31 09:01:45,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:01:45,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:01:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:01:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:15 UTC)" executed successfully 2020-03-31 09:01:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:56.924497+00:00 (in 9.999192 seconds) storagereplication stdout | 2020-03-31 09:01:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:56.924497+00:00 (in 9.999192 seconds) 2020-03-31 09:01:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:56 UTC)" (scheduled at 2020-03-31 09:01:46.924497+00:00) 2020-03-31 09:01:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:01:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:56 UTC)" (scheduled at 2020-03-31 09:01:46.924497+00:00) 2020-03-31 09:01:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 46, 926068), True, datetime.datetime(2020, 3, 31, 9, 1, 46, 926068), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:01:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 46, 926068), True, datetime.datetime(2020, 3, 31, 9, 1, 46, 926068), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:01:46,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:01:46,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:46,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:46,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:01:46,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:46,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:01:56 UTC)" executed successfully 2020-03-31 09:01:48,273 [518] [DEBUG] [app] Starting request: urn:request:e3bc59b6-edb6-420a-a823-a4a56cc9eb0e (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:48,273 [518] [DEBUG] [app] Starting request: urn:request:e3bc59b6-edb6-420a-a823-a4a56cc9eb0e (/health/instance) 2020-03-31 09:01:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:48,282 [555] [DEBUG] [app] Starting request: urn:request:87d73396-2d90-4cae-b17b-d207d7e805d6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:48,282 [555] [DEBUG] [app] Starting request: urn:request:87d73396-2d90-4cae-b17b-d207d7e805d6 (/v1/_internal_ping) 2020-03-31 09:01:48,284 [555] [DEBUG] [app] Ending request: urn:request:87d73396-2d90-4cae-b17b-d207d7e805d6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:48,284 [555] [DEBUG] [app] Ending request: urn:request:87d73396-2d90-4cae-b17b-d207d7e805d6 (/v1/_internal_ping) 2020-03-31 09:01:48,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:48,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:01:48,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:48,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:48,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:01:48,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:01:48,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:01:48,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:48,289 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:48,289 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:48,290 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:48,290 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:48,313 [520] [DEBUG] [app] Starting request: urn:request:9b46bec4-aeed-4e09-930f-567e6ae79ca7 (/_internal_ping) 2020-03-31 09:01:48,314 [520] [DEBUG] [app] Ending request: urn:request:9b46bec4-aeed-4e09-930f-567e6ae79ca7 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:48,315 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:48,313 [520] [DEBUG] [app] Starting request: urn:request:9b46bec4-aeed-4e09-930f-567e6ae79ca7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:48,314 [520] [DEBUG] [app] Ending request: urn:request:9b46bec4-aeed-4e09-930f-567e6ae79ca7 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:48,315 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:01:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:01:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:48,327 [534] [DEBUG] [app] Starting request: urn:request:5d4962f9-65bd-4996-a3b5-3c80fa07a387 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:48,327 [534] [DEBUG] [app] Starting request: urn:request:5d4962f9-65bd-4996-a3b5-3c80fa07a387 (/c1/_internal_ping) 2020-03-31 09:01:48,328 [534] [DEBUG] [app] Ending request: urn:request:5d4962f9-65bd-4996-a3b5-3c80fa07a387 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:48,328 [534] [DEBUG] [app] Ending request: urn:request:5d4962f9-65bd-4996-a3b5-3c80fa07a387 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:48,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:48,330 [518] [DEBUG] [app] Ending request: urn:request:e3bc59b6-edb6-420a-a823-a4a56cc9eb0e (/health/instance) 2020-03-31 09:01:48,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:01:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:01:48,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:48,330 [518] [DEBUG] [app] Ending request: urn:request:e3bc59b6-edb6-420a-a823-a4a56cc9eb0e (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:48,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:01:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:01:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:54.323352+00:00 (in 4.823613 seconds) 2020-03-31 09:01:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:49 UTC)" (scheduled at 2020-03-31 09:01:49.498966+00:00) 2020-03-31 09:01:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:01:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:54.323352+00:00 (in 4.823613 seconds) 2020-03-31 09:01:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:49 UTC)" (scheduled at 2020-03-31 09:01:49.498966+00:00) 2020-03-31 09:01:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:49 UTC)" executed successfully 2020-03-31 09:01:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:01:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:59.560326+00:00 (in 8.448839 seconds) 2020-03-31 09:01:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:51 UTC)" (scheduled at 2020-03-31 09:01:51.110610+00:00) 2020-03-31 09:01:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:01:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:01:59.560326+00:00 (in 8.448839 seconds) 2020-03-31 09:01:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:51 UTC)" (scheduled at 2020-03-31 09:01:51.110610+00:00) 2020-03-31 09:01:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 51, 112456), True, datetime.datetime(2020, 3, 31, 9, 1, 51, 112456), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:01:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 51, 112456), True, datetime.datetime(2020, 3, 31, 9, 1, 51, 112456), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:01:51,132 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:01:51,132 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:01:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:51 UTC)" executed successfully 2020-03-31 09:01:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:01:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:01.440707+00:00 (in 9.998839 seconds) 2020-03-31 09:01:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:01 UTC)" (scheduled at 2020-03-31 09:01:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:01:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:01.440707+00:00 (in 9.998839 seconds) 2020-03-31 09:01:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:01 UTC)" (scheduled at 2020-03-31 09:01:51.440707+00:00) 2020-03-31 09:01:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:01:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:01:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:01:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:01:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:01:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:01 UTC)" executed successfully 2020-03-31 09:01:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:01:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:22.113221+00:00 (in 29.998056 seconds) 2020-03-31 09:01:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:22 UTC)" (scheduled at 2020-03-31 09:01:52.113221+00:00) securityworker stdout | 2020-03-31 09:01:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:22.113221+00:00 (in 29.998056 seconds) 2020-03-31 09:01:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:22 UTC)" (scheduled at 2020-03-31 09:01:52.113221+00:00) 2020-03-31 09:01:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:01:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:01:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:01:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:01:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:01:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:01:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:01:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:22 UTC)" executed successfully 2020-03-31 09:01:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:01:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:04.323352+00:00 (in 9.999038 seconds) 2020-03-31 09:01:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:04 UTC)" (scheduled at 2020-03-31 09:01:54.323352+00:00) 2020-03-31 09:01:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:01:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:04.323352+00:00 (in 9.999038 seconds) 2020-03-31 09:01:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:04 UTC)" (scheduled at 2020-03-31 09:01:54.323352+00:00) 2020-03-31 09:01:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 54, 325093), True, datetime.datetime(2020, 3, 31, 9, 1, 54, 325093), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:01:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 54, 325093), True, datetime.datetime(2020, 3, 31, 9, 1, 54, 325093), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:01:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:01:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:01:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:04 UTC)" executed successfully 2020-03-31 09:01:54,446 [527] [DEBUG] [app] Starting request: urn:request:df891456-92ec-4c19-bf54-df1103261dc9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:54,446 [527] [DEBUG] [app] Starting request: urn:request:df891456-92ec-4c19-bf54-df1103261dc9 (/health/instance) 2020-03-31 09:01:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:54,454 [549] [DEBUG] [app] Starting request: urn:request:4ac6b967-6353-40a8-9262-9744ac46bd11 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:54,454 [549] [DEBUG] [app] Starting request: urn:request:4ac6b967-6353-40a8-9262-9744ac46bd11 (/v1/_internal_ping) 2020-03-31 09:01:54,455 [549] [DEBUG] [app] Ending request: urn:request:4ac6b967-6353-40a8-9262-9744ac46bd11 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:54,455 [549] [DEBUG] [app] Ending request: urn:request:4ac6b967-6353-40a8-9262-9744ac46bd11 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:54,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:01:54,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:01:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:01:54,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:54,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:54,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:54,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:54,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:54,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:54,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:54,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:54,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:54,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:54,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:54,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:54,489 [527] [DEBUG] [app] Starting request: urn:request:5abd8eec-85a3-4ce3-869a-b4dc167df8fb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:54,489 [527] [DEBUG] [app] Starting request: urn:request:5abd8eec-85a3-4ce3-869a-b4dc167df8fb (/_internal_ping) 2020-03-31 09:01:54,490 [527] [DEBUG] [app] Ending request: urn:request:5abd8eec-85a3-4ce3-869a-b4dc167df8fb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:54,490 [527] [DEBUG] [app] Ending request: urn:request:5abd8eec-85a3-4ce3-869a-b4dc167df8fb (/_internal_ping) 2020-03-31 09:01:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:01:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:01:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:01:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:54,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:54,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:54,505 [537] [DEBUG] [app] Starting request: urn:request:76b28e5e-9dfd-455c-bd9b-9b1a7045db64 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:54,505 [537] [DEBUG] [app] Starting request: urn:request:76b28e5e-9dfd-455c-bd9b-9b1a7045db64 (/c1/_internal_ping) 2020-03-31 09:01:54,506 [537] [DEBUG] [app] Ending request: urn:request:76b28e5e-9dfd-455c-bd9b-9b1a7045db64 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:54,506 [537] [DEBUG] [app] Ending request: urn:request:76b28e5e-9dfd-455c-bd9b-9b1a7045db64 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:01:54,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:54,507 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:54,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:01:54,507 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:54,508 [527] [DEBUG] [app] Ending request: urn:request:df891456-92ec-4c19-bf54-df1103261dc9 (/health/instance) 2020-03-31 09:01:54,508 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:54,508 [527] [DEBUG] [app] Ending request: urn:request:df891456-92ec-4c19-bf54-df1103261dc9 (/health/instance) 2020-03-31 09:01:54,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:54,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:01:54,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:01:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:01:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:03.010318+00:00 (in 6.085016 seconds) 2020-03-31 09:01:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:06 UTC)" (scheduled at 2020-03-31 09:01:56.924497+00:00) 2020-03-31 09:01:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:01:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:03.010318+00:00 (in 6.085016 seconds) 2020-03-31 09:01:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:06 UTC)" (scheduled at 2020-03-31 09:01:56.924497+00:00) 2020-03-31 09:01:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:01:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 56, 926512), True, datetime.datetime(2020, 3, 31, 9, 1, 56, 926512), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:01:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 1, 56, 926512), True, datetime.datetime(2020, 3, 31, 9, 1, 56, 926512), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:01:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:01:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:01:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:01:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:06 UTC)" executed successfully 2020-03-31 09:01:58,274 [525] [DEBUG] [app] Starting request: urn:request:1a3fe694-8a7d-4e00-82fb-9fac38249983 (/health/instance) gunicorn-web stdout | 2020-03-31 09:01:58,274 [525] [DEBUG] [app] Starting request: urn:request:1a3fe694-8a7d-4e00-82fb-9fac38249983 (/health/instance) 2020-03-31 09:01:58,276 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:58,276 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:58,280 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:58,280 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:58,282 [555] [DEBUG] [app] Starting request: urn:request:753e3b78-2f0f-4bba-8a4b-08fc4f825fab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:58,282 [555] [DEBUG] [app] Starting request: urn:request:753e3b78-2f0f-4bba-8a4b-08fc4f825fab (/v1/_internal_ping) 2020-03-31 09:01:58,283 [555] [DEBUG] [app] Ending request: urn:request:753e3b78-2f0f-4bba-8a4b-08fc4f825fab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:01:58,283 [555] [DEBUG] [app] Ending request: urn:request:753e3b78-2f0f-4bba-8a4b-08fc4f825fab (/v1/_internal_ping) 2020-03-31 09:01:58,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:01:58,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:58,284 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:01:58,284 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:58,285 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:01:58,285 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:01:58,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:01:58,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:01:58,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:01:58,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:01:58,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:01:58,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:01:58,295 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:01:58,295 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:01:58,308 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:01:58,308 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:01:58,312 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:58,312 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:58,313 [525] [DEBUG] [app] Starting request: urn:request:64b941f5-098b-4e5b-b10c-2abeb01b41b8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:58,313 [525] [DEBUG] [app] Starting request: urn:request:64b941f5-098b-4e5b-b10c-2abeb01b41b8 (/_internal_ping) 2020-03-31 09:01:58,314 [525] [DEBUG] [app] Ending request: urn:request:64b941f5-098b-4e5b-b10c-2abeb01b41b8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:01:58,314 [525] [DEBUG] [app] Ending request: urn:request:64b941f5-098b-4e5b-b10c-2abeb01b41b8 (/_internal_ping) 2020-03-31 09:01:58,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:58,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:58,315 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:01:58,315 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:01:58,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:58,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:58,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:01:58,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:01:58,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:58,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:01:58,322 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:01:58,322 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:01:58,325 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:01:58,325 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:01:58,326 [541] [DEBUG] [app] Starting request: urn:request:c3a0014e-5291-49b5-915b-68710dff3cd0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:58,326 [541] [DEBUG] [app] Starting request: urn:request:c3a0014e-5291-49b5-915b-68710dff3cd0 (/c1/_internal_ping) 2020-03-31 09:01:58,328 [541] [DEBUG] [app] Ending request: urn:request:c3a0014e-5291-49b5-915b-68710dff3cd0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:01:58,328 [541] [DEBUG] [app] Ending request: urn:request:c3a0014e-5291-49b5-915b-68710dff3cd0 (/c1/_internal_ping) 2020-03-31 09:01:58,328 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:01:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:01:58,328 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:01:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:01:58,330 [525] [DEBUG] [app] Ending request: urn:request:1a3fe694-8a7d-4e00-82fb-9fac38249983 (/health/instance) 2020-03-31 09:01:58,330 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:01:58,330 [525] [DEBUG] [app] Ending request: urn:request:1a3fe694-8a7d-4e00-82fb-9fac38249983 (/health/instance) 2020-03-31 09:01:58,330 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:01:58,331 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:01:58,331 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:01:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:01:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:01:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:01:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:01:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:51.110610+00:00 (in 51.549456 seconds) 2020-03-31 09:01:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:59 UTC)" (scheduled at 2020-03-31 09:01:59.560326+00:00) 2020-03-31 09:01:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:01:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:51.110610+00:00 (in 51.549456 seconds) 2020-03-31 09:01:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:59 UTC)" (scheduled at 2020-03-31 09:01:59.560326+00:00) 2020-03-31 09:01:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:01:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:59 UTC)" executed successfully 2020-03-31 09:02:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:02:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:30.337699+00:00 (in 29.999113 seconds) 2020-03-31 09:02:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:30 UTC)" (scheduled at 2020-03-31 09:02:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:02:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:30.337699+00:00 (in 29.999113 seconds) 2020-03-31 09:02:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:30 UTC)" (scheduled at 2020-03-31 09:02:00.337699+00:00) 2020-03-31 09:02:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 2, 0, 339569), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:02:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 2, 0, 339569), False, 50, 1, 0]) 2020-03-31 09:02:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:02:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:02:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:02:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:30 UTC)" executed successfully 2020-03-31 09:02:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:02:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:11.440707+00:00 (in 9.999161 seconds) 2020-03-31 09:02:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:11 UTC)" (scheduled at 2020-03-31 09:02:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:02:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:11.440707+00:00 (in 9.999161 seconds) 2020-03-31 09:02:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:11 UTC)" (scheduled at 2020-03-31 09:02:01.440707+00:00) 2020-03-31 09:02:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:02:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:02:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:02:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:02:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:02:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:11 UTC)" executed successfully 2020-03-31 09:02:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:02:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:06.924497+00:00 (in 3.913439 seconds) 2020-03-31 09:02:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:03 UTC)" (scheduled at 2020-03-31 09:02:03.010318+00:00) storagereplication stdout | 2020-03-31 09:02:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:06.924497+00:00 (in 3.913439 seconds) 2020-03-31 09:02:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:03 UTC)" (scheduled at 2020-03-31 09:02:03.010318+00:00) 2020-03-31 09:02:03,012 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:02:03,012 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), u'imagestoragereplication/%']) 2020-03-31 09:02:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), True, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:02:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), True, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), 0, u'imagestoragereplication/%']) 2020-03-31 09:02:03,031 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), True, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:02:03,031 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), True, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 2, 3, 11922), u'imagestoragereplication/%']) 2020-03-31 09:02:03,036 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:02:03,036 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:03,036 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:03,036 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:03 UTC)" executed successfully 2020-03-31 09:02:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:02:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:02:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 09:02:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:02:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:02:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:24.410927+00:00 (in 21.079985 seconds) 2020-03-31 09:02:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:33 UTC)" (scheduled at 2020-03-31 09:02:03.330112+00:00) 2020-03-31 09:02:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:02:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:24.410927+00:00 (in 21.079985 seconds) 2020-03-31 09:02:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:33 UTC)" (scheduled at 2020-03-31 09:02:03.330112+00:00) 2020-03-31 09:02:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 3, 331896), True, datetime.datetime(2020, 3, 31, 9, 2, 3, 331896), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:02:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 3, 331896), True, datetime.datetime(2020, 3, 31, 9, 2, 3, 331896), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:02:03,348 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:02:03,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:03,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:02:03,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:33 UTC)" executed successfully 2020-03-31 09:02:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:14.323352+00:00 (in 9.999178 seconds) 2020-03-31 09:02:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:14 UTC)" (scheduled at 2020-03-31 09:02:04.323352+00:00) 2020-03-31 09:02:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:02:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:14.323352+00:00 (in 9.999178 seconds) 2020-03-31 09:02:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:14 UTC)" (scheduled at 2020-03-31 09:02:04.323352+00:00) 2020-03-31 09:02:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 4, 325156), True, datetime.datetime(2020, 3, 31, 9, 2, 4, 325156), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:02:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 4, 325156), True, datetime.datetime(2020, 3, 31, 9, 2, 4, 325156), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:02:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:02:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:14 UTC)" executed successfully 2020-03-31 09:02:04,446 [518] [DEBUG] [app] Starting request: urn:request:ada881ff-27d9-4dda-8128-5e7f51e3edeb (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:04,446 [518] [DEBUG] [app] Starting request: urn:request:ada881ff-27d9-4dda-8128-5e7f51e3edeb (/health/instance) 2020-03-31 09:02:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:04,458 [540] [DEBUG] [app] Starting request: urn:request:4b8c1fff-4ca2-4867-8121-414de47c9123 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:04,458 [540] [DEBUG] [app] Starting request: urn:request:4b8c1fff-4ca2-4867-8121-414de47c9123 (/v1/_internal_ping) 2020-03-31 09:02:04,461 [540] [DEBUG] [app] Ending request: urn:request:4b8c1fff-4ca2-4867-8121-414de47c9123 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:04,461 [540] [DEBUG] [app] Ending request: urn:request:4b8c1fff-4ca2-4867-8121-414de47c9123 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.008) 2020-03-31 09:02:04,463 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:04,463 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:02:04,463 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:04,464 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:04,463 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:04,464 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:04,469 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:02:04,469 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:04,474 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:04,474 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:04,475 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:04,475 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:04,476 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:04,476 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:04,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:04,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:04,494 [527] [DEBUG] [app] Starting request: urn:request:d08cf64a-3b03-4f51-9216-6a24aea32683 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:04,494 [527] [DEBUG] [app] Starting request: urn:request:d08cf64a-3b03-4f51-9216-6a24aea32683 (/_internal_ping) 2020-03-31 09:02:04,495 [527] [DEBUG] [app] Ending request: urn:request:d08cf64a-3b03-4f51-9216-6a24aea32683 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:04,495 [527] [DEBUG] [app] Ending request: urn:request:d08cf64a-3b03-4f51-9216-6a24aea32683 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:02:04,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:04,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:02:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:02:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:04,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:04,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:04,510 [532] [DEBUG] [app] Starting request: urn:request:a75a59fc-85f9-4724-94b3-8b03751a89e5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:04,510 [532] [DEBUG] [app] Starting request: urn:request:a75a59fc-85f9-4724-94b3-8b03751a89e5 (/c1/_internal_ping) 2020-03-31 09:02:04,510 [532] [DEBUG] [app] Ending request: urn:request:a75a59fc-85f9-4724-94b3-8b03751a89e5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:04,510 [532] [DEBUG] [app] Ending request: urn:request:a75a59fc-85f9-4724-94b3-8b03751a89e5 (/c1/_internal_ping) 2020-03-31 09:02:04,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:04,511 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:04,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:02:04,511 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:04,513 [518] [DEBUG] [app] Ending request: urn:request:ada881ff-27d9-4dda-8128-5e7f51e3edeb (/health/instance) 2020-03-31 09:02:04,513 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:04,513 [518] [DEBUG] [app] Ending request: urn:request:ada881ff-27d9-4dda-8128-5e7f51e3edeb (/health/instance) 2020-03-31 09:02:04,513 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:04,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:02:04,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:02:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:02:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:16.924497+00:00 (in 9.998861 seconds) 2020-03-31 09:02:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:16 UTC)" (scheduled at 2020-03-31 09:02:06.924497+00:00) 2020-03-31 09:02:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:02:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:16.924497+00:00 (in 9.998861 seconds) 2020-03-31 09:02:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:16 UTC)" (scheduled at 2020-03-31 09:02:06.924497+00:00) 2020-03-31 09:02:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 6, 926605), True, datetime.datetime(2020, 3, 31, 9, 2, 6, 926605), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:02:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 6, 926605), True, datetime.datetime(2020, 3, 31, 9, 2, 6, 926605), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:02:07,106 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:02:07,106 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:07,106 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:07,106 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:07,106 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:07,106 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:16 UTC)" executed successfully 2020-03-31 09:02:08,273 [526] [DEBUG] [app] Starting request: urn:request:9f45d77f-bc24-4f31-b9fb-30453aff1a31 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:08,273 [526] [DEBUG] [app] Starting request: urn:request:9f45d77f-bc24-4f31-b9fb-30453aff1a31 (/health/instance) 2020-03-31 09:02:08,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:08,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:08,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:08,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:08,281 [554] [DEBUG] [app] Starting request: urn:request:720f074b-07ee-4d37-9aa4-03c3b5c11892 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:08,281 [554] [DEBUG] [app] Starting request: urn:request:720f074b-07ee-4d37-9aa4-03c3b5c11892 (/v1/_internal_ping) 2020-03-31 09:02:08,282 [554] [DEBUG] [app] Ending request: urn:request:720f074b-07ee-4d37-9aa4-03c3b5c11892 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:08,282 [554] [DEBUG] [app] Ending request: urn:request:720f074b-07ee-4d37-9aa4-03c3b5c11892 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:08,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:08,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:02:08,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:08,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:08,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:08,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:08,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:02:08,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:08,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:08,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:08,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:08,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:08,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:08,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:08,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:08,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:08,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:08,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:08,313 [526] [DEBUG] [app] Starting request: urn:request:b2a82bef-8d05-4a6f-82c6-b35e68d9adb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:08,313 [526] [DEBUG] [app] Starting request: urn:request:b2a82bef-8d05-4a6f-82c6-b35e68d9adb2 (/_internal_ping) 2020-03-31 09:02:08,314 [526] [DEBUG] [app] Ending request: urn:request:b2a82bef-8d05-4a6f-82c6-b35e68d9adb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:08,314 [526] [DEBUG] [app] Ending request: urn:request:b2a82bef-8d05-4a6f-82c6-b35e68d9adb2 (/_internal_ping) 2020-03-31 09:02:08,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:02:08,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:08,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:02:08,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:08,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:08,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:08,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:02:08,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:08,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:08,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:02:08,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:08,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:08,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:08,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:08,326 [543] [DEBUG] [app] Starting request: urn:request:32f2e2c3-82ec-4500-b278-0334cef4184f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:08,326 [543] [DEBUG] [app] Starting request: urn:request:32f2e2c3-82ec-4500-b278-0334cef4184f (/c1/_internal_ping) 2020-03-31 09:02:08,327 [543] [DEBUG] [app] Ending request: urn:request:32f2e2c3-82ec-4500-b278-0334cef4184f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:08,327 [543] [DEBUG] [app] Ending request: urn:request:32f2e2c3-82ec-4500-b278-0334cef4184f (/c1/_internal_ping) 2020-03-31 09:02:08,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:08,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:08,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:08,329 [526] [DEBUG] [app] Ending request: urn:request:9f45d77f-bc24-4f31-b9fb-30453aff1a31 (/health/instance) 2020-03-31 09:02:08,329 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:02:08,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:08,330 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:02:08,329 [526] [DEBUG] [app] Ending request: urn:request:9f45d77f-bc24-4f31-b9fb-30453aff1a31 (/health/instance) 2020-03-31 09:02:08,329 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:08,330 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:02:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:02:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:21.440707+00:00 (in 9.999176 seconds) 2020-03-31 09:02:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:21 UTC)" (scheduled at 2020-03-31 09:02:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:02:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:21.440707+00:00 (in 9.999176 seconds) 2020-03-31 09:02:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:21 UTC)" (scheduled at 2020-03-31 09:02:11.440707+00:00) 2020-03-31 09:02:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:02:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:02:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:02:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:02:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:02:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:21 UTC)" executed successfully 2020-03-31 09:02:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:24.323352+00:00 (in 9.998878 seconds) 2020-03-31 09:02:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:24 UTC)" (scheduled at 2020-03-31 09:02:14.323352+00:00) 2020-03-31 09:02:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:02:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:24.323352+00:00 (in 9.998878 seconds) 2020-03-31 09:02:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:24 UTC)" (scheduled at 2020-03-31 09:02:14.323352+00:00) 2020-03-31 09:02:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 14, 325132), True, datetime.datetime(2020, 3, 31, 9, 2, 14, 325132), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:02:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 14, 325132), True, datetime.datetime(2020, 3, 31, 9, 2, 14, 325132), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:02:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:02:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:24 UTC)" executed successfully 2020-03-31 09:02:14,446 [518] [DEBUG] [app] Starting request: urn:request:842afc05-1f65-492f-bbda-ece1076b9b14 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:14,446 [518] [DEBUG] [app] Starting request: urn:request:842afc05-1f65-492f-bbda-ece1076b9b14 (/health/instance) 2020-03-31 09:02:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:14,455 [552] [DEBUG] [app] Starting request: urn:request:21edf935-e611-4ad3-b1f3-838d96ab963f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:14,455 [552] [DEBUG] [app] Starting request: urn:request:21edf935-e611-4ad3-b1f3-838d96ab963f (/v1/_internal_ping) 2020-03-31 09:02:14,456 [552] [DEBUG] [app] Ending request: urn:request:21edf935-e611-4ad3-b1f3-838d96ab963f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:14,456 [552] [DEBUG] [app] Ending request: urn:request:21edf935-e611-4ad3-b1f3-838d96ab963f (/v1/_internal_ping) 2020-03-31 09:02:14,457 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:02:14,457 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:02:14,460 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:02:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:02:14,460 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:14,461 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:14,461 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:14,462 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:14,462 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:14,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:14,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:14,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:14,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:14,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:14,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:14,492 [527] [DEBUG] [app] Starting request: urn:request:102b5dbc-caf8-49fb-a2d6-c58852df5ce2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:14,492 [527] [DEBUG] [app] Starting request: urn:request:102b5dbc-caf8-49fb-a2d6-c58852df5ce2 (/_internal_ping) 2020-03-31 09:02:14,493 [527] [DEBUG] [app] Ending request: urn:request:102b5dbc-caf8-49fb-a2d6-c58852df5ce2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:14,493 [527] [DEBUG] [app] Ending request: urn:request:102b5dbc-caf8-49fb-a2d6-c58852df5ce2 (/_internal_ping) 2020-03-31 09:02:14,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:02:14,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:02:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:02:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:02:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:02:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:02:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:14,506 [537] [DEBUG] [app] Starting request: urn:request:19bb2155-56d6-46cf-be9d-4f7c82c69f73 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:14,506 [537] [DEBUG] [app] Starting request: urn:request:19bb2155-56d6-46cf-be9d-4f7c82c69f73 (/c1/_internal_ping) 2020-03-31 09:02:14,507 [537] [DEBUG] [app] Ending request: urn:request:19bb2155-56d6-46cf-be9d-4f7c82c69f73 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:14,507 [537] [DEBUG] [app] Ending request: urn:request:19bb2155-56d6-46cf-be9d-4f7c82c69f73 (/c1/_internal_ping) 2020-03-31 09:02:14,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:14,508 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:14,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:02:14,508 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:14,509 [518] [DEBUG] [app] Ending request: urn:request:842afc05-1f65-492f-bbda-ece1076b9b14 (/health/instance) 2020-03-31 09:02:14,510 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:02:14,509 [518] [DEBUG] [app] Ending request: urn:request:842afc05-1f65-492f-bbda-ece1076b9b14 (/health/instance) 2020-03-31 09:02:14,510 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:02:14,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:02:14,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:02:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:02:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:45.064514+00:00 (in 29.999224 seconds) 2020-03-31 09:02:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:45 UTC)" (scheduled at 2020-03-31 09:02:15.064514+00:00) gcworker stdout | 2020-03-31 09:02:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:45.064514+00:00 (in 29.999224 seconds) 2020-03-31 09:02:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:45 UTC)" (scheduled at 2020-03-31 09:02:15.064514+00:00) 2020-03-31 09:02:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435735, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:02:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435735, 1209600, 500, 1, 0]) 2020-03-31 09:02:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:02:15,084 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:02:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:02:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:02:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:45 UTC)" executed successfully 2020-03-31 09:02:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:02:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:26.924497+00:00 (in 9.999196 seconds) 2020-03-31 09:02:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:26 UTC)" (scheduled at 2020-03-31 09:02:16.924497+00:00) 2020-03-31 09:02:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:02:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:26.924497+00:00 (in 9.999196 seconds) 2020-03-31 09:02:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:26 UTC)" (scheduled at 2020-03-31 09:02:16.924497+00:00) 2020-03-31 09:02:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 16, 925997), True, datetime.datetime(2020, 3, 31, 9, 2, 16, 925997), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:02:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 16, 925997), True, datetime.datetime(2020, 3, 31, 9, 2, 16, 925997), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:02:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:02:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:26 UTC)" executed successfully 2020-03-31 09:02:18,273 [518] [DEBUG] [app] Starting request: urn:request:ece30c23-7200-470f-9d0f-1243844fea93 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:18,273 [518] [DEBUG] [app] Starting request: urn:request:ece30c23-7200-470f-9d0f-1243844fea93 (/health/instance) 2020-03-31 09:02:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:18,277 [551] [DEBUG] [app] Starting request: urn:request:a2ee1c62-9ed2-414e-afc1-f9f8ae657880 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:18,277 [551] [DEBUG] [app] Starting request: urn:request:a2ee1c62-9ed2-414e-afc1-f9f8ae657880 (/v1/_internal_ping) 2020-03-31 09:02:18,278 [551] [DEBUG] [app] Ending request: urn:request:a2ee1c62-9ed2-414e-afc1-f9f8ae657880 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:18,278 [551] [DEBUG] [app] Ending request: urn:request:a2ee1c62-9ed2-414e-afc1-f9f8ae657880 (/v1/_internal_ping) 2020-03-31 09:02:18,279 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:02:18,279 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:18,281 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:02:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:18,282 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:18,283 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:18,281 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:02:18,282 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:18,283 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:18,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:18,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:18,309 [525] [DEBUG] [app] Starting request: urn:request:debbb7e2-762c-4ecc-b543-2d6f088b4cca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:18,309 [525] [DEBUG] [app] Starting request: urn:request:debbb7e2-762c-4ecc-b543-2d6f088b4cca (/_internal_ping) 2020-03-31 09:02:18,310 [525] [DEBUG] [app] Ending request: urn:request:debbb7e2-762c-4ecc-b543-2d6f088b4cca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:18,310 [525] [DEBUG] [app] Ending request: urn:request:debbb7e2-762c-4ecc-b543-2d6f088b4cca (/_internal_ping) 2020-03-31 09:02:18,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:02:18,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:02:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:02:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:02:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:02:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:18,324 [537] [DEBUG] [app] Starting request: urn:request:2a65d17f-a5f3-4f99-834b-4b27f09b164d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:18,324 [537] [DEBUG] [app] Starting request: urn:request:2a65d17f-a5f3-4f99-834b-4b27f09b164d (/c1/_internal_ping) 2020-03-31 09:02:18,325 [537] [DEBUG] [app] Ending request: urn:request:2a65d17f-a5f3-4f99-834b-4b27f09b164d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:18,325 [537] [DEBUG] [app] Ending request: urn:request:2a65d17f-a5f3-4f99-834b-4b27f09b164d (/c1/_internal_ping) 2020-03-31 09:02:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:18,327 [518] [DEBUG] [app] Ending request: urn:request:ece30c23-7200-470f-9d0f-1243844fea93 (/health/instance) 2020-03-31 09:02:18,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 10.131.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:02:18,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:18,327 [518] [DEBUG] [app] Ending request: urn:request:ece30c23-7200-470f-9d0f-1243844fea93 (/health/instance) 2020-03-31 09:02:18,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 10.131.0.1 () - - [31/Mar/2020:09:02:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-verbs stdout | 2020-03-31 09:02:18,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:02:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:31.440707+00:00 (in 9.999199 seconds) 2020-03-31 09:02:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:31 UTC)" (scheduled at 2020-03-31 09:02:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:02:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:31.440707+00:00 (in 9.999199 seconds) 2020-03-31 09:02:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:31 UTC)" (scheduled at 2020-03-31 09:02:21.440707+00:00) 2020-03-31 09:02:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:02:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:02:21,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:02:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:02:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:02:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:31 UTC)" executed successfully 2020-03-31 09:02:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:02:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:52.113221+00:00 (in 29.999143 seconds) 2020-03-31 09:02:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:52 UTC)" (scheduled at 2020-03-31 09:02:22.113221+00:00) securityworker stdout | 2020-03-31 09:02:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:52.113221+00:00 (in 29.999143 seconds) 2020-03-31 09:02:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:52 UTC)" (scheduled at 2020-03-31 09:02:22.113221+00:00) 2020-03-31 09:02:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:02:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:02:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:02:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:02:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:02:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:02:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:02:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:52 UTC)" executed successfully 2020-03-31 09:02:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:34 UTC)" (scheduled at 2020-03-31 09:02:24.323352+00:00) 2020-03-31 09:02:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:31.498397+00:00 (in 7.174312 seconds) notificationworker stdout | 2020-03-31 09:02:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:34 UTC)" (scheduled at 2020-03-31 09:02:24.323352+00:00) 2020-03-31 09:02:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:31.498397+00:00 (in 7.174312 seconds) 2020-03-31 09:02:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 24, 325161), True, datetime.datetime(2020, 3, 31, 9, 2, 24, 325161), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:02:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 24, 325161), True, datetime.datetime(2020, 3, 31, 9, 2, 24, 325161), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:02:24,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:02:24,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:24,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:24,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:34 UTC)" executed successfully 2020-03-31 09:02:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:02:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:33.330112+00:00 (in 8.918383 seconds) 2020-03-31 09:02:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:24 UTC)" (scheduled at 2020-03-31 09:02:24.410927+00:00) 2020-03-31 09:02:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:02:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:33.330112+00:00 (in 8.918383 seconds) 2020-03-31 09:02:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:24 UTC)" (scheduled at 2020-03-31 09:02:24.410927+00:00) 2020-03-31 09:02:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:24 UTC)" executed successfully 2020-03-31 09:02:24,446 [518] [DEBUG] [app] Starting request: urn:request:6991fcaa-c4cd-44f9-a1a0-7d3d11a73cb1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:24,446 [518] [DEBUG] [app] Starting request: urn:request:6991fcaa-c4cd-44f9-a1a0-7d3d11a73cb1 (/health/instance) 2020-03-31 09:02:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:24,455 [556] [DEBUG] [app] Starting request: urn:request:865a889c-5c0c-48f7-926c-ff67cf8587a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:24,455 [556] [DEBUG] [app] Starting request: urn:request:865a889c-5c0c-48f7-926c-ff67cf8587a0 (/v1/_internal_ping) 2020-03-31 09:02:24,457 [556] [DEBUG] [app] Ending request: urn:request:865a889c-5c0c-48f7-926c-ff67cf8587a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:24,457 [556] [DEBUG] [app] Ending request: urn:request:865a889c-5c0c-48f7-926c-ff67cf8587a0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:24,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:02:24,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:24,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:24,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:02:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:24,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:02:24,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:24,462 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:24,462 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:24,463 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:24,463 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:24,497 [518] [DEBUG] [app] Starting request: urn:request:3e0a990f-f1f3-4bc8-b27e-4326ec25ebba (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:24,497 [518] [DEBUG] [app] Starting request: urn:request:3e0a990f-f1f3-4bc8-b27e-4326ec25ebba (/_internal_ping) 2020-03-31 09:02:24,498 [518] [DEBUG] [app] Ending request: urn:request:3e0a990f-f1f3-4bc8-b27e-4326ec25ebba (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:24,498 [518] [DEBUG] [app] Ending request: urn:request:3e0a990f-f1f3-4bc8-b27e-4326ec25ebba (/_internal_ping) 2020-03-31 09:02:24,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:02:24,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:02:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:02:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:02:24,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:24,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:24,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:24,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:24,514 [532] [DEBUG] [app] Starting request: urn:request:b88d8cfe-85df-42db-8940-297c5160b545 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:24,514 [532] [DEBUG] [app] Starting request: urn:request:b88d8cfe-85df-42db-8940-297c5160b545 (/c1/_internal_ping) 2020-03-31 09:02:24,515 [532] [DEBUG] [app] Ending request: urn:request:b88d8cfe-85df-42db-8940-297c5160b545 (/c1/_internal_ping) 2020-03-31 09:02:24,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:24,516 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:02:24,515 [532] [DEBUG] [app] Ending request: urn:request:b88d8cfe-85df-42db-8940-297c5160b545 (/c1/_internal_ping) 2020-03-31 09:02:24,518 [518] [DEBUG] [app] Ending request: urn:request:6991fcaa-c4cd-44f9-a1a0-7d3d11a73cb1 (/health/instance) 2020-03-31 09:02:24,518 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:24,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 10.131.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:24,521 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:02:24,516 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:24,518 [518] [DEBUG] [app] Ending request: urn:request:6991fcaa-c4cd-44f9-a1a0-7d3d11a73cb1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:24,518 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) gunicorn-web stdout | 2020-03-31 09:02:24,521 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:02:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:36.924497+00:00 (in 9.999222 seconds) storagereplication stdout | 2020-03-31 09:02:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:36.924497+00:00 (in 9.999222 seconds) 2020-03-31 09:02:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:36 UTC)" (scheduled at 2020-03-31 09:02:26.924497+00:00) 2020-03-31 09:02:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:02:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:36 UTC)" (scheduled at 2020-03-31 09:02:26.924497+00:00) 2020-03-31 09:02:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 26, 926279), True, datetime.datetime(2020, 3, 31, 9, 2, 26, 926279), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:02:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 26, 926279), True, datetime.datetime(2020, 3, 31, 9, 2, 26, 926279), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:02:26,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:02:26,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:26,952 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:26,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:26,952 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:26,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:36 UTC)" executed successfully 2020-03-31 09:02:28,272 [518] [DEBUG] [app] Starting request: urn:request:6f01c5e4-beb8-4e80-a013-2a092d7b16d3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:28,272 [518] [DEBUG] [app] Starting request: urn:request:6f01c5e4-beb8-4e80-a013-2a092d7b16d3 (/health/instance) 2020-03-31 09:02:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:28,277 [556] [DEBUG] [app] Starting request: urn:request:f121c3b6-2bbe-4e81-8f26-b7a60f8bb9da (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:28,277 [556] [DEBUG] [app] Starting request: urn:request:f121c3b6-2bbe-4e81-8f26-b7a60f8bb9da (/v1/_internal_ping) 2020-03-31 09:02:28,278 [556] [DEBUG] [app] Ending request: urn:request:f121c3b6-2bbe-4e81-8f26-b7a60f8bb9da (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:28,278 [556] [DEBUG] [app] Ending request: urn:request:f121c3b6-2bbe-4e81-8f26-b7a60f8bb9da (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:28,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:02:28,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:28,281 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:02:28,281 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:28,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:28,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:28,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:28,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:28,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:28,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:28,312 [523] [DEBUG] [app] Starting request: urn:request:4356581a-0529-4349-b89b-6c7bc835d9c3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:28,312 [523] [DEBUG] [app] Starting request: urn:request:4356581a-0529-4349-b89b-6c7bc835d9c3 (/_internal_ping) 2020-03-31 09:02:28,313 [523] [DEBUG] [app] Ending request: urn:request:4356581a-0529-4349-b89b-6c7bc835d9c3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:28,313 [523] [DEBUG] [app] Ending request: urn:request:4356581a-0529-4349-b89b-6c7bc835d9c3 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:02:28,313 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:28,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:28,313 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:28,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:02:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:02:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:02:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:28,327 [532] [DEBUG] [app] Starting request: urn:request:1bde7fdb-8506-4ae8-86c1-7de469645b8e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:28,327 [532] [DEBUG] [app] Starting request: urn:request:1bde7fdb-8506-4ae8-86c1-7de469645b8e (/c1/_internal_ping) 2020-03-31 09:02:28,327 [532] [DEBUG] [app] Ending request: urn:request:1bde7fdb-8506-4ae8-86c1-7de469645b8e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:28,327 [532] [DEBUG] [app] Ending request: urn:request:1bde7fdb-8506-4ae8-86c1-7de469645b8e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:28,328 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:02:28,328 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:28,329 [518] [DEBUG] [app] Ending request: urn:request:6f01c5e4-beb8-4e80-a013-2a092d7b16d3 (/health/instance) 2020-03-31 09:02:28,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:28,329 [518] [DEBUG] [app] Ending request: urn:request:6f01c5e4-beb8-4e80-a013-2a092d7b16d3 (/health/instance) 2020-03-31 09:02:28,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:28,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:02:28,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:02:30,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:02:30,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:00.337699+00:00 (in 29.998155 seconds) 2020-03-31 09:02:30,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:00 UTC)" (scheduled at 2020-03-31 09:02:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:02:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:00.337699+00:00 (in 29.998155 seconds) 2020-03-31 09:02:30,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:00 UTC)" (scheduled at 2020-03-31 09:02:30.337699+00:00) 2020-03-31 09:02:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 2, 30, 340381), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:02:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 2, 30, 340381), False, 50, 1, 0]) 2020-03-31 09:02:30,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:02:30,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:02:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:02:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:00 UTC)" executed successfully 2020-03-31 09:02:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:02:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:41.440707+00:00 (in 9.998841 seconds) 2020-03-31 09:02:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:41 UTC)" (scheduled at 2020-03-31 09:02:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:02:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:41.440707+00:00 (in 9.998841 seconds) 2020-03-31 09:02:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:41 UTC)" (scheduled at 2020-03-31 09:02:31.440707+00:00) 2020-03-31 09:02:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:02:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:02:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:02:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:02:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:02:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:41 UTC)" executed successfully 2020-03-31 09:02:31,503 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:31,503 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:31,504 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:34.323352+00:00 (in 2.819597 seconds) 2020-03-31 09:02:31,504 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:31 UTC)" (scheduled at 2020-03-31 09:02:31.498397+00:00) notificationworker stdout | 2020-03-31 09:02:31,504 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:34.323352+00:00 (in 2.819597 seconds) 2020-03-31 09:02:31,504 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:31 UTC)" (scheduled at 2020-03-31 09:02:31.498397+00:00) 2020-03-31 09:02:31,505 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), u'notification/%']) notificationworker stdout | 2020-03-31 09:02:31,505 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), u'notification/%']) 2020-03-31 09:02:31,519 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), True, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:02:31,519 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), True, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), 0, u'notification/%']) 2020-03-31 09:02:31,523 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), True, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), u'notification/%']) notificationworker stdout | 2020-03-31 09:02:31,523 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), True, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 2, 31, 504562), u'notification/%']) 2020-03-31 09:02:31,527 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:02:31,527 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:31,529 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:31,529 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:07:31 UTC)" executed successfully 2020-03-31 09:02:31,529 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:02:31,529 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:31,530 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:02:31,530 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:31,531 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics notificationworker stdout | 2020-03-31 09:02:31,531 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:31,533 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:02:31,533 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:31,534 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:02:31,534 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:31,535 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics notificationworker stdout | 2020-03-31 09:02:31,535 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:02:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:02:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:03.330112+00:00 (in 29.998628 seconds) 2020-03-31 09:02:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:03 UTC)" (scheduled at 2020-03-31 09:02:33.330112+00:00) 2020-03-31 09:02:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:02:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:03.330112+00:00 (in 29.998628 seconds) 2020-03-31 09:02:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:03 UTC)" (scheduled at 2020-03-31 09:02:33.330112+00:00) 2020-03-31 09:02:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 33, 332384), True, datetime.datetime(2020, 3, 31, 9, 2, 33, 332384), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:02:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 33, 332384), True, datetime.datetime(2020, 3, 31, 9, 2, 33, 332384), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:02:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:02:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:02:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:03 UTC)" executed successfully 2020-03-31 09:02:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:02:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:43.539999+00:00 (in 10.172423 seconds) 2020-03-31 09:02:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:33 UTC)" (scheduled at 2020-03-31 09:02:33.366791+00:00) 2020-03-31 09:02:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:02:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:43.539999+00:00 (in 10.172423 seconds) 2020-03-31 09:02:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:33 UTC)" (scheduled at 2020-03-31 09:02:33.366791+00:00) 2020-03-31 09:02:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 33, 368570), True, datetime.datetime(2020, 3, 31, 9, 2, 33, 368570), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:02:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 33, 368570), True, datetime.datetime(2020, 3, 31, 9, 2, 33, 368570), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:02:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:02:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:02:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:33 UTC)" executed successfully 2020-03-31 09:02:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:44.323352+00:00 (in 9.999112 seconds) 2020-03-31 09:02:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:44 UTC)" (scheduled at 2020-03-31 09:02:34.323352+00:00) 2020-03-31 09:02:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:02:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:44.323352+00:00 (in 9.999112 seconds) 2020-03-31 09:02:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:44 UTC)" (scheduled at 2020-03-31 09:02:34.323352+00:00) 2020-03-31 09:02:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 34, 325144), True, datetime.datetime(2020, 3, 31, 9, 2, 34, 325144), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:02:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 34, 325144), True, datetime.datetime(2020, 3, 31, 9, 2, 34, 325144), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:02:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:02:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:44 UTC)" executed successfully 2020-03-31 09:02:34,446 [527] [DEBUG] [app] Starting request: urn:request:8e05af34-31db-4f88-a7f3-876ed10117ca (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:34,446 [527] [DEBUG] [app] Starting request: urn:request:8e05af34-31db-4f88-a7f3-876ed10117ca (/health/instance) 2020-03-31 09:02:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:34,454 [538] [DEBUG] [app] Starting request: urn:request:5a5b78ab-8da8-4394-af8c-a923dfb367f4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:34,454 [538] [DEBUG] [app] Starting request: urn:request:5a5b78ab-8da8-4394-af8c-a923dfb367f4 (/v1/_internal_ping) 2020-03-31 09:02:34,455 [538] [DEBUG] [app] Ending request: urn:request:5a5b78ab-8da8-4394-af8c-a923dfb367f4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:34,455 [538] [DEBUG] [app] Ending request: urn:request:5a5b78ab-8da8-4394-af8c-a923dfb367f4 (/v1/_internal_ping) 2020-03-31 09:02:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:02:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:02:34,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:02:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:02:34,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:34,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:34,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:34,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:34,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:34,491 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:34,491 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:34,497 [527] [DEBUG] [app] Starting request: urn:request:3f7b9034-c842-4dc0-89b6-c570a94a1768 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:34,497 [527] [DEBUG] [app] Starting request: urn:request:3f7b9034-c842-4dc0-89b6-c570a94a1768 (/_internal_ping) 2020-03-31 09:02:34,498 [527] [DEBUG] [app] Ending request: urn:request:3f7b9034-c842-4dc0-89b6-c570a94a1768 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:34,498 [527] [DEBUG] [app] Ending request: urn:request:3f7b9034-c842-4dc0-89b6-c570a94a1768 (/_internal_ping) 2020-03-31 09:02:34,499 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:02:34,499 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:02:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:02:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:34,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:34,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:02:34,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:34,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:34,513 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:34,513 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:34,514 [537] [DEBUG] [app] Starting request: urn:request:a6608f37-09a3-4f8a-b654-570b58447c9f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:34,514 [537] [DEBUG] [app] Starting request: urn:request:a6608f37-09a3-4f8a-b654-570b58447c9f (/c1/_internal_ping) 2020-03-31 09:02:34,515 [537] [DEBUG] [app] Ending request: urn:request:a6608f37-09a3-4f8a-b654-570b58447c9f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:34,515 [537] [DEBUG] [app] Ending request: urn:request:a6608f37-09a3-4f8a-b654-570b58447c9f (/c1/_internal_ping) 2020-03-31 09:02:34,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:34,515 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:34,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:34,517 [527] [DEBUG] [app] Ending request: urn:request:8e05af34-31db-4f88-a7f3-876ed10117ca (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:02:34,515 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:34,518 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:34,518 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:02:34,517 [527] [DEBUG] [app] Ending request: urn:request:8e05af34-31db-4f88-a7f3-876ed10117ca (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:34,518 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:34,518 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:02:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:02:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:38.010934+00:00 (in 1.085312 seconds) 2020-03-31 09:02:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:46 UTC)" (scheduled at 2020-03-31 09:02:36.924497+00:00) 2020-03-31 09:02:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:02:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:38.010934+00:00 (in 1.085312 seconds) 2020-03-31 09:02:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:46 UTC)" (scheduled at 2020-03-31 09:02:36.924497+00:00) 2020-03-31 09:02:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 36, 926509), True, datetime.datetime(2020, 3, 31, 9, 2, 36, 926509), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:02:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 36, 926509), True, datetime.datetime(2020, 3, 31, 9, 2, 36, 926509), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:02:36,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:02:36,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:46 UTC)" executed successfully 2020-03-31 09:02:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:02:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:46.924497+00:00 (in 8.912769 seconds) 2020-03-31 09:02:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:38 UTC)" (scheduled at 2020-03-31 09:02:38.010934+00:00) 2020-03-31 09:02:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:02:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:46.924497+00:00 (in 8.912769 seconds) 2020-03-31 09:02:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:38 UTC)" (scheduled at 2020-03-31 09:02:38.010934+00:00) 2020-03-31 09:02:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:38 UTC)" executed successfully 2020-03-31 09:02:38,272 [526] [DEBUG] [app] Starting request: urn:request:2d0b25e6-1899-4081-850b-4ca5182b7d7c (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:38,272 [526] [DEBUG] [app] Starting request: urn:request:2d0b25e6-1899-4081-850b-4ca5182b7d7c (/health/instance) 2020-03-31 09:02:38,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:38,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:38,279 [553] [DEBUG] [app] Starting request: urn:request:2c6e6eb8-ef64-4fbc-a6f4-d66889465609 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:38,279 [553] [DEBUG] [app] Starting request: urn:request:2c6e6eb8-ef64-4fbc-a6f4-d66889465609 (/v1/_internal_ping) 2020-03-31 09:02:38,280 [553] [DEBUG] [app] Ending request: urn:request:2c6e6eb8-ef64-4fbc-a6f4-d66889465609 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:38,280 [553] [DEBUG] [app] Ending request: urn:request:2c6e6eb8-ef64-4fbc-a6f4-d66889465609 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:38,281 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:38,281 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:02:38,281 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:38,282 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:38,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:02:38,281 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:38,282 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:38,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:38,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:38,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:38,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:38,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:38,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:38,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:38,309 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:38,309 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:38,310 [526] [DEBUG] [app] Starting request: urn:request:d1689574-276a-4d22-98fe-a993eafae256 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:38,310 [526] [DEBUG] [app] Starting request: urn:request:d1689574-276a-4d22-98fe-a993eafae256 (/_internal_ping) 2020-03-31 09:02:38,311 [526] [DEBUG] [app] Ending request: urn:request:d1689574-276a-4d22-98fe-a993eafae256 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:38,311 [526] [DEBUG] [app] Ending request: urn:request:d1689574-276a-4d22-98fe-a993eafae256 (/_internal_ping) 2020-03-31 09:02:38,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:02:38,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:38,312 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:02:38,312 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:38,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:38,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:02:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:02:38,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:38,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:38,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:38,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:38,324 [537] [DEBUG] [app] Starting request: urn:request:a4e3fab3-3aee-4832-919f-829a4fd60338 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:38,324 [537] [DEBUG] [app] Starting request: urn:request:a4e3fab3-3aee-4832-919f-829a4fd60338 (/c1/_internal_ping) 2020-03-31 09:02:38,325 [537] [DEBUG] [app] Ending request: urn:request:a4e3fab3-3aee-4832-919f-829a4fd60338 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:38,325 [537] [DEBUG] [app] Ending request: urn:request:a4e3fab3-3aee-4832-919f-829a4fd60338 (/c1/_internal_ping) 2020-03-31 09:02:38,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:38,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:38,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:38,327 [526] [DEBUG] [app] Ending request: urn:request:2d0b25e6-1899-4081-850b-4ca5182b7d7c (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:02:38,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:38,328 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:38,327 [526] [DEBUG] [app] Ending request: urn:request:2d0b25e6-1899-4081-850b-4ca5182b7d7c (/health/instance) 2020-03-31 09:02:38,328 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:02:38,328 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:38,328 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:02:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:02:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:51.440707+00:00 (in 9.998986 seconds) 2020-03-31 09:02:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:51 UTC)" (scheduled at 2020-03-31 09:02:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:02:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:51.440707+00:00 (in 9.998986 seconds) 2020-03-31 09:02:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:51 UTC)" (scheduled at 2020-03-31 09:02:41.440707+00:00) 2020-03-31 09:02:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:02:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:02:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:02:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:02:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:02:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:51 UTC)" executed successfully 2020-03-31 09:02:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:02:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:33.366791+00:00 (in 49.825574 seconds) 2020-03-31 09:02:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:43 UTC)" (scheduled at 2020-03-31 09:02:43.539999+00:00) 2020-03-31 09:02:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:02:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:33.366791+00:00 (in 49.825574 seconds) 2020-03-31 09:02:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:43 UTC)" (scheduled at 2020-03-31 09:02:43.539999+00:00) 2020-03-31 09:02:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:02:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:43 UTC)" executed successfully 2020-03-31 09:02:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:49.498966+00:00 (in 5.174733 seconds) 2020-03-31 09:02:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:54 UTC)" (scheduled at 2020-03-31 09:02:44.323352+00:00) 2020-03-31 09:02:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:02:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:49.498966+00:00 (in 5.174733 seconds) 2020-03-31 09:02:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:54 UTC)" (scheduled at 2020-03-31 09:02:44.323352+00:00) 2020-03-31 09:02:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 44, 325263), True, datetime.datetime(2020, 3, 31, 9, 2, 44, 325263), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:02:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 44, 325263), True, datetime.datetime(2020, 3, 31, 9, 2, 44, 325263), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:02:44,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:02:44,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:54 UTC)" executed successfully 2020-03-31 09:02:44,446 [527] [DEBUG] [app] Starting request: urn:request:a9a6255e-2f00-4ca1-8b32-0b9bbda6372a (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:44,446 [527] [DEBUG] [app] Starting request: urn:request:a9a6255e-2f00-4ca1-8b32-0b9bbda6372a (/health/instance) 2020-03-31 09:02:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:44,455 [540] [DEBUG] [app] Starting request: urn:request:6459b3e1-b5af-42e1-815a-f64544525180 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:44,455 [540] [DEBUG] [app] Starting request: urn:request:6459b3e1-b5af-42e1-815a-f64544525180 (/v1/_internal_ping) 2020-03-31 09:02:44,456 [540] [DEBUG] [app] Ending request: urn:request:6459b3e1-b5af-42e1-815a-f64544525180 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:44,456 [540] [DEBUG] [app] Ending request: urn:request:6459b3e1-b5af-42e1-815a-f64544525180 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:44,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:02:44,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:44,459 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:02:44,459 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:44,460 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:44,460 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:44,461 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:44,461 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:44,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:44,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:44,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:44,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:44,487 [527] [DEBUG] [app] Starting request: urn:request:6d130970-2710-4b77-8bdf-4ed15ecdb58e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:44,487 [527] [DEBUG] [app] Starting request: urn:request:6d130970-2710-4b77-8bdf-4ed15ecdb58e (/_internal_ping) 2020-03-31 09:02:44,488 [527] [DEBUG] [app] Ending request: urn:request:6d130970-2710-4b77-8bdf-4ed15ecdb58e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:44,488 [527] [DEBUG] [app] Ending request: urn:request:6d130970-2710-4b77-8bdf-4ed15ecdb58e (/_internal_ping) 2020-03-31 09:02:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:02:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:44,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:02:44,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:44,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:44,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:02:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:44,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:44,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:02:44,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:44,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:44,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:44,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:44,500 [543] [DEBUG] [app] Starting request: urn:request:0743256a-38d0-4306-bb7e-c3b3d1503f2f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:44,500 [543] [DEBUG] [app] Starting request: urn:request:0743256a-38d0-4306-bb7e-c3b3d1503f2f (/c1/_internal_ping) 2020-03-31 09:02:44,501 [543] [DEBUG] [app] Ending request: urn:request:0743256a-38d0-4306-bb7e-c3b3d1503f2f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:44,501 [543] [DEBUG] [app] Ending request: urn:request:0743256a-38d0-4306-bb7e-c3b3d1503f2f (/c1/_internal_ping) 2020-03-31 09:02:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:44,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:44,504 [527] [DEBUG] [app] Ending request: urn:request:a9a6255e-2f00-4ca1-8b32-0b9bbda6372a (/health/instance) 2020-03-31 09:02:44,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:02:44,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:02:44,504 [527] [DEBUG] [app] Ending request: urn:request:a9a6255e-2f00-4ca1-8b32-0b9bbda6372a (/health/instance) 2020-03-31 09:02:44,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:02:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:02:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:15.064514+00:00 (in 29.999001 seconds) 2020-03-31 09:02:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:15 UTC)" (scheduled at 2020-03-31 09:02:45.064514+00:00) gcworker stdout | 2020-03-31 09:02:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:15.064514+00:00 (in 29.999001 seconds) 2020-03-31 09:02:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:15 UTC)" (scheduled at 2020-03-31 09:02:45.064514+00:00) 2020-03-31 09:02:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435765, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:02:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435765, 1209600, 500, 1, 0]) 2020-03-31 09:02:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:02:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:02:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:02:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:15 UTC)" executed successfully 2020-03-31 09:02:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:02:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:56.924497+00:00 (in 9.998823 seconds) 2020-03-31 09:02:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:56 UTC)" (scheduled at 2020-03-31 09:02:46.924497+00:00) 2020-03-31 09:02:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:02:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:56.924497+00:00 (in 9.998823 seconds) 2020-03-31 09:02:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:56 UTC)" (scheduled at 2020-03-31 09:02:46.924497+00:00) 2020-03-31 09:02:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 46, 926304), True, datetime.datetime(2020, 3, 31, 9, 2, 46, 926304), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:02:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 46, 926304), True, datetime.datetime(2020, 3, 31, 9, 2, 46, 926304), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:02:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:02:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:02:56 UTC)" executed successfully 2020-03-31 09:02:48,273 [518] [DEBUG] [app] Starting request: urn:request:c98b6eaa-7eb8-4ec2-9db6-9deaf36316c8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:48,273 [518] [DEBUG] [app] Starting request: urn:request:c98b6eaa-7eb8-4ec2-9db6-9deaf36316c8 (/health/instance) 2020-03-31 09:02:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:48,280 [555] [DEBUG] [app] Starting request: urn:request:b058ed4e-20a3-4042-8e64-f08ce6f3ea5a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:48,280 [555] [DEBUG] [app] Starting request: urn:request:b058ed4e-20a3-4042-8e64-f08ce6f3ea5a (/v1/_internal_ping) 2020-03-31 09:02:48,281 [555] [DEBUG] [app] Ending request: urn:request:b058ed4e-20a3-4042-8e64-f08ce6f3ea5a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:48,281 [555] [DEBUG] [app] Ending request: urn:request:b058ed4e-20a3-4042-8e64-f08ce6f3ea5a (/v1/_internal_ping) 2020-03-31 09:02:48,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:02:48,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:48,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:48,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:48,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:48,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:02:48,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:02:48,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:48,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:48,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:48,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:48,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:48,312 [527] [DEBUG] [app] Starting request: urn:request:e49f6b93-79a5-4f00-9a70-0b5133904cfe (/_internal_ping) 2020-03-31 09:02:48,313 [527] [DEBUG] [app] Ending request: urn:request:e49f6b93-79a5-4f00-9a70-0b5133904cfe (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:02:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:48,312 [527] [DEBUG] [app] Starting request: urn:request:e49f6b93-79a5-4f00-9a70-0b5133904cfe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:48,313 [527] [DEBUG] [app] Ending request: urn:request:e49f6b93-79a5-4f00-9a70-0b5133904cfe (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:02:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:02:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:02:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:02:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:48,325 [543] [DEBUG] [app] Starting request: urn:request:23bcf3aa-702c-49fc-a605-77885321442d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:48,325 [543] [DEBUG] [app] Starting request: urn:request:23bcf3aa-702c-49fc-a605-77885321442d (/c1/_internal_ping) 2020-03-31 09:02:48,326 [543] [DEBUG] [app] Ending request: urn:request:23bcf3aa-702c-49fc-a605-77885321442d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:48,326 [543] [DEBUG] [app] Ending request: urn:request:23bcf3aa-702c-49fc-a605-77885321442d (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:48,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:02:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:02:48,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:48,328 [518] [DEBUG] [app] Ending request: urn:request:c98b6eaa-7eb8-4ec2-9db6-9deaf36316c8 (/health/instance) 2020-03-31 09:02:48,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:48,328 [518] [DEBUG] [app] Ending request: urn:request:c98b6eaa-7eb8-4ec2-9db6-9deaf36316c8 (/health/instance) 2020-03-31 09:02:48,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:02:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:02:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:49 UTC)" (scheduled at 2020-03-31 09:02:49.498966+00:00) 2020-03-31 09:02:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:54.323352+00:00 (in 4.823381 seconds) notificationworker stdout | 2020-03-31 09:02:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:49 UTC)" (scheduled at 2020-03-31 09:02:49.498966+00:00) 2020-03-31 09:02:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:54.323352+00:00 (in 4.823381 seconds) 2020-03-31 09:02:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:49 UTC)" executed successfully 2020-03-31 09:02:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:02:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:51 UTC)" (scheduled at 2020-03-31 09:02:51.110610+00:00) 2020-03-31 09:02:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:59.560326+00:00 (in 8.448064 seconds) 2020-03-31 09:02:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:02:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:02:51 UTC)" (scheduled at 2020-03-31 09:02:51.110610+00:00) 2020-03-31 09:02:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:02:59.560326+00:00 (in 8.448064 seconds) 2020-03-31 09:02:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 51, 113256), True, datetime.datetime(2020, 3, 31, 9, 2, 51, 113256), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:02:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 51, 113256), True, datetime.datetime(2020, 3, 31, 9, 2, 51, 113256), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:02:51,128 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:02:51,128 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:02:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:51 UTC)" executed successfully 2020-03-31 09:02:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:02:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:01.440707+00:00 (in 9.999178 seconds) 2020-03-31 09:02:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:01 UTC)" (scheduled at 2020-03-31 09:02:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:02:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:01.440707+00:00 (in 9.999178 seconds) 2020-03-31 09:02:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:01 UTC)" (scheduled at 2020-03-31 09:02:51.440707+00:00) 2020-03-31 09:02:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:02:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:02:51,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:02:51,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:02:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:02:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:01 UTC)" executed successfully 2020-03-31 09:02:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:02:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:52 UTC)" (scheduled at 2020-03-31 09:02:52.113221+00:00) securityworker stdout | 2020-03-31 09:02:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:02:52 UTC)" (scheduled at 2020-03-31 09:02:52.113221+00:00) 2020-03-31 09:02:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:02:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:22.113221+00:00 (in 29.999263 seconds) securityworker stdout | 2020-03-31 09:02:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:02:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:22.113221+00:00 (in 29.999263 seconds) 2020-03-31 09:02:52,126 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:02:52,126 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:02:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:02:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:02:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:52,129 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:02:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:52,129 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:22 UTC)" executed successfully 2020-03-31 09:02:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:02:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:04.323352+00:00 (in 9.998559 seconds) 2020-03-31 09:02:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:04 UTC)" (scheduled at 2020-03-31 09:02:54.323352+00:00) 2020-03-31 09:02:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:02:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:04.323352+00:00 (in 9.998559 seconds) 2020-03-31 09:02:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:04 UTC)" (scheduled at 2020-03-31 09:02:54.323352+00:00) 2020-03-31 09:02:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 54, 325474), True, datetime.datetime(2020, 3, 31, 9, 2, 54, 325474), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:02:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 54, 325474), True, datetime.datetime(2020, 3, 31, 9, 2, 54, 325474), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:02:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:54,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:02:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:02:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:04 UTC)" executed successfully 2020-03-31 09:02:54,445 [527] [DEBUG] [app] Starting request: urn:request:24bcaf0e-60c7-4d29-a16e-4325130b5e07 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:54,445 [527] [DEBUG] [app] Starting request: urn:request:24bcaf0e-60c7-4d29-a16e-4325130b5e07 (/health/instance) 2020-03-31 09:02:54,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:54,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:54,452 [542] [DEBUG] [app] Starting request: urn:request:008c135a-9b82-42f5-a3da-448ec26f50f9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:54,452 [542] [DEBUG] [app] Starting request: urn:request:008c135a-9b82-42f5-a3da-448ec26f50f9 (/v1/_internal_ping) 2020-03-31 09:02:54,454 [542] [DEBUG] [app] Ending request: urn:request:008c135a-9b82-42f5-a3da-448ec26f50f9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:54,454 [542] [DEBUG] [app] Ending request: urn:request:008c135a-9b82-42f5-a3da-448ec26f50f9 (/v1/_internal_ping) 2020-03-31 09:02:54,454 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:54,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:02:54,454 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:54,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:54,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:02:54,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:02:54,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:02:54,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:54,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:54,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:54,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:54,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:54,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:54,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:54,482 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:54,482 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:54,484 [526] [DEBUG] [app] Starting request: urn:request:ed0fdbb6-5f2e-4c97-aa85-1d65318781e7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:54,484 [526] [DEBUG] [app] Starting request: urn:request:ed0fdbb6-5f2e-4c97-aa85-1d65318781e7 (/_internal_ping) 2020-03-31 09:02:54,485 [526] [DEBUG] [app] Ending request: urn:request:ed0fdbb6-5f2e-4c97-aa85-1d65318781e7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:54,485 [526] [DEBUG] [app] Ending request: urn:request:ed0fdbb6-5f2e-4c97-aa85-1d65318781e7 (/_internal_ping) 2020-03-31 09:02:54,485 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:54,485 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:02:54,485 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:54,485 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:02:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:02:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:02:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:54,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:54,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:54,499 [543] [DEBUG] [app] Starting request: urn:request:d5413d63-6c27-4e38-95df-7efe73ab1f68 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:54,499 [543] [DEBUG] [app] Starting request: urn:request:d5413d63-6c27-4e38-95df-7efe73ab1f68 (/c1/_internal_ping) 2020-03-31 09:02:54,500 [543] [DEBUG] [app] Ending request: urn:request:d5413d63-6c27-4e38-95df-7efe73ab1f68 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:54,500 [543] [DEBUG] [app] Ending request: urn:request:d5413d63-6c27-4e38-95df-7efe73ab1f68 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:54,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:54,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:54,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:02:54,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:54,503 [527] [DEBUG] [app] Ending request: urn:request:24bcaf0e-60c7-4d29-a16e-4325130b5e07 (/health/instance) 2020-03-31 09:02:54,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:54,503 [527] [DEBUG] [app] Ending request: urn:request:24bcaf0e-60c7-4d29-a16e-4325130b5e07 (/health/instance) 2020-03-31 09:02:54,503 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:02:54,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:02:54,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:02:56,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:02:56,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:06.924497+00:00 (in 9.999298 seconds) 2020-03-31 09:02:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:06 UTC)" (scheduled at 2020-03-31 09:02:56.924497+00:00) 2020-03-31 09:02:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:02:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:06.924497+00:00 (in 9.999298 seconds) 2020-03-31 09:02:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:06 UTC)" (scheduled at 2020-03-31 09:02:56.924497+00:00) 2020-03-31 09:02:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:02:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 56, 925993), True, datetime.datetime(2020, 3, 31, 9, 2, 56, 925993), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:02:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 2, 56, 925993), True, datetime.datetime(2020, 3, 31, 9, 2, 56, 925993), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:02:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:02:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:02:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:02:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:06 UTC)" executed successfully 2020-03-31 09:02:58,273 [527] [DEBUG] [app] Starting request: urn:request:7dfbd6d9-cab4-4e01-986d-1e2a4aa88847 (/health/instance) gunicorn-web stdout | 2020-03-31 09:02:58,273 [527] [DEBUG] [app] Starting request: urn:request:7dfbd6d9-cab4-4e01-986d-1e2a4aa88847 (/health/instance) 2020-03-31 09:02:58,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:58,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:58,278 [553] [DEBUG] [app] Starting request: urn:request:c28c02e0-ee4e-46ed-838b-a2b2e95c4144 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:58,278 [553] [DEBUG] [app] Starting request: urn:request:c28c02e0-ee4e-46ed-838b-a2b2e95c4144 (/v1/_internal_ping) 2020-03-31 09:02:58,279 [553] [DEBUG] [app] Ending request: urn:request:c28c02e0-ee4e-46ed-838b-a2b2e95c4144 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:02:58,279 [553] [DEBUG] [app] Ending request: urn:request:c28c02e0-ee4e-46ed-838b-a2b2e95c4144 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:58,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:02:58,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:02:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:58,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:02:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:02:58,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:02:58,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:02:58,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:02:58,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:02:58,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:02:58,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:02:58,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:02:58,303 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:02:58,303 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:02:58,307 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:58,307 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:58,308 [527] [DEBUG] [app] Starting request: urn:request:35edf190-4b24-43b0-8760-a7ff764f5ede (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:58,308 [527] [DEBUG] [app] Starting request: urn:request:35edf190-4b24-43b0-8760-a7ff764f5ede (/_internal_ping) 2020-03-31 09:02:58,309 [527] [DEBUG] [app] Ending request: urn:request:35edf190-4b24-43b0-8760-a7ff764f5ede (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:02:58,309 [527] [DEBUG] [app] Ending request: urn:request:35edf190-4b24-43b0-8760-a7ff764f5ede (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:02:58,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:58,310 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:02:58,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:58,310 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:02:58,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:58,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:02:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:02:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:02:58,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:58,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:02:58,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:02:58,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:02:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:02:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:02:58,322 [541] [DEBUG] [app] Starting request: urn:request:0727c09a-8bc7-48d9-a5ff-1262389c340e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:58,322 [541] [DEBUG] [app] Starting request: urn:request:0727c09a-8bc7-48d9-a5ff-1262389c340e (/c1/_internal_ping) 2020-03-31 09:02:58,323 [541] [DEBUG] [app] Ending request: urn:request:0727c09a-8bc7-48d9-a5ff-1262389c340e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:02:58,323 [541] [DEBUG] [app] Ending request: urn:request:0727c09a-8bc7-48d9-a5ff-1262389c340e (/c1/_internal_ping) 2020-03-31 09:02:58,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:02:58,323 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:02:58,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:02:58,323 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:02:58,325 [527] [DEBUG] [app] Ending request: urn:request:7dfbd6d9-cab4-4e01-986d-1e2a4aa88847 (/health/instance) 2020-03-31 09:02:58,325 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:02:58,325 [527] [DEBUG] [app] Ending request: urn:request:7dfbd6d9-cab4-4e01-986d-1e2a4aa88847 (/health/instance) 2020-03-31 09:02:58,325 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:02:58,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.053 49 0.053) gunicorn-web stdout | 2020-03-31 09:02:58,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:02:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:02:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.053 49 0.053) 2020-03-31 09:02:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:02:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:02:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:51.110610+00:00 (in 51.548210 seconds) 2020-03-31 09:02:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:59 UTC)" (scheduled at 2020-03-31 09:02:59.560326+00:00) 2020-03-31 09:02:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:02:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:51.110610+00:00 (in 51.548210 seconds) 2020-03-31 09:02:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:59 UTC)" (scheduled at 2020-03-31 09:02:59.560326+00:00) 2020-03-31 09:02:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:02:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:03:59 UTC)" executed successfully 2020-03-31 09:03:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:30 UTC)" (scheduled at 2020-03-31 09:03:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:03:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:30 UTC)" (scheduled at 2020-03-31 09:03:00.337699+00:00) 2020-03-31 09:03:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:30.337699+00:00 (in 29.999149 seconds) 2020-03-31 09:03:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 3, 0, 339425), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:03:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:30.337699+00:00 (in 29.999149 seconds) 2020-03-31 09:03:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 3, 0, 339425), False, 50, 1, 0]) 2020-03-31 09:03:00,362 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:03:00,362 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:03:00,362 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:00,362 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:03:00,362 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:00,362 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:30 UTC)" executed successfully 2020-03-31 09:03:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:03:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:11.440707+00:00 (in 9.999105 seconds) 2020-03-31 09:03:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:11 UTC)" (scheduled at 2020-03-31 09:03:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:03:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:11.440707+00:00 (in 9.999105 seconds) 2020-03-31 09:03:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:11 UTC)" (scheduled at 2020-03-31 09:03:01.440707+00:00) 2020-03-31 09:03:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:03:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:03:01,456 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:03:01,456 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:03:01,456 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:01,456 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:03:01,456 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:01,456 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:11 UTC)" executed successfully 2020-03-31 09:03:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:03:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:24.410927+00:00 (in 21.080002 seconds) 2020-03-31 09:03:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:33 UTC)" (scheduled at 2020-03-31 09:03:03.330112+00:00) 2020-03-31 09:03:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:03:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:24.410927+00:00 (in 21.080002 seconds) 2020-03-31 09:03:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:33 UTC)" (scheduled at 2020-03-31 09:03:03.330112+00:00) 2020-03-31 09:03:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 3, 331758), True, datetime.datetime(2020, 3, 31, 9, 3, 3, 331758), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:03:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 3, 331758), True, datetime.datetime(2020, 3, 31, 9, 3, 3, 331758), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:03:03,347 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:03,347 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:03:03,347 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:03,347 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:03,348 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:03:03,348 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:33 UTC)" executed successfully 2020-03-31 09:03:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:03:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:14.323352+00:00 (in 9.999307 seconds) 2020-03-31 09:03:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:14 UTC)" (scheduled at 2020-03-31 09:03:04.323352+00:00) 2020-03-31 09:03:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:03:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:14.323352+00:00 (in 9.999307 seconds) 2020-03-31 09:03:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:14 UTC)" (scheduled at 2020-03-31 09:03:04.323352+00:00) 2020-03-31 09:03:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 4, 325010), True, datetime.datetime(2020, 3, 31, 9, 3, 4, 325010), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:03:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 4, 325010), True, datetime.datetime(2020, 3, 31, 9, 3, 4, 325010), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:03:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:03:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:03:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:14 UTC)" executed successfully 2020-03-31 09:03:04,446 [518] [DEBUG] [app] Starting request: urn:request:c6d0b03e-67c5-4b76-9f98-b55d578cb4b4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:04,446 [518] [DEBUG] [app] Starting request: urn:request:c6d0b03e-67c5-4b76-9f98-b55d578cb4b4 (/health/instance) 2020-03-31 09:03:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:04,453 [553] [DEBUG] [app] Starting request: urn:request:7b93d05f-b0d7-4254-837b-d5de136d19f9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:04,453 [553] [DEBUG] [app] Starting request: urn:request:7b93d05f-b0d7-4254-837b-d5de136d19f9 (/v1/_internal_ping) 2020-03-31 09:03:04,454 [553] [DEBUG] [app] Ending request: urn:request:7b93d05f-b0d7-4254-837b-d5de136d19f9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:04,454 [553] [DEBUG] [app] Ending request: urn:request:7b93d05f-b0d7-4254-837b-d5de136d19f9 (/v1/_internal_ping) 2020-03-31 09:03:04,454 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:03:04,454 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:03:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:04,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:04,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:04,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:04,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:04,487 [526] [DEBUG] [app] Starting request: urn:request:5bfa676f-d315-4c48-b324-e22c780111d5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:04,487 [526] [DEBUG] [app] Starting request: urn:request:5bfa676f-d315-4c48-b324-e22c780111d5 (/_internal_ping) 2020-03-31 09:03:04,488 [526] [DEBUG] [app] Ending request: urn:request:5bfa676f-d315-4c48-b324-e22c780111d5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:04,488 [526] [DEBUG] [app] Ending request: urn:request:5bfa676f-d315-4c48-b324-e22c780111d5 (/_internal_ping) 2020-03-31 09:03:04,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:03:04,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:03:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:03:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:03:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:03:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:04,501 [541] [DEBUG] [app] Starting request: urn:request:bb15af90-c8e6-4813-9541-f9ba7d987894 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:04,501 [541] [DEBUG] [app] Starting request: urn:request:bb15af90-c8e6-4813-9541-f9ba7d987894 (/c1/_internal_ping) 2020-03-31 09:03:04,502 [541] [DEBUG] [app] Ending request: urn:request:bb15af90-c8e6-4813-9541-f9ba7d987894 (/c1/_internal_ping) 2020-03-31 09:03:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:04,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:03:04,502 [541] [DEBUG] [app] Ending request: urn:request:bb15af90-c8e6-4813-9541-f9ba7d987894 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:04,504 [518] [DEBUG] [app] Ending request: urn:request:c6d0b03e-67c5-4b76-9f98-b55d578cb4b4 (/health/instance) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:04,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-verbs stdout | 2020-03-31 09:03:04,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:04,504 [518] [DEBUG] [app] Ending request: urn:request:c6d0b03e-67c5-4b76-9f98-b55d578cb4b4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:04,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:03:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:03:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:16.924497+00:00 (in 9.999204 seconds) 2020-03-31 09:03:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:16 UTC)" (scheduled at 2020-03-31 09:03:06.924497+00:00) 2020-03-31 09:03:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:03:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:16.924497+00:00 (in 9.999204 seconds) 2020-03-31 09:03:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:16 UTC)" (scheduled at 2020-03-31 09:03:06.924497+00:00) 2020-03-31 09:03:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 6, 926090), True, datetime.datetime(2020, 3, 31, 9, 3, 6, 926090), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:03:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 6, 926090), True, datetime.datetime(2020, 3, 31, 9, 3, 6, 926090), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:03:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:03:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:03:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:16 UTC)" executed successfully 2020-03-31 09:03:08,273 [527] [DEBUG] [app] Starting request: urn:request:b4c6f161-eed4-412d-bc21-b20b235a1b7b (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:08,273 [527] [DEBUG] [app] Starting request: urn:request:b4c6f161-eed4-412d-bc21-b20b235a1b7b (/health/instance) 2020-03-31 09:03:08,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:08,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:08,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:08,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:08,280 [555] [DEBUG] [app] Starting request: urn:request:ac38fd76-2877-475a-b699-2ae84dacb970 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:08,280 [555] [DEBUG] [app] Starting request: urn:request:ac38fd76-2877-475a-b699-2ae84dacb970 (/v1/_internal_ping) 2020-03-31 09:03:08,281 [555] [DEBUG] [app] Ending request: urn:request:ac38fd76-2877-475a-b699-2ae84dacb970 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:08,281 [555] [DEBUG] [app] Ending request: urn:request:ac38fd76-2877-475a-b699-2ae84dacb970 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:08,281 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:08,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:03:08,281 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:08,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:08,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:03:08,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:08,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:08,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:08,285 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:08,285 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:08,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:08,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:08,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:08,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:08,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:08,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:08,311 [527] [DEBUG] [app] Starting request: urn:request:b4a65ab5-31a3-4836-9636-0ca2f1fb06e2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:08,311 [527] [DEBUG] [app] Starting request: urn:request:b4a65ab5-31a3-4836-9636-0ca2f1fb06e2 (/_internal_ping) 2020-03-31 09:03:08,312 [527] [DEBUG] [app] Ending request: urn:request:b4a65ab5-31a3-4836-9636-0ca2f1fb06e2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:08,312 [527] [DEBUG] [app] Ending request: urn:request:b4a65ab5-31a3-4836-9636-0ca2f1fb06e2 (/_internal_ping) 2020-03-31 09:03:08,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:08,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:08,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:08,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:03:08,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:08,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:03:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:08,324 [535] [DEBUG] [app] Starting request: urn:request:46a2ed94-62d1-47bb-b129-755a23ae1650 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:08,324 [535] [DEBUG] [app] Starting request: urn:request:46a2ed94-62d1-47bb-b129-755a23ae1650 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:08,325 [535] [DEBUG] [app] Ending request: urn:request:46a2ed94-62d1-47bb-b129-755a23ae1650 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:08,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:08,326 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:03:08,325 [535] [DEBUG] [app] Ending request: urn:request:46a2ed94-62d1-47bb-b129-755a23ae1650 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:08,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:03:08,326 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:08,327 [527] [DEBUG] [app] Ending request: urn:request:b4c6f161-eed4-412d-bc21-b20b235a1b7b (/health/instance) 2020-03-31 09:03:08,327 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:08,327 [527] [DEBUG] [app] Ending request: urn:request:b4c6f161-eed4-412d-bc21-b20b235a1b7b (/health/instance) 2020-03-31 09:03:08,327 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:08,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:03:08,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:03:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:03:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:21.440707+00:00 (in 9.998983 seconds) 2020-03-31 09:03:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:21 UTC)" (scheduled at 2020-03-31 09:03:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:03:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:21.440707+00:00 (in 9.998983 seconds) 2020-03-31 09:03:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:21 UTC)" (scheduled at 2020-03-31 09:03:11.440707+00:00) 2020-03-31 09:03:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:03:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:03:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:03:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:03:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:03:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:21 UTC)" executed successfully 2020-03-31 09:03:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:03:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:24.323352+00:00 (in 9.999097 seconds) 2020-03-31 09:03:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:24 UTC)" (scheduled at 2020-03-31 09:03:14.323352+00:00) 2020-03-31 09:03:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:03:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:24.323352+00:00 (in 9.999097 seconds) 2020-03-31 09:03:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:24 UTC)" (scheduled at 2020-03-31 09:03:14.323352+00:00) 2020-03-31 09:03:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 14, 325282), True, datetime.datetime(2020, 3, 31, 9, 3, 14, 325282), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:03:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 14, 325282), True, datetime.datetime(2020, 3, 31, 9, 3, 14, 325282), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:03:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:03:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:03:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:24 UTC)" executed successfully 2020-03-31 09:03:14,446 [526] [DEBUG] [app] Starting request: urn:request:639ad4c8-085b-4d8f-ab06-9dad7816cab7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:14,446 [526] [DEBUG] [app] Starting request: urn:request:639ad4c8-085b-4d8f-ab06-9dad7816cab7 (/health/instance) 2020-03-31 09:03:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:14,453 [548] [DEBUG] [app] Starting request: urn:request:a3429dcd-7d64-428b-b53f-563fee949d0f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:14,453 [548] [DEBUG] [app] Starting request: urn:request:a3429dcd-7d64-428b-b53f-563fee949d0f (/v1/_internal_ping) 2020-03-31 09:03:14,454 [548] [DEBUG] [app] Ending request: urn:request:a3429dcd-7d64-428b-b53f-563fee949d0f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:14,454 [548] [DEBUG] [app] Ending request: urn:request:a3429dcd-7d64-428b-b53f-563fee949d0f (/v1/_internal_ping) 2020-03-31 09:03:14,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:03:14,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:03:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:14,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:14,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:14,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:14,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:14,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:14,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:14,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:14,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:14,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:14,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:14,486 [526] [DEBUG] [app] Starting request: urn:request:7fd640c1-fcff-45e2-9c09-ea9308f13cbf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:14,486 [526] [DEBUG] [app] Starting request: urn:request:7fd640c1-fcff-45e2-9c09-ea9308f13cbf (/_internal_ping) 2020-03-31 09:03:14,487 [526] [DEBUG] [app] Ending request: urn:request:7fd640c1-fcff-45e2-9c09-ea9308f13cbf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:14,487 [526] [DEBUG] [app] Ending request: urn:request:7fd640c1-fcff-45e2-9c09-ea9308f13cbf (/_internal_ping) 2020-03-31 09:03:14,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:14,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:14,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:14,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:14,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:14,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:14,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:14,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:03:14,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:14,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:14,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:14,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:14,500 [543] [DEBUG] [app] Starting request: urn:request:f0ae7627-25f9-4f04-96b7-ae1f3db1e85a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:14,500 [543] [DEBUG] [app] Starting request: urn:request:f0ae7627-25f9-4f04-96b7-ae1f3db1e85a (/c1/_internal_ping) 2020-03-31 09:03:14,501 [543] [DEBUG] [app] Ending request: urn:request:f0ae7627-25f9-4f04-96b7-ae1f3db1e85a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:14,501 [543] [DEBUG] [app] Ending request: urn:request:f0ae7627-25f9-4f04-96b7-ae1f3db1e85a (/c1/_internal_ping) 2020-03-31 09:03:14,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:14,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:14,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:14,503 [526] [DEBUG] [app] Ending request: urn:request:639ad4c8-085b-4d8f-ab06-9dad7816cab7 (/health/instance) 2020-03-31 09:03:14,503 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:03:14,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:03:14,503 [526] [DEBUG] [app] Ending request: urn:request:639ad4c8-085b-4d8f-ab06-9dad7816cab7 (/health/instance) 2020-03-31 09:03:14,503 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:14,504 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:03:14,504 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:03:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:03:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:45.064514+00:00 (in 29.999124 seconds) 2020-03-31 09:03:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:45 UTC)" (scheduled at 2020-03-31 09:03:15.064514+00:00) gcworker stdout | 2020-03-31 09:03:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:45.064514+00:00 (in 29.999124 seconds) 2020-03-31 09:03:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:45 UTC)" (scheduled at 2020-03-31 09:03:15.064514+00:00) 2020-03-31 09:03:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435795, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:03:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435795, 1209600, 500, 1, 0]) 2020-03-31 09:03:15,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:03:15,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:03:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:03:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:45 UTC)" executed successfully 2020-03-31 09:03:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:26.924497+00:00 (in 9.999257 seconds) 2020-03-31 09:03:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:26 UTC)" (scheduled at 2020-03-31 09:03:16.924497+00:00) 2020-03-31 09:03:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:03:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:26.924497+00:00 (in 9.999257 seconds) 2020-03-31 09:03:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:26 UTC)" (scheduled at 2020-03-31 09:03:16.924497+00:00) 2020-03-31 09:03:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 16, 926084), True, datetime.datetime(2020, 3, 31, 9, 3, 16, 926084), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:03:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 16, 926084), True, datetime.datetime(2020, 3, 31, 9, 3, 16, 926084), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:03:16,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:03:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:03:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:26 UTC)" executed successfully 2020-03-31 09:03:18,273 [518] [DEBUG] [app] Starting request: urn:request:b0cf0767-dc10-4b86-874b-029712e9ddf1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:18,273 [518] [DEBUG] [app] Starting request: urn:request:b0cf0767-dc10-4b86-874b-029712e9ddf1 (/health/instance) 2020-03-31 09:03:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:18,281 [551] [DEBUG] [app] Starting request: urn:request:8d971248-478d-4fff-abb0-5af8ade7cf89 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:18,281 [551] [DEBUG] [app] Starting request: urn:request:8d971248-478d-4fff-abb0-5af8ade7cf89 (/v1/_internal_ping) 2020-03-31 09:03:18,282 [551] [DEBUG] [app] Ending request: urn:request:8d971248-478d-4fff-abb0-5af8ade7cf89 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:18,282 [551] [DEBUG] [app] Ending request: urn:request:8d971248-478d-4fff-abb0-5af8ade7cf89 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:18,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:03:18,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:18,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:03:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:03:18,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:18,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:18,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:18,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:18,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:18,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:18,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:18,313 [527] [DEBUG] [app] Starting request: urn:request:1c4e7b2a-509d-4beb-99fc-50b2da56d6ab (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:18,313 [527] [DEBUG] [app] Starting request: urn:request:1c4e7b2a-509d-4beb-99fc-50b2da56d6ab (/_internal_ping) 2020-03-31 09:03:18,314 [527] [DEBUG] [app] Ending request: urn:request:1c4e7b2a-509d-4beb-99fc-50b2da56d6ab (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:18,314 [527] [DEBUG] [app] Ending request: urn:request:1c4e7b2a-509d-4beb-99fc-50b2da56d6ab (/_internal_ping) 2020-03-31 09:03:18,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:18,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:03:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:03:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:18,329 [543] [DEBUG] [app] Starting request: urn:request:5461ec3a-14e8-4e94-8ca2-aa5d0d1ff658 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:18,329 [543] [DEBUG] [app] Starting request: urn:request:5461ec3a-14e8-4e94-8ca2-aa5d0d1ff658 (/c1/_internal_ping) 2020-03-31 09:03:18,330 [543] [DEBUG] [app] Ending request: urn:request:5461ec3a-14e8-4e94-8ca2-aa5d0d1ff658 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:18,330 [543] [DEBUG] [app] Ending request: urn:request:5461ec3a-14e8-4e94-8ca2-aa5d0d1ff658 (/c1/_internal_ping) 2020-03-31 09:03:18,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:03:18,332 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:18,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:03:18,333 [518] [DEBUG] [app] Ending request: urn:request:b0cf0767-dc10-4b86-874b-029712e9ddf1 (/health/instance) 2020-03-31 09:03:18,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:03:18,332 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:18,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:03:18,333 [518] [DEBUG] [app] Ending request: urn:request:b0cf0767-dc10-4b86-874b-029712e9ddf1 (/health/instance) 2020-03-31 09:03:18,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:18,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:03:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:03:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:31.440707+00:00 (in 9.999085 seconds) 2020-03-31 09:03:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:31 UTC)" (scheduled at 2020-03-31 09:03:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:03:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:31.440707+00:00 (in 9.999085 seconds) 2020-03-31 09:03:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:31 UTC)" (scheduled at 2020-03-31 09:03:21.440707+00:00) 2020-03-31 09:03:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:03:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:03:21,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:03:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:03:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:03:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:31 UTC)" executed successfully 2020-03-31 09:03:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:03:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:52.113221+00:00 (in 29.997821 seconds) 2020-03-31 09:03:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:52 UTC)" (scheduled at 2020-03-31 09:03:22.113221+00:00) securityworker stdout | 2020-03-31 09:03:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:52.113221+00:00 (in 29.997821 seconds) 2020-03-31 09:03:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:52 UTC)" (scheduled at 2020-03-31 09:03:22.113221+00:00) 2020-03-31 09:03:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:03:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:03:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:03:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:03:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:03:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:03:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:03:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:03:52 UTC)" executed successfully 2020-03-31 09:03:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:03:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:34.323352+00:00 (in 9.998926 seconds) 2020-03-31 09:03:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:34 UTC)" (scheduled at 2020-03-31 09:03:24.323352+00:00) 2020-03-31 09:03:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:03:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:34.323352+00:00 (in 9.998926 seconds) 2020-03-31 09:03:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:34 UTC)" (scheduled at 2020-03-31 09:03:24.323352+00:00) 2020-03-31 09:03:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 24, 325337), True, datetime.datetime(2020, 3, 31, 9, 3, 24, 325337), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:03:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 24, 325337), True, datetime.datetime(2020, 3, 31, 9, 3, 24, 325337), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:03:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:03:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:03:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:34 UTC)" executed successfully 2020-03-31 09:03:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:03:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:24 UTC)" (scheduled at 2020-03-31 09:03:24.410927+00:00) 2020-03-31 09:03:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:03:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:24 UTC)" (scheduled at 2020-03-31 09:03:24.410927+00:00) 2020-03-31 09:03:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:33.330112+00:00 (in 8.918180 seconds) 2020-03-31 09:03:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:03:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:33.330112+00:00 (in 8.918180 seconds) 2020-03-31 09:03:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:24 UTC)" executed successfully 2020-03-31 09:03:24,445 [518] [DEBUG] [app] Starting request: urn:request:9c45485f-e40d-4fbb-8c51-76ea12a7a53d (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:24,445 [518] [DEBUG] [app] Starting request: urn:request:9c45485f-e40d-4fbb-8c51-76ea12a7a53d (/health/instance) 2020-03-31 09:03:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:24,453 [550] [DEBUG] [app] Starting request: urn:request:802db20f-0b19-4eea-8eba-fe73923abcea (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:24,453 [550] [DEBUG] [app] Starting request: urn:request:802db20f-0b19-4eea-8eba-fe73923abcea (/v1/_internal_ping) 2020-03-31 09:03:24,455 [550] [DEBUG] [app] Ending request: urn:request:802db20f-0b19-4eea-8eba-fe73923abcea (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:24,455 [550] [DEBUG] [app] Ending request: urn:request:802db20f-0b19-4eea-8eba-fe73923abcea (/v1/_internal_ping) 2020-03-31 09:03:24,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:03:24,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:03:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:24,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:24,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:24,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:24,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:24,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:24,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:24,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:24,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:24,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:24,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:24,494 [518] [DEBUG] [app] Starting request: urn:request:5b53dc8e-6224-46f8-8c8e-d3cb12fa0aee (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:24,494 [518] [DEBUG] [app] Starting request: urn:request:5b53dc8e-6224-46f8-8c8e-d3cb12fa0aee (/_internal_ping) 2020-03-31 09:03:24,495 [518] [DEBUG] [app] Ending request: urn:request:5b53dc8e-6224-46f8-8c8e-d3cb12fa0aee (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:24,495 [518] [DEBUG] [app] Ending request: urn:request:5b53dc8e-6224-46f8-8c8e-d3cb12fa0aee (/_internal_ping) 2020-03-31 09:03:24,496 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:03:24,496 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:03:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:03:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:03:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:24,509 [543] [DEBUG] [app] Starting request: urn:request:8ba1bbea-eba0-4585-8c22-870562eb2a10 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:24,509 [543] [DEBUG] [app] Starting request: urn:request:8ba1bbea-eba0-4585-8c22-870562eb2a10 (/c1/_internal_ping) 2020-03-31 09:03:24,510 [543] [DEBUG] [app] Ending request: urn:request:8ba1bbea-eba0-4585-8c22-870562eb2a10 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:24,510 [543] [DEBUG] [app] Ending request: urn:request:8ba1bbea-eba0-4585-8c22-870562eb2a10 (/c1/_internal_ping) 2020-03-31 09:03:24,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:24,511 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:24,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:24,512 [518] [DEBUG] [app] Ending request: urn:request:9c45485f-e40d-4fbb-8c51-76ea12a7a53d (/health/instance) 2020-03-31 09:03:24,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:03:24,511 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:24,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:03:24,512 [518] [DEBUG] [app] Ending request: urn:request:9c45485f-e40d-4fbb-8c51-76ea12a7a53d (/health/instance) 2020-03-31 09:03:24,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:24,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:03:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:03:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:36.924497+00:00 (in 9.999202 seconds) 2020-03-31 09:03:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:36 UTC)" (scheduled at 2020-03-31 09:03:26.924497+00:00) 2020-03-31 09:03:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:03:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:36.924497+00:00 (in 9.999202 seconds) 2020-03-31 09:03:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:36 UTC)" (scheduled at 2020-03-31 09:03:26.924497+00:00) 2020-03-31 09:03:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 26, 925908), True, datetime.datetime(2020, 3, 31, 9, 3, 26, 925908), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:03:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 26, 925908), True, datetime.datetime(2020, 3, 31, 9, 3, 26, 925908), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:03:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:03:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:03:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:36 UTC)" executed successfully 2020-03-31 09:03:28,273 [527] [DEBUG] [app] Starting request: urn:request:7b7a11da-d87f-4ff3-8784-ebe4924025fb (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:28,273 [527] [DEBUG] [app] Starting request: urn:request:7b7a11da-d87f-4ff3-8784-ebe4924025fb (/health/instance) 2020-03-31 09:03:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:28,282 [538] [DEBUG] [app] Starting request: urn:request:2386b1fa-f571-43f7-867b-7efc84f8a95d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:28,282 [538] [DEBUG] [app] Starting request: urn:request:2386b1fa-f571-43f7-867b-7efc84f8a95d (/v1/_internal_ping) 2020-03-31 09:03:28,283 [538] [DEBUG] [app] Ending request: urn:request:2386b1fa-f571-43f7-867b-7efc84f8a95d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:28,283 [538] [DEBUG] [app] Ending request: urn:request:2386b1fa-f571-43f7-867b-7efc84f8a95d (/v1/_internal_ping) 2020-03-31 09:03:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:03:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:28,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:03:28,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:28,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:28,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:28,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:28,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:28,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:28,317 [518] [DEBUG] [app] Starting request: urn:request:82595155-4943-41a4-aed0-c2bd1abc3e12 (/_internal_ping) 2020-03-31 09:03:28,318 [518] [DEBUG] [app] Ending request: urn:request:82595155-4943-41a4-aed0-c2bd1abc3e12 (/_internal_ping) 2020-03-31 09:03:28,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:28,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:03:28,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:28,317 [518] [DEBUG] [app] Starting request: urn:request:82595155-4943-41a4-aed0-c2bd1abc3e12 (/_internal_ping) 2020-03-31 09:03:28,318 [518] [DEBUG] [app] Ending request: urn:request:82595155-4943-41a4-aed0-c2bd1abc3e12 (/_internal_ping) 2020-03-31 09:03:28,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:28,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:03:28,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:28,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:28,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:28,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:03:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:03:28,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:28,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:28,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:28,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:28,335 [543] [DEBUG] [app] Starting request: urn:request:f63fbb69-1c7e-4485-a93b-2f7c89df2b93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:28,335 [543] [DEBUG] [app] Starting request: urn:request:f63fbb69-1c7e-4485-a93b-2f7c89df2b93 (/c1/_internal_ping) 2020-03-31 09:03:28,336 [543] [DEBUG] [app] Ending request: urn:request:f63fbb69-1c7e-4485-a93b-2f7c89df2b93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:28,336 [543] [DEBUG] [app] Ending request: urn:request:f63fbb69-1c7e-4485-a93b-2f7c89df2b93 (/c1/_internal_ping) 2020-03-31 09:03:28,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:28,337 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:28,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:28,339 [527] [DEBUG] [app] Ending request: urn:request:7b7a11da-d87f-4ff3-8784-ebe4924025fb (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:03:28,337 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:28,339 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:28,339 [527] [DEBUG] [app] Ending request: urn:request:7b7a11da-d87f-4ff3-8784-ebe4924025fb (/health/instance) 2020-03-31 09:03:28,340 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:03:28,339 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:28,340 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:03:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:03:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:00 UTC)" (scheduled at 2020-03-31 09:03:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:03:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:00 UTC)" (scheduled at 2020-03-31 09:03:30.337699+00:00) 2020-03-31 09:03:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 3, 30, 341975), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:03:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 3, 30, 341975), False, 50, 1, 0]) 2020-03-31 09:03:30,344 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:00.337699+00:00 (in 29.996579 seconds) buildlogsarchiver stdout | 2020-03-31 09:03:30,344 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:00.337699+00:00 (in 29.996579 seconds) 2020-03-31 09:03:30,360 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:03:30,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:30,361 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:03:30,360 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:03:30,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:30,361 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:00 UTC)" executed successfully 2020-03-31 09:03:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:03:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:41.440707+00:00 (in 9.999269 seconds) 2020-03-31 09:03:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:41 UTC)" (scheduled at 2020-03-31 09:03:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:03:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:41.440707+00:00 (in 9.999269 seconds) 2020-03-31 09:03:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:41 UTC)" (scheduled at 2020-03-31 09:03:31.440707+00:00) 2020-03-31 09:03:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:03:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:03:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:03:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:03:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:03:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:41 UTC)" executed successfully 2020-03-31 09:03:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:03:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:03.330112+00:00 (in 29.998582 seconds) 2020-03-31 09:03:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:03 UTC)" (scheduled at 2020-03-31 09:03:33.330112+00:00) 2020-03-31 09:03:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:03:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:03.330112+00:00 (in 29.998582 seconds) 2020-03-31 09:03:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:03 UTC)" (scheduled at 2020-03-31 09:03:33.330112+00:00) 2020-03-31 09:03:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 33, 332338), True, datetime.datetime(2020, 3, 31, 9, 3, 33, 332338), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:03:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 33, 332338), True, datetime.datetime(2020, 3, 31, 9, 3, 33, 332338), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:03:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:03:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:03:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:03 UTC)" executed successfully 2020-03-31 09:03:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:03:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:43.539999+00:00 (in 10.171763 seconds) 2020-03-31 09:03:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:33 UTC)" (scheduled at 2020-03-31 09:03:33.366791+00:00) 2020-03-31 09:03:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:03:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:43.539999+00:00 (in 10.171763 seconds) 2020-03-31 09:03:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:33 UTC)" (scheduled at 2020-03-31 09:03:33.366791+00:00) 2020-03-31 09:03:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 33, 369293), True, datetime.datetime(2020, 3, 31, 9, 3, 33, 369293), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:03:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 33, 369293), True, datetime.datetime(2020, 3, 31, 9, 3, 33, 369293), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:03:33,386 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:03:33,386 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:03:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:33 UTC)" executed successfully 2020-03-31 09:03:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:03:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:44.323352+00:00 (in 9.999222 seconds) 2020-03-31 09:03:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:44 UTC)" (scheduled at 2020-03-31 09:03:34.323352+00:00) 2020-03-31 09:03:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:03:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:44.323352+00:00 (in 9.999222 seconds) 2020-03-31 09:03:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:44 UTC)" (scheduled at 2020-03-31 09:03:34.323352+00:00) 2020-03-31 09:03:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 34, 325128), True, datetime.datetime(2020, 3, 31, 9, 3, 34, 325128), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:03:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 34, 325128), True, datetime.datetime(2020, 3, 31, 9, 3, 34, 325128), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:03:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:03:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:03:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:44 UTC)" executed successfully 2020-03-31 09:03:34,446 [526] [DEBUG] [app] Starting request: urn:request:ec2da698-35ef-459b-a8a9-c12d7781ae86 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:34,446 [526] [DEBUG] [app] Starting request: urn:request:ec2da698-35ef-459b-a8a9-c12d7781ae86 (/health/instance) 2020-03-31 09:03:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:34,454 [551] [DEBUG] [app] Starting request: urn:request:2c1021cd-531d-45bd-a8c8-07f0dd8eccc3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:34,454 [551] [DEBUG] [app] Starting request: urn:request:2c1021cd-531d-45bd-a8c8-07f0dd8eccc3 (/v1/_internal_ping) 2020-03-31 09:03:34,456 [551] [DEBUG] [app] Ending request: urn:request:2c1021cd-531d-45bd-a8c8-07f0dd8eccc3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:34,456 [551] [DEBUG] [app] Ending request: urn:request:2c1021cd-531d-45bd-a8c8-07f0dd8eccc3 (/v1/_internal_ping) 2020-03-31 09:03:34,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:03:34,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:34,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:03:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:03:34,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:34,461 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:34,461 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:34,462 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:34,462 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:34,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:34,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:34,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:34,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:34,486 [527] [DEBUG] [app] Starting request: urn:request:06753bcd-dd00-40f7-acbc-f2539a25c036 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:34,486 [527] [DEBUG] [app] Starting request: urn:request:06753bcd-dd00-40f7-acbc-f2539a25c036 (/_internal_ping) 2020-03-31 09:03:34,487 [527] [DEBUG] [app] Ending request: urn:request:06753bcd-dd00-40f7-acbc-f2539a25c036 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:34,487 [527] [DEBUG] [app] Ending request: urn:request:06753bcd-dd00-40f7-acbc-f2539a25c036 (/_internal_ping) 2020-03-31 09:03:34,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:34,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:03:34,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:34,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:03:34,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:34,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:34,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:34,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:03:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:34,502 [535] [DEBUG] [app] Starting request: urn:request:ed1b1912-d3fd-4cbb-a1c9-cd2426066222 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:34,502 [535] [DEBUG] [app] Starting request: urn:request:ed1b1912-d3fd-4cbb-a1c9-cd2426066222 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:34,503 [535] [DEBUG] [app] Ending request: urn:request:ed1b1912-d3fd-4cbb-a1c9-cd2426066222 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:34,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:34,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:03:34,503 [535] [DEBUG] [app] Ending request: urn:request:ed1b1912-d3fd-4cbb-a1c9-cd2426066222 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:34,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:03:34,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:34,505 [526] [DEBUG] [app] Ending request: urn:request:ec2da698-35ef-459b-a8a9-c12d7781ae86 (/health/instance) 2020-03-31 09:03:34,505 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:34,505 [526] [DEBUG] [app] Ending request: urn:request:ec2da698-35ef-459b-a8a9-c12d7781ae86 (/health/instance) 2020-03-31 09:03:34,505 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:34,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:03:34,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:03:36,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:03:36,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:38.010934+00:00 (in 1.085707 seconds) 2020-03-31 09:03:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:46 UTC)" (scheduled at 2020-03-31 09:03:36.924497+00:00) 2020-03-31 09:03:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:03:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:38.010934+00:00 (in 1.085707 seconds) 2020-03-31 09:03:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:46 UTC)" (scheduled at 2020-03-31 09:03:36.924497+00:00) 2020-03-31 09:03:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 36, 925887), True, datetime.datetime(2020, 3, 31, 9, 3, 36, 925887), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:03:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 36, 925887), True, datetime.datetime(2020, 3, 31, 9, 3, 36, 925887), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:03:36,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:03:36,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:03:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:46 UTC)" executed successfully 2020-03-31 09:03:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:03:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:38 UTC)" (scheduled at 2020-03-31 09:03:38.010934+00:00) 2020-03-31 09:03:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:46.924497+00:00 (in 8.912879 seconds) storagereplication stdout | 2020-03-31 09:03:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:38 UTC)" (scheduled at 2020-03-31 09:03:38.010934+00:00) 2020-03-31 09:03:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:46.924497+00:00 (in 8.912879 seconds) 2020-03-31 09:03:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:03:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:38 UTC)" executed successfully 2020-03-31 09:03:38,272 [518] [DEBUG] [app] Starting request: urn:request:b898e460-fc02-4b0a-8cfa-517fefa74751 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:38,272 [518] [DEBUG] [app] Starting request: urn:request:b898e460-fc02-4b0a-8cfa-517fefa74751 (/health/instance) 2020-03-31 09:03:38,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:38,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:38,280 [548] [DEBUG] [app] Starting request: urn:request:a6bfc62b-c414-461f-b9e2-e59b4fda9b4a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:38,280 [548] [DEBUG] [app] Starting request: urn:request:a6bfc62b-c414-461f-b9e2-e59b4fda9b4a (/v1/_internal_ping) 2020-03-31 09:03:38,281 [548] [DEBUG] [app] Ending request: urn:request:a6bfc62b-c414-461f-b9e2-e59b4fda9b4a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:38,281 [548] [DEBUG] [app] Ending request: urn:request:a6bfc62b-c414-461f-b9e2-e59b4fda9b4a (/v1/_internal_ping) 2020-03-31 09:03:38,281 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:03:38,281 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:03:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:38,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:38,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:38,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:38,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:38,287 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:38,287 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:38,313 [525] [DEBUG] [app] Starting request: urn:request:7a3bb081-f10f-4f62-b3bf-a67ef2057c90 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:38,313 [525] [DEBUG] [app] Starting request: urn:request:7a3bb081-f10f-4f62-b3bf-a67ef2057c90 (/_internal_ping) 2020-03-31 09:03:38,314 [525] [DEBUG] [app] Ending request: urn:request:7a3bb081-f10f-4f62-b3bf-a67ef2057c90 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:38,314 [525] [DEBUG] [app] Ending request: urn:request:7a3bb081-f10f-4f62-b3bf-a67ef2057c90 (/_internal_ping) 2020-03-31 09:03:38,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:38,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:03:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:03:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:03:38,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:38,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:38,330 [537] [DEBUG] [app] Starting request: urn:request:d4125910-e706-4b62-9726-942a5a8087ab (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:38,330 [537] [DEBUG] [app] Starting request: urn:request:d4125910-e706-4b62-9726-942a5a8087ab (/c1/_internal_ping) 2020-03-31 09:03:38,330 [537] [DEBUG] [app] Ending request: urn:request:d4125910-e706-4b62-9726-942a5a8087ab (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:38,330 [537] [DEBUG] [app] Ending request: urn:request:d4125910-e706-4b62-9726-942a5a8087ab (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:38,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:38,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:03:38,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:38,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:38,332 [518] [DEBUG] [app] Ending request: urn:request:b898e460-fc02-4b0a-8cfa-517fefa74751 (/health/instance) 2020-03-31 09:03:38,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:38,332 [518] [DEBUG] [app] Ending request: urn:request:b898e460-fc02-4b0a-8cfa-517fefa74751 (/health/instance) 2020-03-31 09:03:38,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:38,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:03:38,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:03:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:03:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:51.440707+00:00 (in 9.999245 seconds) 2020-03-31 09:03:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:51 UTC)" (scheduled at 2020-03-31 09:03:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:03:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:51.440707+00:00 (in 9.999245 seconds) 2020-03-31 09:03:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:51 UTC)" (scheduled at 2020-03-31 09:03:41.440707+00:00) 2020-03-31 09:03:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:03:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:03:41,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:03:41,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:03:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:03:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:51 UTC)" executed successfully 2020-03-31 09:03:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:03:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:33.366791+00:00 (in 49.825860 seconds) 2020-03-31 09:03:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:43 UTC)" (scheduled at 2020-03-31 09:03:43.539999+00:00) 2020-03-31 09:03:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:03:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:33.366791+00:00 (in 49.825860 seconds) 2020-03-31 09:03:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:43 UTC)" (scheduled at 2020-03-31 09:03:43.539999+00:00) 2020-03-31 09:03:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:43 UTC)" executed successfully 2020-03-31 09:03:44,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:03:44,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:49.498966+00:00 (in 5.172524 seconds) 2020-03-31 09:03:44,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:54 UTC)" (scheduled at 2020-03-31 09:03:44.323352+00:00) 2020-03-31 09:03:44,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:03:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:49.498966+00:00 (in 5.172524 seconds) 2020-03-31 09:03:44,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:54 UTC)" (scheduled at 2020-03-31 09:03:44.323352+00:00) 2020-03-31 09:03:44,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:44,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 44, 327392), True, datetime.datetime(2020, 3, 31, 9, 3, 44, 327392), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:03:44,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 44, 327392), True, datetime.datetime(2020, 3, 31, 9, 3, 44, 327392), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:03:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:03:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:03:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:54 UTC)" executed successfully 2020-03-31 09:03:44,446 [518] [DEBUG] [app] Starting request: urn:request:48a40c58-ec4f-42ba-b7fd-7bbaf217330a (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:44,446 [518] [DEBUG] [app] Starting request: urn:request:48a40c58-ec4f-42ba-b7fd-7bbaf217330a (/health/instance) 2020-03-31 09:03:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:44,454 [538] [DEBUG] [app] Starting request: urn:request:3a54cc71-201e-4832-a0f2-22b3f0aedc02 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:44,454 [538] [DEBUG] [app] Starting request: urn:request:3a54cc71-201e-4832-a0f2-22b3f0aedc02 (/v1/_internal_ping) 2020-03-31 09:03:44,455 [538] [DEBUG] [app] Ending request: urn:request:3a54cc71-201e-4832-a0f2-22b3f0aedc02 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:44,455 [538] [DEBUG] [app] Ending request: urn:request:3a54cc71-201e-4832-a0f2-22b3f0aedc02 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:44,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:03:44,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:03:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:44,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:44,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:44,489 [518] [DEBUG] [app] Starting request: urn:request:1c0bea29-427c-4a9a-b79f-ca37c3f0a781 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:44,489 [518] [DEBUG] [app] Starting request: urn:request:1c0bea29-427c-4a9a-b79f-ca37c3f0a781 (/_internal_ping) 2020-03-31 09:03:44,490 [518] [DEBUG] [app] Ending request: urn:request:1c0bea29-427c-4a9a-b79f-ca37c3f0a781 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:44,490 [518] [DEBUG] [app] Ending request: urn:request:1c0bea29-427c-4a9a-b79f-ca37c3f0a781 (/_internal_ping) 2020-03-31 09:03:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:03:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:44,502 [537] [DEBUG] [app] Starting request: urn:request:0da5c04c-b5bd-48e3-92fd-2de7040c1622 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:44,502 [537] [DEBUG] [app] Starting request: urn:request:0da5c04c-b5bd-48e3-92fd-2de7040c1622 (/c1/_internal_ping) 2020-03-31 09:03:44,504 [537] [DEBUG] [app] Ending request: urn:request:0da5c04c-b5bd-48e3-92fd-2de7040c1622 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:44,504 [537] [DEBUG] [app] Ending request: urn:request:0da5c04c-b5bd-48e3-92fd-2de7040c1622 (/c1/_internal_ping) 2020-03-31 09:03:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:44,504 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:44,506 [518] [DEBUG] [app] Ending request: urn:request:48a40c58-ec4f-42ba-b7fd-7bbaf217330a (/health/instance) 2020-03-31 09:03:44,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:03:44,504 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:44,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:03:44,506 [518] [DEBUG] [app] Ending request: urn:request:48a40c58-ec4f-42ba-b7fd-7bbaf217330a (/health/instance) 2020-03-31 09:03:44,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:44,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:03:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:03:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:15.064514+00:00 (in 29.999163 seconds) 2020-03-31 09:03:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:15 UTC)" (scheduled at 2020-03-31 09:03:45.064514+00:00) gcworker stdout | 2020-03-31 09:03:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:15.064514+00:00 (in 29.999163 seconds) 2020-03-31 09:03:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:15 UTC)" (scheduled at 2020-03-31 09:03:45.064514+00:00) 2020-03-31 09:03:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435825, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:03:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435825, 1209600, 500, 1, 0]) 2020-03-31 09:03:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:03:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:03:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:03:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:15 UTC)" executed successfully 2020-03-31 09:03:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:03:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:56.924497+00:00 (in 9.998542 seconds) 2020-03-31 09:03:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:56 UTC)" (scheduled at 2020-03-31 09:03:46.924497+00:00) 2020-03-31 09:03:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:03:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:56.924497+00:00 (in 9.998542 seconds) 2020-03-31 09:03:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:56 UTC)" (scheduled at 2020-03-31 09:03:46.924497+00:00) 2020-03-31 09:03:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 46, 926712), True, datetime.datetime(2020, 3, 31, 9, 3, 46, 926712), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:03:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 46, 926712), True, datetime.datetime(2020, 3, 31, 9, 3, 46, 926712), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:03:46,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:03:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:03:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:03:56 UTC)" executed successfully 2020-03-31 09:03:48,273 [518] [DEBUG] [app] Starting request: urn:request:09af8d11-fd9f-4f4c-a2eb-157b2e31276e (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:48,273 [518] [DEBUG] [app] Starting request: urn:request:09af8d11-fd9f-4f4c-a2eb-157b2e31276e (/health/instance) 2020-03-31 09:03:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:48,278 [545] [DEBUG] [app] Starting request: urn:request:14789b2d-2333-4543-9fd1-1f478bc2e1b2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:48,278 [545] [DEBUG] [app] Starting request: urn:request:14789b2d-2333-4543-9fd1-1f478bc2e1b2 (/v1/_internal_ping) 2020-03-31 09:03:48,280 [545] [DEBUG] [app] Ending request: urn:request:14789b2d-2333-4543-9fd1-1f478bc2e1b2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:48,280 [545] [DEBUG] [app] Ending request: urn:request:14789b2d-2333-4543-9fd1-1f478bc2e1b2 (/v1/_internal_ping) 2020-03-31 09:03:48,280 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:03:48,280 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:03:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:48,283 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:48,283 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:48,285 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:48,285 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:48,286 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:48,286 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:48,310 [519] [DEBUG] [app] Starting request: urn:request:b56c69c0-7b5d-4b41-9c6b-05474dd6d586 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:48,310 [519] [DEBUG] [app] Starting request: urn:request:b56c69c0-7b5d-4b41-9c6b-05474dd6d586 (/_internal_ping) 2020-03-31 09:03:48,311 [519] [DEBUG] [app] Ending request: urn:request:b56c69c0-7b5d-4b41-9c6b-05474dd6d586 (/_internal_ping) 2020-03-31 09:03:48,312 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:48,311 [519] [DEBUG] [app] Ending request: urn:request:b56c69c0-7b5d-4b41-9c6b-05474dd6d586 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:48,312 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:03:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:03:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:03:48,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:48,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:48,323 [543] [DEBUG] [app] Starting request: urn:request:676992ce-c7a6-4538-94f4-ce8c926b44ad (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:48,323 [543] [DEBUG] [app] Starting request: urn:request:676992ce-c7a6-4538-94f4-ce8c926b44ad (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:48,324 [543] [DEBUG] [app] Ending request: urn:request:676992ce-c7a6-4538-94f4-ce8c926b44ad (/c1/_internal_ping) 2020-03-31 09:03:48,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:03:48,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:03:48,324 [543] [DEBUG] [app] Ending request: urn:request:676992ce-c7a6-4538-94f4-ce8c926b44ad (/c1/_internal_ping) 2020-03-31 09:03:48,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:48,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:48,326 [518] [DEBUG] [app] Ending request: urn:request:09af8d11-fd9f-4f4c-a2eb-157b2e31276e (/health/instance) 2020-03-31 09:03:48,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:48,326 [518] [DEBUG] [app] Ending request: urn:request:09af8d11-fd9f-4f4c-a2eb-157b2e31276e (/health/instance) 2020-03-31 09:03:48,326 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:48,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:03:48,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:03:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:03:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:54.323352+00:00 (in 4.823413 seconds) 2020-03-31 09:03:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:49 UTC)" (scheduled at 2020-03-31 09:03:49.498966+00:00) 2020-03-31 09:03:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:03:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:54.323352+00:00 (in 4.823413 seconds) 2020-03-31 09:03:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:49 UTC)" (scheduled at 2020-03-31 09:03:49.498966+00:00) 2020-03-31 09:03:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:49 UTC)" executed successfully 2020-03-31 09:03:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:03:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:59.560326+00:00 (in 8.448809 seconds) 2020-03-31 09:03:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:51 UTC)" (scheduled at 2020-03-31 09:03:51.110610+00:00) 2020-03-31 09:03:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:03:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:03:59.560326+00:00 (in 8.448809 seconds) 2020-03-31 09:03:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:51 UTC)" (scheduled at 2020-03-31 09:03:51.110610+00:00) 2020-03-31 09:03:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:51,113 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 51, 112274), True, datetime.datetime(2020, 3, 31, 9, 3, 51, 112274), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:03:51,113 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 51, 112274), True, datetime.datetime(2020, 3, 31, 9, 3, 51, 112274), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:03:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:03:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:51 UTC)" executed successfully 2020-03-31 09:03:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:03:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:01 UTC)" (scheduled at 2020-03-31 09:03:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:03:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:01 UTC)" (scheduled at 2020-03-31 09:03:51.440707+00:00) 2020-03-31 09:03:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:03:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:03:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:01.440707+00:00 (in 9.998986 seconds) repositoryactioncounter stdout | 2020-03-31 09:03:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:01.440707+00:00 (in 9.998986 seconds) 2020-03-31 09:03:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:03:51,461 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:03:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:03:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:03:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:01 UTC)" executed successfully 2020-03-31 09:03:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:03:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:22 UTC)" (scheduled at 2020-03-31 09:03:52.113221+00:00) securityworker stdout | 2020-03-31 09:03:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:22 UTC)" (scheduled at 2020-03-31 09:03:52.113221+00:00) 2020-03-31 09:03:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:22.113221+00:00 (in 29.999258 seconds) 2020-03-31 09:03:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:03:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:22.113221+00:00 (in 29.999258 seconds) 2020-03-31 09:03:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:03:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:03:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:03:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:03:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:03:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:03:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:22 UTC)" executed successfully 2020-03-31 09:03:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:03:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:04.323352+00:00 (in 9.998914 seconds) 2020-03-31 09:03:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:04 UTC)" (scheduled at 2020-03-31 09:03:54.323352+00:00) 2020-03-31 09:03:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:03:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:04.323352+00:00 (in 9.998914 seconds) 2020-03-31 09:03:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:04 UTC)" (scheduled at 2020-03-31 09:03:54.323352+00:00) 2020-03-31 09:03:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 54, 325407), True, datetime.datetime(2020, 3, 31, 9, 3, 54, 325407), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:03:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 54, 325407), True, datetime.datetime(2020, 3, 31, 9, 3, 54, 325407), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:03:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:03:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:03:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:04 UTC)" executed successfully 2020-03-31 09:03:54,446 [518] [DEBUG] [app] Starting request: urn:request:e623623c-b394-4dfe-bdeb-66af93cec009 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:54,446 [518] [DEBUG] [app] Starting request: urn:request:e623623c-b394-4dfe-bdeb-66af93cec009 (/health/instance) 2020-03-31 09:03:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:54,454 [538] [DEBUG] [app] Starting request: urn:request:36bcb76d-2b4d-4c76-ada5-41f2d66b8b42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:54,454 [538] [DEBUG] [app] Starting request: urn:request:36bcb76d-2b4d-4c76-ada5-41f2d66b8b42 (/v1/_internal_ping) 2020-03-31 09:03:54,455 [538] [DEBUG] [app] Ending request: urn:request:36bcb76d-2b4d-4c76-ada5-41f2d66b8b42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:54,455 [538] [DEBUG] [app] Ending request: urn:request:36bcb76d-2b4d-4c76-ada5-41f2d66b8b42 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:03:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:03:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:03:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:54,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:54,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:54,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:54,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:54,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:54,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:54,489 [518] [DEBUG] [app] Starting request: urn:request:d8a25920-8293-4927-9787-18cfe43bc006 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:54,489 [518] [DEBUG] [app] Starting request: urn:request:d8a25920-8293-4927-9787-18cfe43bc006 (/_internal_ping) 2020-03-31 09:03:54,490 [518] [DEBUG] [app] Ending request: urn:request:d8a25920-8293-4927-9787-18cfe43bc006 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:54,490 [518] [DEBUG] [app] Ending request: urn:request:d8a25920-8293-4927-9787-18cfe43bc006 (/_internal_ping) 2020-03-31 09:03:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:03:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:03:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:03:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:54,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:54,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:54,503 [543] [DEBUG] [app] Starting request: urn:request:3c1000ed-4a60-402f-b770-28292f7aa4dc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:54,503 [543] [DEBUG] [app] Starting request: urn:request:3c1000ed-4a60-402f-b770-28292f7aa4dc (/c1/_internal_ping) 2020-03-31 09:03:54,504 [543] [DEBUG] [app] Ending request: urn:request:3c1000ed-4a60-402f-b770-28292f7aa4dc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:54,504 [543] [DEBUG] [app] Ending request: urn:request:3c1000ed-4a60-402f-b770-28292f7aa4dc (/c1/_internal_ping) 2020-03-31 09:03:54,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:54,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:54,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:03:54,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:54,506 [518] [DEBUG] [app] Ending request: urn:request:e623623c-b394-4dfe-bdeb-66af93cec009 (/health/instance) 2020-03-31 09:03:54,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:03:54,506 [518] [DEBUG] [app] Ending request: urn:request:e623623c-b394-4dfe-bdeb-66af93cec009 (/health/instance) 2020-03-31 09:03:54,506 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:54,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:03:54,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:03:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:03:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:06.924497+00:00 (in 9.999240 seconds) 2020-03-31 09:03:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:06 UTC)" (scheduled at 2020-03-31 09:03:56.924497+00:00) 2020-03-31 09:03:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:03:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:06.924497+00:00 (in 9.999240 seconds) 2020-03-31 09:03:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:06 UTC)" (scheduled at 2020-03-31 09:03:56.924497+00:00) 2020-03-31 09:03:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:03:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 56, 926123), True, datetime.datetime(2020, 3, 31, 9, 3, 56, 926123), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:03:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 3, 56, 926123), True, datetime.datetime(2020, 3, 31, 9, 3, 56, 926123), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:03:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:03:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:03:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:03:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:06 UTC)" executed successfully 2020-03-31 09:03:58,273 [518] [DEBUG] [app] Starting request: urn:request:974739c2-7dd8-42b9-88f3-0f51dbebc931 (/health/instance) gunicorn-web stdout | 2020-03-31 09:03:58,273 [518] [DEBUG] [app] Starting request: urn:request:974739c2-7dd8-42b9-88f3-0f51dbebc931 (/health/instance) 2020-03-31 09:03:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:58,279 [538] [DEBUG] [app] Starting request: urn:request:b21d4c66-f422-4f37-a8f1-dfee7659fa42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:58,279 [538] [DEBUG] [app] Starting request: urn:request:b21d4c66-f422-4f37-a8f1-dfee7659fa42 (/v1/_internal_ping) 2020-03-31 09:03:58,280 [538] [DEBUG] [app] Ending request: urn:request:b21d4c66-f422-4f37-a8f1-dfee7659fa42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:03:58,280 [538] [DEBUG] [app] Ending request: urn:request:b21d4c66-f422-4f37-a8f1-dfee7659fa42 (/v1/_internal_ping) 2020-03-31 09:03:58,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:03:58,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:03:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:03:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:03:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:03:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:03:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:03:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:03:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:03:58,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:03:58,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:03:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:03:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:03:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:58,309 [518] [DEBUG] [app] Starting request: urn:request:1092d8ab-13a6-4895-b964-09210276a778 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:58,309 [518] [DEBUG] [app] Starting request: urn:request:1092d8ab-13a6-4895-b964-09210276a778 (/_internal_ping) 2020-03-31 09:03:58,310 [518] [DEBUG] [app] Ending request: urn:request:1092d8ab-13a6-4895-b964-09210276a778 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:03:58,310 [518] [DEBUG] [app] Ending request: urn:request:1092d8ab-13a6-4895-b964-09210276a778 (/_internal_ping) 2020-03-31 09:03:58,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:58,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:03:58,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:03:58,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:03:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:03:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:03:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:03:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:03:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:03:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:03:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:03:58,323 [543] [DEBUG] [app] Starting request: urn:request:520363db-1233-471b-87a3-9f935ae3b64a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:58,323 [543] [DEBUG] [app] Starting request: urn:request:520363db-1233-471b-87a3-9f935ae3b64a (/c1/_internal_ping) 2020-03-31 09:03:58,323 [543] [DEBUG] [app] Ending request: urn:request:520363db-1233-471b-87a3-9f935ae3b64a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:03:58,323 [543] [DEBUG] [app] Ending request: urn:request:520363db-1233-471b-87a3-9f935ae3b64a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:58,324 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:03:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:03:58,324 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:03:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:03:58,325 [518] [DEBUG] [app] Ending request: urn:request:974739c2-7dd8-42b9-88f3-0f51dbebc931 (/health/instance) 2020-03-31 09:03:58,325 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:58,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) gunicorn-web stdout | 2020-03-31 09:03:58,325 [518] [DEBUG] [app] Ending request: urn:request:974739c2-7dd8-42b9-88f3-0f51dbebc931 (/health/instance) 2020-03-31 09:03:58,325 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:03:58,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:03:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:03:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) 2020-03-31 09:03:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:03:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:03:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:51.110610+00:00 (in 51.549428 seconds) 2020-03-31 09:03:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:59 UTC)" (scheduled at 2020-03-31 09:03:59.560326+00:00) 2020-03-31 09:03:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:03:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:51.110610+00:00 (in 51.549428 seconds) 2020-03-31 09:03:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:59 UTC)" (scheduled at 2020-03-31 09:03:59.560326+00:00) 2020-03-31 09:03:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:03:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:04:59 UTC)" executed successfully 2020-03-31 09:04:00,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:04:00,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:30.337699+00:00 (in 29.996489 seconds) 2020-03-31 09:04:00,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:30 UTC)" (scheduled at 2020-03-31 09:04:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:04:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:30.337699+00:00 (in 29.996489 seconds) 2020-03-31 09:04:00,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:30 UTC)" (scheduled at 2020-03-31 09:04:00.337699+00:00) 2020-03-31 09:04:00,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 4, 0, 342140), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:04:00,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 4, 0, 342140), False, 50, 1, 0]) 2020-03-31 09:04:00,360 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:04:00,360 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:04:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:00,361 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:04:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:00,361 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:30 UTC)" executed successfully 2020-03-31 09:04:01,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:04:01,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:11.440707+00:00 (in 9.995417 seconds) 2020-03-31 09:04:01,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:11 UTC)" (scheduled at 2020-03-31 09:04:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:04:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:11.440707+00:00 (in 9.995417 seconds) 2020-03-31 09:04:01,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:11 UTC)" (scheduled at 2020-03-31 09:04:01.440707+00:00) 2020-03-31 09:04:01,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:04:01,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:04:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:04:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:04:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:04:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:11 UTC)" executed successfully 2020-03-31 09:04:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:03 UTC)" (scheduled at 2020-03-31 09:04:03.330112+00:00) security_notification_worker stdout | 2020-03-31 09:04:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:23.410371+00:00 (in 20.079264 seconds) 2020-03-31 09:04:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:04:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:03 UTC)" (scheduled at 2020-03-31 09:04:03.330112+00:00) security_notification_worker stdout | 2020-03-31 09:04:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:23.410371+00:00 (in 20.079264 seconds) 2020-03-31 09:04:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 3, 332639), True, datetime.datetime(2020, 3, 31, 9, 4, 3, 332639), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:04:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 3, 332639), True, datetime.datetime(2020, 3, 31, 9, 4, 3, 332639), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:04:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:04:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:04:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:33 UTC)" executed successfully 2020-03-31 09:04:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:04:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:14.323352+00:00 (in 9.998660 seconds) 2020-03-31 09:04:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:14 UTC)" (scheduled at 2020-03-31 09:04:04.323352+00:00) 2020-03-31 09:04:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:04:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:14.323352+00:00 (in 9.998660 seconds) 2020-03-31 09:04:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:14 UTC)" (scheduled at 2020-03-31 09:04:04.323352+00:00) 2020-03-31 09:04:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 4, 325979), True, datetime.datetime(2020, 3, 31, 9, 4, 4, 325979), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:04:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 4, 325979), True, datetime.datetime(2020, 3, 31, 9, 4, 4, 325979), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:04:04,349 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:04:04,349 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:04,350 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:04,350 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:04:04,350 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:04,350 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:14 UTC)" executed successfully 2020-03-31 09:04:04,446 [526] [DEBUG] [app] Starting request: urn:request:66a7334c-6a9b-42be-8409-d4196c4bcef3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:04,446 [526] [DEBUG] [app] Starting request: urn:request:66a7334c-6a9b-42be-8409-d4196c4bcef3 (/health/instance) 2020-03-31 09:04:04,450 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:04,450 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:04,454 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:04,454 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:04,455 [538] [DEBUG] [app] Starting request: urn:request:7389872d-1ca4-44ca-a972-eaf8118d43d3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:04,455 [538] [DEBUG] [app] Starting request: urn:request:7389872d-1ca4-44ca-a972-eaf8118d43d3 (/v1/_internal_ping) 2020-03-31 09:04:04,456 [538] [DEBUG] [app] Ending request: urn:request:7389872d-1ca4-44ca-a972-eaf8118d43d3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:04,456 [538] [DEBUG] [app] Ending request: urn:request:7389872d-1ca4-44ca-a972-eaf8118d43d3 (/v1/_internal_ping) 2020-03-31 09:04:04,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:04,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:04:04,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:04,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:04,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:04,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:04:04,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:04,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:04,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:04,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:04,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:04,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:04,474 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:04,474 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:04,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:04,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:04,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:04,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:04,491 [525] [DEBUG] [app] Starting request: urn:request:ac4b13b0-3b46-4bea-8980-249eb5c5b2c5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:04,491 [525] [DEBUG] [app] Starting request: urn:request:ac4b13b0-3b46-4bea-8980-249eb5c5b2c5 (/_internal_ping) 2020-03-31 09:04:04,492 [525] [DEBUG] [app] Ending request: urn:request:ac4b13b0-3b46-4bea-8980-249eb5c5b2c5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:04,492 [525] [DEBUG] [app] Ending request: urn:request:ac4b13b0-3b46-4bea-8980-249eb5c5b2c5 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:04:04,492 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:04,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:04,492 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:04,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:04,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:04,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:04,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:04,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:04:04,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:04,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:04,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:04,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:04,505 [537] [DEBUG] [app] Starting request: urn:request:423d4409-6aa6-4f20-b039-2a5785f065a8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:04,505 [537] [DEBUG] [app] Starting request: urn:request:423d4409-6aa6-4f20-b039-2a5785f065a8 (/c1/_internal_ping) 2020-03-31 09:04:04,506 [537] [DEBUG] [app] Ending request: urn:request:423d4409-6aa6-4f20-b039-2a5785f065a8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:04,506 [537] [DEBUG] [app] Ending request: urn:request:423d4409-6aa6-4f20-b039-2a5785f065a8 (/c1/_internal_ping) 2020-03-31 09:04:04,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:04,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:04,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:04:04,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:04,508 [526] [DEBUG] [app] Ending request: urn:request:66a7334c-6a9b-42be-8409-d4196c4bcef3 (/health/instance) 2020-03-31 09:04:04,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:04,508 [526] [DEBUG] [app] Ending request: urn:request:66a7334c-6a9b-42be-8409-d4196c4bcef3 (/health/instance) 2020-03-31 09:04:04,508 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:04,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:04:04,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:04:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:16.924497+00:00 (in 9.999066 seconds) 2020-03-31 09:04:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:16 UTC)" (scheduled at 2020-03-31 09:04:06.924497+00:00) 2020-03-31 09:04:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:04:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:04:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:16.924497+00:00 (in 9.999066 seconds) 2020-03-31 09:04:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:16 UTC)" (scheduled at 2020-03-31 09:04:06.924497+00:00) 2020-03-31 09:04:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 6, 926517), True, datetime.datetime(2020, 3, 31, 9, 4, 6, 926517), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:04:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:04:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 6, 926517), True, datetime.datetime(2020, 3, 31, 9, 4, 6, 926517), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:04:06,950 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:04:06,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:06,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:06,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:04:06,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:06,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:16 UTC)" executed successfully 2020-03-31 09:04:08,273 [518] [DEBUG] [app] Starting request: urn:request:71fb77e1-20a9-4597-8cee-ec0486a1de11 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:08,273 [518] [DEBUG] [app] Starting request: urn:request:71fb77e1-20a9-4597-8cee-ec0486a1de11 (/health/instance) 2020-03-31 09:04:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:08,280 [556] [DEBUG] [app] Starting request: urn:request:31f1fc7f-0c91-4785-82ff-3903aaae572c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:08,280 [556] [DEBUG] [app] Starting request: urn:request:31f1fc7f-0c91-4785-82ff-3903aaae572c (/v1/_internal_ping) 2020-03-31 09:04:08,281 [556] [DEBUG] [app] Ending request: urn:request:31f1fc7f-0c91-4785-82ff-3903aaae572c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:08,281 [556] [DEBUG] [app] Ending request: urn:request:31f1fc7f-0c91-4785-82ff-3903aaae572c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:04:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:08,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:08,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:08,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:08,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:08,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:08,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:08,314 [518] [DEBUG] [app] Starting request: urn:request:3544d214-4027-4df3-8ed6-435a03de9439 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:08,314 [518] [DEBUG] [app] Starting request: urn:request:3544d214-4027-4df3-8ed6-435a03de9439 (/_internal_ping) 2020-03-31 09:04:08,315 [518] [DEBUG] [app] Ending request: urn:request:3544d214-4027-4df3-8ed6-435a03de9439 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:08,315 [518] [DEBUG] [app] Ending request: urn:request:3544d214-4027-4df3-8ed6-435a03de9439 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:04:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:04:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:04:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:04:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:08,327 [536] [DEBUG] [app] Starting request: urn:request:4cf3f9c7-30f3-4030-abcc-4ec4d5f5e0b9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:08,327 [536] [DEBUG] [app] Starting request: urn:request:4cf3f9c7-30f3-4030-abcc-4ec4d5f5e0b9 (/c1/_internal_ping) 2020-03-31 09:04:08,328 [536] [DEBUG] [app] Ending request: urn:request:4cf3f9c7-30f3-4030-abcc-4ec4d5f5e0b9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:08,328 [536] [DEBUG] [app] Ending request: urn:request:4cf3f9c7-30f3-4030-abcc-4ec4d5f5e0b9 (/c1/_internal_ping) 2020-03-31 09:04:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:08,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:08,330 [518] [DEBUG] [app] Ending request: urn:request:71fb77e1-20a9-4597-8cee-ec0486a1de11 (/health/instance) 2020-03-31 09:04:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:04:08,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:04:08,330 [518] [DEBUG] [app] Ending request: urn:request:71fb77e1-20a9-4597-8cee-ec0486a1de11 (/health/instance) 2020-03-31 09:04:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:04:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:04:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:21.440707+00:00 (in 9.999102 seconds) 2020-03-31 09:04:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:21 UTC)" (scheduled at 2020-03-31 09:04:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:04:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:21.440707+00:00 (in 9.999102 seconds) 2020-03-31 09:04:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:21 UTC)" (scheduled at 2020-03-31 09:04:11.440707+00:00) 2020-03-31 09:04:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:04:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:04:11,474 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:04:11,474 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:04:11,475 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:11,475 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:04:11,475 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:11,475 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:21 UTC)" executed successfully 2020-03-31 09:04:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:04:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:24.323352+00:00 (in 9.997998 seconds) 2020-03-31 09:04:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:24 UTC)" (scheduled at 2020-03-31 09:04:14.323352+00:00) 2020-03-31 09:04:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:04:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:24.323352+00:00 (in 9.997998 seconds) 2020-03-31 09:04:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:24 UTC)" (scheduled at 2020-03-31 09:04:14.323352+00:00) 2020-03-31 09:04:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 14, 326091), True, datetime.datetime(2020, 3, 31, 9, 4, 14, 326091), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:04:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 14, 326091), True, datetime.datetime(2020, 3, 31, 9, 4, 14, 326091), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:04:14,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:04:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:04:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:24 UTC)" executed successfully 2020-03-31 09:04:14,446 [518] [DEBUG] [app] Starting request: urn:request:c3863972-d553-4c50-91d5-0c87f980a0b4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:14,446 [518] [DEBUG] [app] Starting request: urn:request:c3863972-d553-4c50-91d5-0c87f980a0b4 (/health/instance) 2020-03-31 09:04:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:14,453 [542] [DEBUG] [app] Starting request: urn:request:6f7755d3-cc93-45d7-a2db-a8d7189abbb5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:14,453 [542] [DEBUG] [app] Starting request: urn:request:6f7755d3-cc93-45d7-a2db-a8d7189abbb5 (/v1/_internal_ping) 2020-03-31 09:04:14,454 [542] [DEBUG] [app] Ending request: urn:request:6f7755d3-cc93-45d7-a2db-a8d7189abbb5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:14,454 [542] [DEBUG] [app] Ending request: urn:request:6f7755d3-cc93-45d7-a2db-a8d7189abbb5 (/v1/_internal_ping) 2020-03-31 09:04:14,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:04:14,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:14,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:04:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:04:14,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:14,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:14,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:14,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:14,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:14,488 [519] [DEBUG] [app] Starting request: urn:request:3714828f-dffa-4c3c-ba19-1e4367479b62 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:14,488 [519] [DEBUG] [app] Starting request: urn:request:3714828f-dffa-4c3c-ba19-1e4367479b62 (/_internal_ping) 2020-03-31 09:04:14,489 [519] [DEBUG] [app] Ending request: urn:request:3714828f-dffa-4c3c-ba19-1e4367479b62 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:14,489 [519] [DEBUG] [app] Ending request: urn:request:3714828f-dffa-4c3c-ba19-1e4367479b62 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:04:14,490 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:14,490 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:14,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:14,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:04:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:14,504 [537] [DEBUG] [app] Starting request: urn:request:6b520d53-b5ff-4564-9c64-98ccc5f01955 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:14,504 [537] [DEBUG] [app] Starting request: urn:request:6b520d53-b5ff-4564-9c64-98ccc5f01955 (/c1/_internal_ping) 2020-03-31 09:04:14,505 [537] [DEBUG] [app] Ending request: urn:request:6b520d53-b5ff-4564-9c64-98ccc5f01955 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:14,505 [537] [DEBUG] [app] Ending request: urn:request:6b520d53-b5ff-4564-9c64-98ccc5f01955 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:14,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:04:14,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:14,507 [518] [DEBUG] [app] Ending request: urn:request:c3863972-d553-4c50-91d5-0c87f980a0b4 (/health/instance) 2020-03-31 09:04:14,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:14,507 [518] [DEBUG] [app] Ending request: urn:request:c3863972-d553-4c50-91d5-0c87f980a0b4 (/health/instance) 2020-03-31 09:04:14,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:14,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:04:14,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:04:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:04:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:45.064514+00:00 (in 29.999059 seconds) 2020-03-31 09:04:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:45 UTC)" (scheduled at 2020-03-31 09:04:15.064514+00:00) gcworker stdout | 2020-03-31 09:04:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:45.064514+00:00 (in 29.999059 seconds) 2020-03-31 09:04:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:45 UTC)" (scheduled at 2020-03-31 09:04:15.064514+00:00) 2020-03-31 09:04:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435855, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:04:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435855, 1209600, 500, 1, 0]) 2020-03-31 09:04:15,089 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:04:15,089 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:04:15,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:15,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:04:15,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:15,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:45 UTC)" executed successfully 2020-03-31 09:04:16,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:04:16,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:26.924497+00:00 (in 9.999365 seconds) 2020-03-31 09:04:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:26 UTC)" (scheduled at 2020-03-31 09:04:16.924497+00:00) 2020-03-31 09:04:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:04:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:26.924497+00:00 (in 9.999365 seconds) 2020-03-31 09:04:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:26 UTC)" (scheduled at 2020-03-31 09:04:16.924497+00:00) 2020-03-31 09:04:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 16, 925814), True, datetime.datetime(2020, 3, 31, 9, 4, 16, 925814), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:04:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 16, 925814), True, datetime.datetime(2020, 3, 31, 9, 4, 16, 925814), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:04:16,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:04:16,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:04:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:26 UTC)" executed successfully 2020-03-31 09:04:18,273 [527] [DEBUG] [app] Starting request: urn:request:57fc03ff-de9f-47bb-a638-7b20c1be3166 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:18,273 [527] [DEBUG] [app] Starting request: urn:request:57fc03ff-de9f-47bb-a638-7b20c1be3166 (/health/instance) 2020-03-31 09:04:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:18,281 [550] [DEBUG] [app] Starting request: urn:request:a6fa3896-1400-48df-81b6-fe2e8811d1bf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:18,281 [550] [DEBUG] [app] Starting request: urn:request:a6fa3896-1400-48df-81b6-fe2e8811d1bf (/v1/_internal_ping) 2020-03-31 09:04:18,282 [550] [DEBUG] [app] Ending request: urn:request:a6fa3896-1400-48df-81b6-fe2e8811d1bf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:18,282 [550] [DEBUG] [app] Ending request: urn:request:a6fa3896-1400-48df-81b6-fe2e8811d1bf (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:18,283 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:04:18,283 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:18,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:04:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:04:18,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:18,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:18,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:18,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:18,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:18,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:18,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:18,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:18,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:18,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:18,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:18,310 [527] [DEBUG] [app] Starting request: urn:request:ef6afd29-49f3-412e-a9e2-91075adbbbe0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:18,310 [527] [DEBUG] [app] Starting request: urn:request:ef6afd29-49f3-412e-a9e2-91075adbbbe0 (/_internal_ping) 2020-03-31 09:04:18,311 [527] [DEBUG] [app] Ending request: urn:request:ef6afd29-49f3-412e-a9e2-91075adbbbe0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:18,311 [527] [DEBUG] [app] Ending request: urn:request:ef6afd29-49f3-412e-a9e2-91075adbbbe0 (/_internal_ping) 2020-03-31 09:04:18,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:18,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:18,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:18,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:18,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:18,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:18,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:18,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:18,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:18,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:04:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:18,323 [536] [DEBUG] [app] Starting request: urn:request:dffe1962-ba07-4d44-93ca-a76011fe1bb9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:18,323 [536] [DEBUG] [app] Starting request: urn:request:dffe1962-ba07-4d44-93ca-a76011fe1bb9 (/c1/_internal_ping) 2020-03-31 09:04:18,323 [536] [DEBUG] [app] Ending request: urn:request:dffe1962-ba07-4d44-93ca-a76011fe1bb9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:18,323 [536] [DEBUG] [app] Ending request: urn:request:dffe1962-ba07-4d44-93ca-a76011fe1bb9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:18,324 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:18,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:04:18,324 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:18,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:18,326 [527] [DEBUG] [app] Ending request: urn:request:57fc03ff-de9f-47bb-a638-7b20c1be3166 (/health/instance) 2020-03-31 09:04:18,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:18,326 [527] [DEBUG] [app] Ending request: urn:request:57fc03ff-de9f-47bb-a638-7b20c1be3166 (/health/instance) 2020-03-31 09:04:18,326 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:18,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:04:18,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:04:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:04:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:31.440707+00:00 (in 9.998992 seconds) 2020-03-31 09:04:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:31 UTC)" (scheduled at 2020-03-31 09:04:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:04:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:31.440707+00:00 (in 9.998992 seconds) 2020-03-31 09:04:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:31 UTC)" (scheduled at 2020-03-31 09:04:21.440707+00:00) 2020-03-31 09:04:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:04:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:04:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:04:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:04:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:04:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:31 UTC)" executed successfully 2020-03-31 09:04:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:04:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:52.113221+00:00 (in 29.998191 seconds) 2020-03-31 09:04:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:52 UTC)" (scheduled at 2020-03-31 09:04:22.113221+00:00) securityworker stdout | 2020-03-31 09:04:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:52.113221+00:00 (in 29.998191 seconds) 2020-03-31 09:04:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:52 UTC)" (scheduled at 2020-03-31 09:04:22.113221+00:00) 2020-03-31 09:04:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:04:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:04:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:04:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:04:22,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:04:22,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:04:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:04:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:04:52 UTC)" executed successfully 2020-03-31 09:04:22,907 [184] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2020-03-31 09:04:22,907 [184] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:22,908 [184] [INFO] [apscheduler.executors.default] Running job "_gc_expired_tokens (trigger: interval[1:00:00], next run at: 2020-03-31 09:04:22 UTC)" (scheduled at 2020-03-31 09:04:22.907313+00:00) expiredappspecifictokenworker stdout | 2020-03-31 09:04:22,908 [184] [INFO] [apscheduler.executors.default] Running job "_gc_expired_tokens (trigger: interval[1:00:00], next run at: 2020-03-31 09:04:22 UTC)" (scheduled at 2020-03-31 09:04:22.907313+00:00) 2020-03-31 09:04:22,908 [184] [DEBUG] [__main__] Garbage collecting expired app specific tokens with window: 1 day, 0:00:00 expiredappspecifictokenworker stdout | 2020-03-31 09:04:22,908 [184] [DEBUG] [__main__] Garbage collecting expired app specific tokens with window: 1 day, 0:00:00 2020-03-31 09:04:22,910 [184] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:04:22.907313+00:00 (in 3599.999421 seconds) expiredappspecifictokenworker stdout | 2020-03-31 09:04:22,910 [184] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:04:22.907313+00:00 (in 3599.999421 seconds) 2020-03-31 09:04:22,911 [184] [DEBUG] [peewee] ('DELETE FROM "appspecificauthtoken" WHERE ("appspecificauthtoken"."expiration" < %s)', [datetime.datetime(2020, 3, 30, 9, 4, 22, 910342)]) expiredappspecifictokenworker stdout | 2020-03-31 09:04:22,911 [184] [DEBUG] [peewee] ('DELETE FROM "appspecificauthtoken" WHERE ("appspecificauthtoken"."expiration" < %s)', [datetime.datetime(2020, 3, 30, 9, 4, 22, 910342)]) 2020-03-31 09:04:22,927 [184] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2020-03-31 09:04:22,927 [184] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:22,927 [184] [INFO] [apscheduler.executors.default] Job "_gc_expired_tokens (trigger: interval[1:00:00], next run at: 2020-03-31 10:04:22 UTC)" executed successfully expiredappspecifictokenworker stdout | 2020-03-31 09:04:22,927 [184] [INFO] [apscheduler.executors.default] Job "_gc_expired_tokens (trigger: interval[1:00:00], next run at: 2020-03-31 10:04:22 UTC)" executed successfully 2020-03-31 09:04:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:04:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:23,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:24.410927+00:00 (in 0.998836 seconds) 2020-03-31 09:04:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:23 UTC)" (scheduled at 2020-03-31 09:04:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:04:23,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:24.410927+00:00 (in 0.998836 seconds) 2020-03-31 09:04:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:23 UTC)" (scheduled at 2020-03-31 09:04:23.410371+00:00) 2020-03-31 09:04:23,414 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:04:23,414 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), u'security_notification/%']) 2020-03-31 09:04:23,429 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), True, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:04:23,429 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), True, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), 0, u'security_notification/%']) 2020-03-31 09:04:23,434 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), True, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:04:23,434 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), True, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 4, 23, 412957), u'security_notification/%']) 2020-03-31 09:04:23,437 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:04:23,437 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:23,439 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:04:23,439 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:23 UTC)" executed successfully 2020-03-31 09:04:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:04:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:23,442 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:04:23,442 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:04:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:04:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:04:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:34.323352+00:00 (in 9.999261 seconds) 2020-03-31 09:04:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:34 UTC)" (scheduled at 2020-03-31 09:04:24.323352+00:00) 2020-03-31 09:04:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:04:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:34.323352+00:00 (in 9.999261 seconds) 2020-03-31 09:04:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:34 UTC)" (scheduled at 2020-03-31 09:04:24.323352+00:00) 2020-03-31 09:04:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 24, 324982), True, datetime.datetime(2020, 3, 31, 9, 4, 24, 324982), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:04:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 24, 324982), True, datetime.datetime(2020, 3, 31, 9, 4, 24, 324982), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:04:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:04:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:04:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:34 UTC)" executed successfully 2020-03-31 09:04:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:04:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:33.330112+00:00 (in 8.918376 seconds) 2020-03-31 09:04:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:24 UTC)" (scheduled at 2020-03-31 09:04:24.410927+00:00) 2020-03-31 09:04:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:04:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:33.330112+00:00 (in 8.918376 seconds) 2020-03-31 09:04:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:24 UTC)" (scheduled at 2020-03-31 09:04:24.410927+00:00) 2020-03-31 09:04:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:04:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:04:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:24 UTC)" executed successfully 2020-03-31 09:04:24,446 [518] [DEBUG] [app] Starting request: urn:request:23590c13-16e8-4666-ad5c-47903018eb25 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:24,446 [518] [DEBUG] [app] Starting request: urn:request:23590c13-16e8-4666-ad5c-47903018eb25 (/health/instance) 2020-03-31 09:04:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:24,454 [538] [DEBUG] [app] Starting request: urn:request:67dd0063-672e-4a3e-bcbe-d192c3cf8f63 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:24,454 [538] [DEBUG] [app] Starting request: urn:request:67dd0063-672e-4a3e-bcbe-d192c3cf8f63 (/v1/_internal_ping) 2020-03-31 09:04:24,455 [538] [DEBUG] [app] Ending request: urn:request:67dd0063-672e-4a3e-bcbe-d192c3cf8f63 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:24,455 [538] [DEBUG] [app] Ending request: urn:request:67dd0063-672e-4a3e-bcbe-d192c3cf8f63 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:04:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:24,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:24,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:24,490 [518] [DEBUG] [app] Starting request: urn:request:5c69bc37-a52b-40de-982f-09e98469404b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:24,490 [518] [DEBUG] [app] Starting request: urn:request:5c69bc37-a52b-40de-982f-09e98469404b (/_internal_ping) 2020-03-31 09:04:24,491 [518] [DEBUG] [app] Ending request: urn:request:5c69bc37-a52b-40de-982f-09e98469404b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:24,491 [518] [DEBUG] [app] Ending request: urn:request:5c69bc37-a52b-40de-982f-09e98469404b (/_internal_ping) 2020-03-31 09:04:24,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:24,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:04:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:04:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:04:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:24,504 [536] [DEBUG] [app] Starting request: urn:request:0fdb0fa1-d7a2-4c60-a6ce-273e96aa7771 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:24,504 [536] [DEBUG] [app] Starting request: urn:request:0fdb0fa1-d7a2-4c60-a6ce-273e96aa7771 (/c1/_internal_ping) 2020-03-31 09:04:24,505 [536] [DEBUG] [app] Ending request: urn:request:0fdb0fa1-d7a2-4c60-a6ce-273e96aa7771 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:24,505 [536] [DEBUG] [app] Ending request: urn:request:0fdb0fa1-d7a2-4c60-a6ce-273e96aa7771 (/c1/_internal_ping) 2020-03-31 09:04:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:24,506 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:24,507 [518] [DEBUG] [app] Ending request: urn:request:23590c13-16e8-4666-ad5c-47903018eb25 (/health/instance) 2020-03-31 09:04:24,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-verbs stdout | 2020-03-31 09:04:24,506 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:24,507 [518] [DEBUG] [app] Ending request: urn:request:23590c13-16e8-4666-ad5c-47903018eb25 (/health/instance) 2020-03-31 09:04:24,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:04:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:04:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:36.924497+00:00 (in 9.999207 seconds) storagereplication stdout | 2020-03-31 09:04:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:36.924497+00:00 (in 9.999207 seconds) 2020-03-31 09:04:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:36 UTC)" (scheduled at 2020-03-31 09:04:26.924497+00:00) 2020-03-31 09:04:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:04:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:36 UTC)" (scheduled at 2020-03-31 09:04:26.924497+00:00) 2020-03-31 09:04:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 26, 926885), True, datetime.datetime(2020, 3, 31, 9, 4, 26, 926885), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:04:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 26, 926885), True, datetime.datetime(2020, 3, 31, 9, 4, 26, 926885), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:04:26,949 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:04:26,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:26,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:26,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:04:26,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:26,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:36 UTC)" executed successfully 2020-03-31 09:04:28,273 [518] [DEBUG] [app] Starting request: urn:request:cdd16266-5d36-454c-96d2-8736fe767d25 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:28,273 [518] [DEBUG] [app] Starting request: urn:request:cdd16266-5d36-454c-96d2-8736fe767d25 (/health/instance) 2020-03-31 09:04:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:28,279 [538] [DEBUG] [app] Starting request: urn:request:494433a7-4fc0-43f5-a181-23ae36780967 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:28,279 [538] [DEBUG] [app] Starting request: urn:request:494433a7-4fc0-43f5-a181-23ae36780967 (/v1/_internal_ping) 2020-03-31 09:04:28,281 [538] [DEBUG] [app] Ending request: urn:request:494433a7-4fc0-43f5-a181-23ae36780967 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:28,281 [538] [DEBUG] [app] Ending request: urn:request:494433a7-4fc0-43f5-a181-23ae36780967 (/v1/_internal_ping) 2020-03-31 09:04:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:04:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:28,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:28,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:28,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:28,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:28,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:28,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:28,313 [518] [DEBUG] [app] Starting request: urn:request:e49ed992-733c-4cdc-96a5-4ba07d1fc649 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:28,313 [518] [DEBUG] [app] Starting request: urn:request:e49ed992-733c-4cdc-96a5-4ba07d1fc649 (/_internal_ping) 2020-03-31 09:04:28,314 [518] [DEBUG] [app] Ending request: urn:request:e49ed992-733c-4cdc-96a5-4ba07d1fc649 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:28,314 [518] [DEBUG] [app] Ending request: urn:request:e49ed992-733c-4cdc-96a5-4ba07d1fc649 (/_internal_ping) 2020-03-31 09:04:28,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:04:28,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:04:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:04:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:04:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:04:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:04:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:28,330 [536] [DEBUG] [app] Starting request: urn:request:8198c0dd-9681-4d66-9267-56ed7de2fde9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:28,330 [536] [DEBUG] [app] Starting request: urn:request:8198c0dd-9681-4d66-9267-56ed7de2fde9 (/c1/_internal_ping) 2020-03-31 09:04:28,331 [536] [DEBUG] [app] Ending request: urn:request:8198c0dd-9681-4d66-9267-56ed7de2fde9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:28,331 [536] [DEBUG] [app] Ending request: urn:request:8198c0dd-9681-4d66-9267-56ed7de2fde9 (/c1/_internal_ping) 2020-03-31 09:04:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:28,331 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:28,333 [518] [DEBUG] [app] Ending request: urn:request:cdd16266-5d36-454c-96d2-8736fe767d25 (/health/instance) 2020-03-31 09:04:28,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:04:28,331 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:28,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:04:28,333 [518] [DEBUG] [app] Ending request: urn:request:cdd16266-5d36-454c-96d2-8736fe767d25 (/health/instance) 2020-03-31 09:04:28,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:28,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:04:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:04:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:00.337699+00:00 (in 29.998860 seconds) 2020-03-31 09:04:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:00 UTC)" (scheduled at 2020-03-31 09:04:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:04:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:00.337699+00:00 (in 29.998860 seconds) 2020-03-31 09:04:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:00 UTC)" (scheduled at 2020-03-31 09:04:30.337699+00:00) 2020-03-31 09:04:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 4, 30, 339848), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:04:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 4, 30, 339848), False, 50, 1, 0]) 2020-03-31 09:04:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:04:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:04:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:04:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:00 UTC)" executed successfully 2020-03-31 09:04:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:04:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:41.440707+00:00 (in 9.999120 seconds) 2020-03-31 09:04:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:41 UTC)" (scheduled at 2020-03-31 09:04:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:04:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:41.440707+00:00 (in 9.999120 seconds) 2020-03-31 09:04:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:41 UTC)" (scheduled at 2020-03-31 09:04:31.440707+00:00) 2020-03-31 09:04:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:04:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:04:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:04:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:04:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:04:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:41 UTC)" executed successfully 2020-03-31 09:04:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:04:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:03.330112+00:00 (in 29.999057 seconds) 2020-03-31 09:04:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:03 UTC)" (scheduled at 2020-03-31 09:04:33.330112+00:00) 2020-03-31 09:04:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:04:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:03.330112+00:00 (in 29.999057 seconds) 2020-03-31 09:04:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:03 UTC)" (scheduled at 2020-03-31 09:04:33.330112+00:00) 2020-03-31 09:04:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 33, 331816), True, datetime.datetime(2020, 3, 31, 9, 4, 33, 331816), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:04:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 33, 331816), True, datetime.datetime(2020, 3, 31, 9, 4, 33, 331816), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:04:33,347 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:04:33,347 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:33,347 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:33,347 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:04:33,347 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:33,347 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:03 UTC)" executed successfully 2020-03-31 09:04:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:04:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:43.539999+00:00 (in 10.171852 seconds) 2020-03-31 09:04:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:33 UTC)" (scheduled at 2020-03-31 09:04:33.366791+00:00) 2020-03-31 09:04:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:04:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:43.539999+00:00 (in 10.171852 seconds) 2020-03-31 09:04:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:33 UTC)" (scheduled at 2020-03-31 09:04:33.366791+00:00) 2020-03-31 09:04:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 33, 368854), True, datetime.datetime(2020, 3, 31, 9, 4, 33, 368854), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:04:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 33, 368854), True, datetime.datetime(2020, 3, 31, 9, 4, 33, 368854), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:04:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:04:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:04:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:33 UTC)" executed successfully 2020-03-31 09:04:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:04:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:44.323352+00:00 (in 9.999214 seconds) 2020-03-31 09:04:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:44 UTC)" (scheduled at 2020-03-31 09:04:34.323352+00:00) 2020-03-31 09:04:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:04:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:44.323352+00:00 (in 9.999214 seconds) 2020-03-31 09:04:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:44 UTC)" (scheduled at 2020-03-31 09:04:34.323352+00:00) 2020-03-31 09:04:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 34, 325130), True, datetime.datetime(2020, 3, 31, 9, 4, 34, 325130), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:04:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 34, 325130), True, datetime.datetime(2020, 3, 31, 9, 4, 34, 325130), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:04:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:34,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:04:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:04:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:44 UTC)" executed successfully 2020-03-31 09:04:34,446 [518] [DEBUG] [app] Starting request: urn:request:c9aac216-e1ab-4d1f-8465-328546a25736 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:34,446 [518] [DEBUG] [app] Starting request: urn:request:c9aac216-e1ab-4d1f-8465-328546a25736 (/health/instance) 2020-03-31 09:04:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:34,454 [555] [DEBUG] [app] Starting request: urn:request:65bae9c0-976f-4ea9-a442-e015842d893b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:34,454 [555] [DEBUG] [app] Starting request: urn:request:65bae9c0-976f-4ea9-a442-e015842d893b (/v1/_internal_ping) 2020-03-31 09:04:34,455 [555] [DEBUG] [app] Ending request: urn:request:65bae9c0-976f-4ea9-a442-e015842d893b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:34,455 [555] [DEBUG] [app] Ending request: urn:request:65bae9c0-976f-4ea9-a442-e015842d893b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:34,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:04:34,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:34,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:34,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:34,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:34,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:34,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:34,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:34,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:34,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:34,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:34,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:34,490 [518] [DEBUG] [app] Starting request: urn:request:8c8c7ac6-0c09-4fb2-bbaf-1fa04fd3a610 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:34,490 [518] [DEBUG] [app] Starting request: urn:request:8c8c7ac6-0c09-4fb2-bbaf-1fa04fd3a610 (/_internal_ping) 2020-03-31 09:04:34,491 [518] [DEBUG] [app] Ending request: urn:request:8c8c7ac6-0c09-4fb2-bbaf-1fa04fd3a610 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:34,491 [518] [DEBUG] [app] Ending request: urn:request:8c8c7ac6-0c09-4fb2-bbaf-1fa04fd3a610 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:04:34,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:34,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:34,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:34,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:04:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:34,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:34,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:34,505 [543] [DEBUG] [app] Starting request: urn:request:0dc4bc56-8245-4132-bf48-d1e7e21dad2b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:34,505 [543] [DEBUG] [app] Starting request: urn:request:0dc4bc56-8245-4132-bf48-d1e7e21dad2b (/c1/_internal_ping) 2020-03-31 09:04:34,506 [543] [DEBUG] [app] Ending request: urn:request:0dc4bc56-8245-4132-bf48-d1e7e21dad2b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:34,506 [543] [DEBUG] [app] Ending request: urn:request:0dc4bc56-8245-4132-bf48-d1e7e21dad2b (/c1/_internal_ping) 2020-03-31 09:04:34,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:34,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:34,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:34,508 [518] [DEBUG] [app] Ending request: urn:request:c9aac216-e1ab-4d1f-8465-328546a25736 (/health/instance) 2020-03-31 09:04:34,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:04:34,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:34,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:04:34,508 [518] [DEBUG] [app] Ending request: urn:request:c9aac216-e1ab-4d1f-8465-328546a25736 (/health/instance) 2020-03-31 09:04:34,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:34,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:04:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:04:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:38.010934+00:00 (in 1.084799 seconds) 2020-03-31 09:04:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:46 UTC)" (scheduled at 2020-03-31 09:04:36.924497+00:00) 2020-03-31 09:04:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:04:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:38.010934+00:00 (in 1.084799 seconds) 2020-03-31 09:04:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:46 UTC)" (scheduled at 2020-03-31 09:04:36.924497+00:00) 2020-03-31 09:04:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 36, 927038), True, datetime.datetime(2020, 3, 31, 9, 4, 36, 927038), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:04:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 36, 927038), True, datetime.datetime(2020, 3, 31, 9, 4, 36, 927038), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:04:36,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:04:36,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:04:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:46 UTC)" executed successfully 2020-03-31 09:04:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:04:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:46.924497+00:00 (in 8.912452 seconds) 2020-03-31 09:04:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:38 UTC)" (scheduled at 2020-03-31 09:04:38.010934+00:00) 2020-03-31 09:04:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:04:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:04:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:46.924497+00:00 (in 8.912452 seconds) 2020-03-31 09:04:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:38 UTC)" (scheduled at 2020-03-31 09:04:38.010934+00:00) 2020-03-31 09:04:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:04:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:38 UTC)" executed successfully 2020-03-31 09:04:38,273 [518] [DEBUG] [app] Starting request: urn:request:74f4f648-5d0d-4856-9cb4-50156535d5e5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:38,273 [518] [DEBUG] [app] Starting request: urn:request:74f4f648-5d0d-4856-9cb4-50156535d5e5 (/health/instance) 2020-03-31 09:04:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:38,278 [556] [DEBUG] [app] Starting request: urn:request:8dc80478-dfc3-4ebb-a8f3-be312dfde576 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:38,278 [556] [DEBUG] [app] Starting request: urn:request:8dc80478-dfc3-4ebb-a8f3-be312dfde576 (/v1/_internal_ping) 2020-03-31 09:04:38,280 [556] [DEBUG] [app] Ending request: urn:request:8dc80478-dfc3-4ebb-a8f3-be312dfde576 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:38,280 [556] [DEBUG] [app] Ending request: urn:request:8dc80478-dfc3-4ebb-a8f3-be312dfde576 (/v1/_internal_ping) 2020-03-31 09:04:38,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:04:38,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:38,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:04:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:04:38,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:38,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:38,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:38,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:38,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:38,312 [518] [DEBUG] [app] Starting request: urn:request:5795ccf4-f33f-4548-ab22-546b1353273d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:38,312 [518] [DEBUG] [app] Starting request: urn:request:5795ccf4-f33f-4548-ab22-546b1353273d (/_internal_ping) 2020-03-31 09:04:38,313 [518] [DEBUG] [app] Ending request: urn:request:5795ccf4-f33f-4548-ab22-546b1353273d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:38,313 [518] [DEBUG] [app] Ending request: urn:request:5795ccf4-f33f-4548-ab22-546b1353273d (/_internal_ping) 2020-03-31 09:04:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:38,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:38,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:04:38,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:38,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:38,325 [543] [DEBUG] [app] Starting request: urn:request:fbbcac01-f797-4b6c-84e3-8112677e2fb9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:38,325 [543] [DEBUG] [app] Starting request: urn:request:fbbcac01-f797-4b6c-84e3-8112677e2fb9 (/c1/_internal_ping) 2020-03-31 09:04:38,326 [543] [DEBUG] [app] Ending request: urn:request:fbbcac01-f797-4b6c-84e3-8112677e2fb9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:38,326 [543] [DEBUG] [app] Ending request: urn:request:fbbcac01-f797-4b6c-84e3-8112677e2fb9 (/c1/_internal_ping) 2020-03-31 09:04:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:38,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:04:38,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:38,329 [518] [DEBUG] [app] Ending request: urn:request:74f4f648-5d0d-4856-9cb4-50156535d5e5 (/health/instance) 2020-03-31 09:04:38,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:38,329 [518] [DEBUG] [app] Ending request: urn:request:74f4f648-5d0d-4856-9cb4-50156535d5e5 (/health/instance) 2020-03-31 09:04:38,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:38,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:04:38,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:04:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:04:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:51.440707+00:00 (in 9.999104 seconds) 2020-03-31 09:04:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:51 UTC)" (scheduled at 2020-03-31 09:04:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:04:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:51.440707+00:00 (in 9.999104 seconds) 2020-03-31 09:04:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:51 UTC)" (scheduled at 2020-03-31 09:04:41.440707+00:00) 2020-03-31 09:04:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:04:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:04:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:04:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:04:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:04:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:51 UTC)" executed successfully 2020-03-31 09:04:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:04:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:14.539408+00:00 (in 30.998458 seconds) 2020-03-31 09:04:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:43 UTC)" (scheduled at 2020-03-31 09:04:43.539999+00:00) 2020-03-31 09:04:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:04:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:14.539408+00:00 (in 30.998458 seconds) 2020-03-31 09:04:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:43 UTC)" (scheduled at 2020-03-31 09:04:43.539999+00:00) 2020-03-31 09:04:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:04:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:04:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:43 UTC)" executed successfully 2020-03-31 09:04:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:04:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:54 UTC)" (scheduled at 2020-03-31 09:04:44.323352+00:00) 2020-03-31 09:04:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:04:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:54 UTC)" (scheduled at 2020-03-31 09:04:44.323352+00:00) 2020-03-31 09:04:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:44,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:49.498966+00:00 (in 5.174582 seconds) notificationworker stdout | 2020-03-31 09:04:44,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:49.498966+00:00 (in 5.174582 seconds) 2020-03-31 09:04:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 44, 325263), True, datetime.datetime(2020, 3, 31, 9, 4, 44, 325263), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:04:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 44, 325263), True, datetime.datetime(2020, 3, 31, 9, 4, 44, 325263), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:04:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:04:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:04:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:54 UTC)" executed successfully 2020-03-31 09:04:44,446 [526] [DEBUG] [app] Starting request: urn:request:4a29535e-9fea-4f93-a443-5702592d69da (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:44,446 [526] [DEBUG] [app] Starting request: urn:request:4a29535e-9fea-4f93-a443-5702592d69da (/health/instance) 2020-03-31 09:04:44,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:44,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:44,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:44,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:44,454 [540] [DEBUG] [app] Starting request: urn:request:1bbf7fb2-3378-486c-8f94-0621fcd898ab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:44,454 [540] [DEBUG] [app] Starting request: urn:request:1bbf7fb2-3378-486c-8f94-0621fcd898ab (/v1/_internal_ping) 2020-03-31 09:04:44,455 [540] [DEBUG] [app] Ending request: urn:request:1bbf7fb2-3378-486c-8f94-0621fcd898ab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:44,455 [540] [DEBUG] [app] Ending request: urn:request:1bbf7fb2-3378-486c-8f94-0621fcd898ab (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:44,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:44,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:04:44,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:44,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:44,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:44,458 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:04:44,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:04:44,458 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:44,459 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:44,459 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:44,460 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:44,460 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:44,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:44,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:44,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:44,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:44,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:44,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:44,485 [526] [DEBUG] [app] Starting request: urn:request:72a5ff4b-6af1-4a85-bc6f-21c44bf422ff (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:44,485 [526] [DEBUG] [app] Starting request: urn:request:72a5ff4b-6af1-4a85-bc6f-21c44bf422ff (/_internal_ping) 2020-03-31 09:04:44,486 [526] [DEBUG] [app] Ending request: urn:request:72a5ff4b-6af1-4a85-bc6f-21c44bf422ff (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:44,486 [526] [DEBUG] [app] Ending request: urn:request:72a5ff4b-6af1-4a85-bc6f-21c44bf422ff (/_internal_ping) 2020-03-31 09:04:44,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:44,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:44,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:44,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:44,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:44,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:44,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:04:44,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:04:44,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:44,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:04:44,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:44,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:44,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:44,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:44,500 [543] [DEBUG] [app] Starting request: urn:request:78b7b19c-75f7-4651-9a72-cf63d9e2d8b7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:44,500 [543] [DEBUG] [app] Starting request: urn:request:78b7b19c-75f7-4651-9a72-cf63d9e2d8b7 (/c1/_internal_ping) 2020-03-31 09:04:44,500 [543] [DEBUG] [app] Ending request: urn:request:78b7b19c-75f7-4651-9a72-cf63d9e2d8b7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:44,500 [543] [DEBUG] [app] Ending request: urn:request:78b7b19c-75f7-4651-9a72-cf63d9e2d8b7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:44,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:44,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:44,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:04:44,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:44,502 [526] [DEBUG] [app] Ending request: urn:request:4a29535e-9fea-4f93-a443-5702592d69da (/health/instance) 2020-03-31 09:04:44,503 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:44,502 [526] [DEBUG] [app] Ending request: urn:request:4a29535e-9fea-4f93-a443-5702592d69da (/health/instance) 2020-03-31 09:04:44,503 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:44,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:04:44,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:04:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:04:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:15.064514+00:00 (in 29.999231 seconds) 2020-03-31 09:04:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:15 UTC)" (scheduled at 2020-03-31 09:04:45.064514+00:00) gcworker stdout | 2020-03-31 09:04:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:15.064514+00:00 (in 29.999231 seconds) 2020-03-31 09:04:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:15 UTC)" (scheduled at 2020-03-31 09:04:45.064514+00:00) 2020-03-31 09:04:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435885, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:04:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435885, 1209600, 500, 1, 0]) 2020-03-31 09:04:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:04:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:04:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:04:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:15 UTC)" executed successfully 2020-03-31 09:04:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:04:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:56.924497+00:00 (in 9.999197 seconds) 2020-03-31 09:04:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:56 UTC)" (scheduled at 2020-03-31 09:04:46.924497+00:00) 2020-03-31 09:04:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:04:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:56.924497+00:00 (in 9.999197 seconds) 2020-03-31 09:04:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:56 UTC)" (scheduled at 2020-03-31 09:04:46.924497+00:00) 2020-03-31 09:04:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 46, 925999), True, datetime.datetime(2020, 3, 31, 9, 4, 46, 925999), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:04:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 46, 925999), True, datetime.datetime(2020, 3, 31, 9, 4, 46, 925999), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:04:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:04:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:04:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:04:56 UTC)" executed successfully 2020-03-31 09:04:48,273 [518] [DEBUG] [app] Starting request: urn:request:17e8e0d4-8b19-42fa-a775-f599c0c2d149 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:48,273 [518] [DEBUG] [app] Starting request: urn:request:17e8e0d4-8b19-42fa-a775-f599c0c2d149 (/health/instance) 2020-03-31 09:04:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:48,281 [549] [DEBUG] [app] Starting request: urn:request:b3412280-13d0-4605-939b-2f6c91bc0932 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:48,281 [549] [DEBUG] [app] Starting request: urn:request:b3412280-13d0-4605-939b-2f6c91bc0932 (/v1/_internal_ping) 2020-03-31 09:04:48,282 [549] [DEBUG] [app] Ending request: urn:request:b3412280-13d0-4605-939b-2f6c91bc0932 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:48,282 [549] [DEBUG] [app] Ending request: urn:request:b3412280-13d0-4605-939b-2f6c91bc0932 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:48,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:04:48,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:48,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:48,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:48,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:48,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:48,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:48,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:48,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:48,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:48,314 [518] [DEBUG] [app] Starting request: urn:request:74bafd93-6d7f-4104-8291-65493e2e4a85 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:48,314 [518] [DEBUG] [app] Starting request: urn:request:74bafd93-6d7f-4104-8291-65493e2e4a85 (/_internal_ping) 2020-03-31 09:04:48,315 [518] [DEBUG] [app] Ending request: urn:request:74bafd93-6d7f-4104-8291-65493e2e4a85 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:48,315 [518] [DEBUG] [app] Ending request: urn:request:74bafd93-6d7f-4104-8291-65493e2e4a85 (/_internal_ping) 2020-03-31 09:04:48,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:48,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:04:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:48,327 [536] [DEBUG] [app] Starting request: urn:request:3481d60d-88d9-4431-89e8-3f1699c51698 (/c1/_internal_ping) 2020-03-31 09:04:48,328 [536] [DEBUG] [app] Ending request: urn:request:3481d60d-88d9-4431-89e8-3f1699c51698 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:04:48,327 [536] [DEBUG] [app] Starting request: urn:request:3481d60d-88d9-4431-89e8-3f1699c51698 (/c1/_internal_ping) 2020-03-31 09:04:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:48,329 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:04:48,328 [536] [DEBUG] [app] Ending request: urn:request:3481d60d-88d9-4431-89e8-3f1699c51698 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:04:48,331 [518] [DEBUG] [app] Ending request: urn:request:17e8e0d4-8b19-42fa-a775-f599c0c2d149 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:48,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:04:48,329 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:04:48,331 [518] [DEBUG] [app] Ending request: urn:request:17e8e0d4-8b19-42fa-a775-f599c0c2d149 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:48,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:04:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:04:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:54.323352+00:00 (in 4.820236 seconds) 2020-03-31 09:04:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:49 UTC)" (scheduled at 2020-03-31 09:04:49.498966+00:00) 2020-03-31 09:04:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:04:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:54.323352+00:00 (in 4.820236 seconds) 2020-03-31 09:04:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:49 UTC)" (scheduled at 2020-03-31 09:04:49.498966+00:00) 2020-03-31 09:04:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:04:49,504 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:04:49,504 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:49 UTC)" executed successfully 2020-03-31 09:04:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:04:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:53.556282+00:00 (in 2.444863 seconds) 2020-03-31 09:04:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:51 UTC)" (scheduled at 2020-03-31 09:04:51.110610+00:00) 2020-03-31 09:04:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:04:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:53.556282+00:00 (in 2.444863 seconds) 2020-03-31 09:04:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:51 UTC)" (scheduled at 2020-03-31 09:04:51.110610+00:00) 2020-03-31 09:04:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 51, 112296), True, datetime.datetime(2020, 3, 31, 9, 4, 51, 112296), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:04:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 51, 112296), True, datetime.datetime(2020, 3, 31, 9, 4, 51, 112296), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:04:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:04:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:04:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:51 UTC)" executed successfully 2020-03-31 09:04:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:04:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:01.440707+00:00 (in 9.999222 seconds) 2020-03-31 09:04:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:01 UTC)" (scheduled at 2020-03-31 09:04:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:04:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:01.440707+00:00 (in 9.999222 seconds) 2020-03-31 09:04:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:01 UTC)" (scheduled at 2020-03-31 09:04:51.440707+00:00) 2020-03-31 09:04:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:04:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:04:51,457 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:04:51,457 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:04:51,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:51,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:04:51,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:51,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:01 UTC)" executed successfully 2020-03-31 09:04:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:04:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:22.113221+00:00 (in 29.998775 seconds) 2020-03-31 09:04:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:22 UTC)" (scheduled at 2020-03-31 09:04:52.113221+00:00) securityworker stdout | 2020-03-31 09:04:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:22.113221+00:00 (in 29.998775 seconds) 2020-03-31 09:04:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:22 UTC)" (scheduled at 2020-03-31 09:04:52.113221+00:00) 2020-03-31 09:04:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:04:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:04:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:04:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:04:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:04:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:04:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:04:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:22 UTC)" executed successfully 2020-03-31 09:04:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:04:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:59.560326+00:00 (in 6.003196 seconds) 2020-03-31 09:04:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:53 UTC)" (scheduled at 2020-03-31 09:04:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:04:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:04:59.560326+00:00 (in 6.003196 seconds) 2020-03-31 09:04:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:53 UTC)" (scheduled at 2020-03-31 09:04:53.556282+00:00) 2020-03-31 09:04:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:04:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), u'namespacegc/%']) 2020-03-31 09:04:53,573 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), True, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:04:53,573 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), True, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), 0, u'namespacegc/%']) 2020-03-31 09:04:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), True, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:04:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), True, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 4, 53, 557903), u'namespacegc/%']) 2020-03-31 09:04:53,582 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:04:53,582 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:53,582 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:04:53,582 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:09:53 UTC)" executed successfully 2020-03-31 09:04:53,585 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:04:53,585 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:04:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:53,587 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:04:53,587 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:04:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:04:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:04.323352+00:00 (in 9.998805 seconds) 2020-03-31 09:04:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:04 UTC)" (scheduled at 2020-03-31 09:04:54.323352+00:00) 2020-03-31 09:04:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:04:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:04.323352+00:00 (in 9.998805 seconds) 2020-03-31 09:04:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:04 UTC)" (scheduled at 2020-03-31 09:04:54.323352+00:00) 2020-03-31 09:04:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 54, 325457), True, datetime.datetime(2020, 3, 31, 9, 4, 54, 325457), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:04:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 54, 325457), True, datetime.datetime(2020, 3, 31, 9, 4, 54, 325457), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:04:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:54,344 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:04:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:04:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:04 UTC)" executed successfully 2020-03-31 09:04:54,446 [518] [DEBUG] [app] Starting request: urn:request:49a2fc11-32e7-4a63-b2d5-28e4221e14b4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:54,446 [518] [DEBUG] [app] Starting request: urn:request:49a2fc11-32e7-4a63-b2d5-28e4221e14b4 (/health/instance) 2020-03-31 09:04:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:54,453 [538] [DEBUG] [app] Starting request: urn:request:59436b17-b653-474d-ae5d-40cfdf4f1c43 (/v1/_internal_ping) 2020-03-31 09:04:54,454 [538] [DEBUG] [app] Ending request: urn:request:59436b17-b653-474d-ae5d-40cfdf4f1c43 (/v1/_internal_ping) 2020-03-31 09:04:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:04:54,453 [538] [DEBUG] [app] Starting request: urn:request:59436b17-b653-474d-ae5d-40cfdf4f1c43 (/v1/_internal_ping) 2020-03-31 09:04:54,454 [538] [DEBUG] [app] Ending request: urn:request:59436b17-b653-474d-ae5d-40cfdf4f1c43 (/v1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:04:54,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:04:54,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:54,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:54,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:54,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:54,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:54,477 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:54,477 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:54,493 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:54,493 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:54,497 [518] [DEBUG] [app] Starting request: urn:request:3cac9729-7926-4a03-aa37-f129fae6f956 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:54,497 [518] [DEBUG] [app] Starting request: urn:request:3cac9729-7926-4a03-aa37-f129fae6f956 (/_internal_ping) 2020-03-31 09:04:54,498 [518] [DEBUG] [app] Ending request: urn:request:3cac9729-7926-4a03-aa37-f129fae6f956 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:54,498 [518] [DEBUG] [app] Ending request: urn:request:3cac9729-7926-4a03-aa37-f129fae6f956 (/_internal_ping) 2020-03-31 09:04:54,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:54,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:04:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:04:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:04:54,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:54,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:54,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:54,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:54,511 [536] [DEBUG] [app] Starting request: urn:request:b9a800e3-a383-40e1-a392-335daaac0dfa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:54,511 [536] [DEBUG] [app] Starting request: urn:request:b9a800e3-a383-40e1-a392-335daaac0dfa (/c1/_internal_ping) 2020-03-31 09:04:54,512 [536] [DEBUG] [app] Ending request: urn:request:b9a800e3-a383-40e1-a392-335daaac0dfa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:54,512 [536] [DEBUG] [app] Ending request: urn:request:b9a800e3-a383-40e1-a392-335daaac0dfa (/c1/_internal_ping) 2020-03-31 09:04:54,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:54,512 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:54,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:54,513 [518] [DEBUG] [app] Ending request: urn:request:49a2fc11-32e7-4a63-b2d5-28e4221e14b4 (/health/instance) 2020-03-31 09:04:54,514 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:04:54,512 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:54,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:04:54,513 [518] [DEBUG] [app] Ending request: urn:request:49a2fc11-32e7-4a63-b2d5-28e4221e14b4 (/health/instance) 2020-03-31 09:04:54,514 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:54,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:04:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:04:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:06.924497+00:00 (in 9.999227 seconds) 2020-03-31 09:04:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:06 UTC)" (scheduled at 2020-03-31 09:04:56.924497+00:00) 2020-03-31 09:04:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:04:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:06.924497+00:00 (in 9.999227 seconds) 2020-03-31 09:04:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:06 UTC)" (scheduled at 2020-03-31 09:04:56.924497+00:00) 2020-03-31 09:04:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:04:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 56, 925988), True, datetime.datetime(2020, 3, 31, 9, 4, 56, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:04:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 4, 56, 925988), True, datetime.datetime(2020, 3, 31, 9, 4, 56, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:04:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:04:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:04:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:04:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:06 UTC)" executed successfully 2020-03-31 09:04:58,273 [527] [DEBUG] [app] Starting request: urn:request:2d45c228-39a4-4b96-b8ef-bbeb450744b0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:04:58,273 [527] [DEBUG] [app] Starting request: urn:request:2d45c228-39a4-4b96-b8ef-bbeb450744b0 (/health/instance) 2020-03-31 09:04:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:58,280 [555] [DEBUG] [app] Starting request: urn:request:57950bca-a6b0-4af0-bc3f-5298d55200b6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:58,280 [555] [DEBUG] [app] Starting request: urn:request:57950bca-a6b0-4af0-bc3f-5298d55200b6 (/v1/_internal_ping) 2020-03-31 09:04:58,281 [555] [DEBUG] [app] Ending request: urn:request:57950bca-a6b0-4af0-bc3f-5298d55200b6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:04:58,281 [555] [DEBUG] [app] Ending request: urn:request:57950bca-a6b0-4af0-bc3f-5298d55200b6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:04:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:04:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:04:58,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:04:58,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:58,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:04:58,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:04:58,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:04:58,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:04:58,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:04:58,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:04:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:04:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:04:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:04:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:04:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:58,314 [518] [DEBUG] [app] Starting request: urn:request:5d6edfac-b418-4c04-ac8d-8b461695633a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:58,314 [518] [DEBUG] [app] Starting request: urn:request:5d6edfac-b418-4c04-ac8d-8b461695633a (/_internal_ping) 2020-03-31 09:04:58,315 [518] [DEBUG] [app] Ending request: urn:request:5d6edfac-b418-4c04-ac8d-8b461695633a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:04:58,315 [518] [DEBUG] [app] Ending request: urn:request:5d6edfac-b418-4c04-ac8d-8b461695633a (/_internal_ping) 2020-03-31 09:04:58,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:04:58,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:04:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:58,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:58,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:04:58,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:04:58,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:04:58,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:58,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:04:58,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:04:58,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:04:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:04:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:04:58,328 [536] [DEBUG] [app] Starting request: urn:request:d0f4ddcc-5669-48eb-98ab-da3df694734c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:58,328 [536] [DEBUG] [app] Starting request: urn:request:d0f4ddcc-5669-48eb-98ab-da3df694734c (/c1/_internal_ping) 2020-03-31 09:04:58,328 [536] [DEBUG] [app] Ending request: urn:request:d0f4ddcc-5669-48eb-98ab-da3df694734c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:04:58,328 [536] [DEBUG] [app] Ending request: urn:request:d0f4ddcc-5669-48eb-98ab-da3df694734c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:04:58,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:04:58,329 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:04:58,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:04:58,329 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:04:58,330 [527] [DEBUG] [app] Ending request: urn:request:2d45c228-39a4-4b96-b8ef-bbeb450744b0 (/health/instance) 2020-03-31 09:04:58,331 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:04:58,330 [527] [DEBUG] [app] Ending request: urn:request:2d45c228-39a4-4b96-b8ef-bbeb450744b0 (/health/instance) 2020-03-31 09:04:58,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:04:58,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:04:58,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:04:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:04:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:04:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:04:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:04:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:51.110610+00:00 (in 51.549229 seconds) 2020-03-31 09:04:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:59 UTC)" (scheduled at 2020-03-31 09:04:59.560326+00:00) 2020-03-31 09:04:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:04:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:04:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:51.110610+00:00 (in 51.549229 seconds) 2020-03-31 09:04:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:59 UTC)" (scheduled at 2020-03-31 09:04:59.560326+00:00) 2020-03-31 09:04:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:04:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:05:59 UTC)" executed successfully 2020-03-31 09:05:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:05:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:30.337699+00:00 (in 29.998951 seconds) 2020-03-31 09:05:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:30 UTC)" (scheduled at 2020-03-31 09:05:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:05:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:30.337699+00:00 (in 29.998951 seconds) 2020-03-31 09:05:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:30 UTC)" (scheduled at 2020-03-31 09:05:00.337699+00:00) 2020-03-31 09:05:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 5, 0, 339478), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:05:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 5, 0, 339478), False, 50, 1, 0]) 2020-03-31 09:05:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:05:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:05:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:05:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:30 UTC)" executed successfully 2020-03-31 09:05:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:05:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:11.440707+00:00 (in 9.999265 seconds) 2020-03-31 09:05:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:11 UTC)" (scheduled at 2020-03-31 09:05:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:05:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:11.440707+00:00 (in 9.999265 seconds) 2020-03-31 09:05:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:11 UTC)" (scheduled at 2020-03-31 09:05:01.440707+00:00) 2020-03-31 09:05:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:05:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:05:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:05:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:05:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:05:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:11 UTC)" executed successfully 2020-03-31 09:05:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:05:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:24.410927+00:00 (in 21.079931 seconds) 2020-03-31 09:05:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:33 UTC)" (scheduled at 2020-03-31 09:05:03.330112+00:00) 2020-03-31 09:05:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:05:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:24.410927+00:00 (in 21.079931 seconds) 2020-03-31 09:05:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:33 UTC)" (scheduled at 2020-03-31 09:05:03.330112+00:00) 2020-03-31 09:05:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 3, 331924), True, datetime.datetime(2020, 3, 31, 9, 5, 3, 331924), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:05:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 3, 331924), True, datetime.datetime(2020, 3, 31, 9, 5, 3, 331924), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:05:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:05:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:05:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:33 UTC)" executed successfully 2020-03-31 09:05:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:05:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:14.323352+00:00 (in 9.999100 seconds) 2020-03-31 09:05:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:14 UTC)" (scheduled at 2020-03-31 09:05:04.323352+00:00) 2020-03-31 09:05:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:05:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:14.323352+00:00 (in 9.999100 seconds) 2020-03-31 09:05:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:14 UTC)" (scheduled at 2020-03-31 09:05:04.323352+00:00) 2020-03-31 09:05:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 4, 325228), True, datetime.datetime(2020, 3, 31, 9, 5, 4, 325228), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:05:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 4, 325228), True, datetime.datetime(2020, 3, 31, 9, 5, 4, 325228), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:05:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:05:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:05:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:14 UTC)" executed successfully 2020-03-31 09:05:04,446 [527] [DEBUG] [app] Starting request: urn:request:d038186d-8c94-4fe3-ba9e-effdcac047a3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:04,446 [527] [DEBUG] [app] Starting request: urn:request:d038186d-8c94-4fe3-ba9e-effdcac047a3 (/health/instance) 2020-03-31 09:05:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:04,454 [538] [DEBUG] [app] Starting request: urn:request:21e379f4-f391-4fb5-bc5e-e85b93b67847 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:04,454 [538] [DEBUG] [app] Starting request: urn:request:21e379f4-f391-4fb5-bc5e-e85b93b67847 (/v1/_internal_ping) 2020-03-31 09:05:04,455 [538] [DEBUG] [app] Ending request: urn:request:21e379f4-f391-4fb5-bc5e-e85b93b67847 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:04,455 [538] [DEBUG] [app] Ending request: urn:request:21e379f4-f391-4fb5-bc5e-e85b93b67847 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:04,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:05:04,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:05:04,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:05:04,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:04,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:04,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:04,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:04,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:04,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:04,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:04,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:04,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:04,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:04,487 [518] [DEBUG] [app] Starting request: urn:request:7a275222-f5b4-4442-8d40-db73a23566f4 (/_internal_ping) 2020-03-31 09:05:04,488 [518] [DEBUG] [app] Ending request: urn:request:7a275222-f5b4-4442-8d40-db73a23566f4 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:04,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:04,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:05:04,487 [518] [DEBUG] [app] Starting request: urn:request:7a275222-f5b4-4442-8d40-db73a23566f4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:04,488 [518] [DEBUG] [app] Ending request: urn:request:7a275222-f5b4-4442-8d40-db73a23566f4 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:04,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:05:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:05:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:05:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:05:04,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:04,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:04,501 [543] [DEBUG] [app] Starting request: urn:request:ac9d67c3-1385-42f3-9c79-03031d34e55f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:04,501 [543] [DEBUG] [app] Starting request: urn:request:ac9d67c3-1385-42f3-9c79-03031d34e55f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:04,502 [543] [DEBUG] [app] Ending request: urn:request:ac9d67c3-1385-42f3-9c79-03031d34e55f (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:04,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:05:04,502 [543] [DEBUG] [app] Ending request: urn:request:ac9d67c3-1385-42f3-9c79-03031d34e55f (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:04,504 [527] [DEBUG] [app] Ending request: urn:request:d038186d-8c94-4fe3-ba9e-effdcac047a3 (/health/instance) 2020-03-31 09:05:04,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:04,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:04,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:05:04,504 [527] [DEBUG] [app] Ending request: urn:request:d038186d-8c94-4fe3-ba9e-effdcac047a3 (/health/instance) 2020-03-31 09:05:04,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:04,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:05:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:05:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:16.924497+00:00 (in 9.998736 seconds) 2020-03-31 09:05:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:16 UTC)" (scheduled at 2020-03-31 09:05:06.924497+00:00) 2020-03-31 09:05:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:05:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:16.924497+00:00 (in 9.998736 seconds) 2020-03-31 09:05:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:16 UTC)" (scheduled at 2020-03-31 09:05:06.924497+00:00) 2020-03-31 09:05:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 6, 926583), True, datetime.datetime(2020, 3, 31, 9, 5, 6, 926583), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:05:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 6, 926583), True, datetime.datetime(2020, 3, 31, 9, 5, 6, 926583), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:05:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:05:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:05:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:16 UTC)" executed successfully 2020-03-31 09:05:08,273 [518] [DEBUG] [app] Starting request: urn:request:df43e378-8f69-4393-89a0-ea2b86851c0a (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:08,273 [518] [DEBUG] [app] Starting request: urn:request:df43e378-8f69-4393-89a0-ea2b86851c0a (/health/instance) 2020-03-31 09:05:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:08,280 [544] [DEBUG] [app] Starting request: urn:request:0b71b4c8-52ba-4ef4-bcae-0ab85859ceb6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:08,280 [544] [DEBUG] [app] Starting request: urn:request:0b71b4c8-52ba-4ef4-bcae-0ab85859ceb6 (/v1/_internal_ping) 2020-03-31 09:05:08,282 [544] [DEBUG] [app] Ending request: urn:request:0b71b4c8-52ba-4ef4-bcae-0ab85859ceb6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:08,282 [544] [DEBUG] [app] Ending request: urn:request:0b71b4c8-52ba-4ef4-bcae-0ab85859ceb6 (/v1/_internal_ping) 2020-03-31 09:05:08,282 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:05:08,282 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:08,284 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:05:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:05:08,286 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:08,284 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:08,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:08,286 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:08,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:08,312 [525] [DEBUG] [app] Starting request: urn:request:18208bb2-edaf-4776-9716-f55b4e565cb4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:08,312 [525] [DEBUG] [app] Starting request: urn:request:18208bb2-edaf-4776-9716-f55b4e565cb4 (/_internal_ping) 2020-03-31 09:05:08,313 [525] [DEBUG] [app] Ending request: urn:request:18208bb2-edaf-4776-9716-f55b4e565cb4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:08,313 [525] [DEBUG] [app] Ending request: urn:request:18208bb2-edaf-4776-9716-f55b4e565cb4 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:05:08,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:08,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:05:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:05:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:05:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:05:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:08,326 [543] [DEBUG] [app] Starting request: urn:request:f9adde59-8404-4d2c-9987-f427bddcb075 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:08,326 [543] [DEBUG] [app] Starting request: urn:request:f9adde59-8404-4d2c-9987-f427bddcb075 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:08,327 [543] [DEBUG] [app] Ending request: urn:request:f9adde59-8404-4d2c-9987-f427bddcb075 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:08,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:08,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:05:08,327 [543] [DEBUG] [app] Ending request: urn:request:f9adde59-8404-4d2c-9987-f427bddcb075 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:08,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:08,329 [518] [DEBUG] [app] Ending request: urn:request:df43e378-8f69-4393-89a0-ea2b86851c0a (/health/instance) 2020-03-31 09:05:08,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:08,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:08,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:05:08,329 [518] [DEBUG] [app] Ending request: urn:request:df43e378-8f69-4393-89a0-ea2b86851c0a (/health/instance) 2020-03-31 09:05:08,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:08,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:05:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:05:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:21.440707+00:00 (in 9.996640 seconds) 2020-03-31 09:05:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:21 UTC)" (scheduled at 2020-03-31 09:05:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:05:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:21.440707+00:00 (in 9.996640 seconds) 2020-03-31 09:05:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:21 UTC)" (scheduled at 2020-03-31 09:05:11.440707+00:00) 2020-03-31 09:05:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:05:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:05:11,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:05:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:05:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:05:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:21 UTC)" executed successfully 2020-03-31 09:05:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:05:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:24.323352+00:00 (in 9.999256 seconds) 2020-03-31 09:05:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:24 UTC)" (scheduled at 2020-03-31 09:05:14.323352+00:00) 2020-03-31 09:05:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:05:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:24.323352+00:00 (in 9.999256 seconds) 2020-03-31 09:05:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:24 UTC)" (scheduled at 2020-03-31 09:05:14.323352+00:00) 2020-03-31 09:05:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 14, 324959), True, datetime.datetime(2020, 3, 31, 9, 5, 14, 324959), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:05:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 14, 324959), True, datetime.datetime(2020, 3, 31, 9, 5, 14, 324959), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:05:14,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:14,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:05:14,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:14,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:24 UTC)" executed successfully 2020-03-31 09:05:14,446 [527] [DEBUG] [app] Starting request: urn:request:316530f0-165b-4730-a01b-fc2edaafdcc0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:14,446 [527] [DEBUG] [app] Starting request: urn:request:316530f0-165b-4730-a01b-fc2edaafdcc0 (/health/instance) 2020-03-31 09:05:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:14,454 [540] [DEBUG] [app] Starting request: urn:request:7bf78c1f-b051-461e-ab35-09d9cc37a734 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:14,454 [540] [DEBUG] [app] Starting request: urn:request:7bf78c1f-b051-461e-ab35-09d9cc37a734 (/v1/_internal_ping) 2020-03-31 09:05:14,455 [540] [DEBUG] [app] Ending request: urn:request:7bf78c1f-b051-461e-ab35-09d9cc37a734 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:14,455 [540] [DEBUG] [app] Ending request: urn:request:7bf78c1f-b051-461e-ab35-09d9cc37a734 (/v1/_internal_ping) 2020-03-31 09:05:14,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:05:14,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:05:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:05:14,459 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:05:14,459 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:14,461 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:14,461 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:14,462 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:14,462 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:14,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:14,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:14,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:14,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:14,487 [527] [DEBUG] [app] Starting request: urn:request:da442a2e-377b-422b-a573-ccf6fdbd04d1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:14,487 [527] [DEBUG] [app] Starting request: urn:request:da442a2e-377b-422b-a573-ccf6fdbd04d1 (/_internal_ping) 2020-03-31 09:05:14,488 [527] [DEBUG] [app] Ending request: urn:request:da442a2e-377b-422b-a573-ccf6fdbd04d1 (/_internal_ping) 2020-03-31 09:05:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:14,488 [527] [DEBUG] [app] Ending request: urn:request:da442a2e-377b-422b-a573-ccf6fdbd04d1 (/_internal_ping) 2020-03-31 09:05:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:05:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:05:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:05:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:05:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:14,501 [536] [DEBUG] [app] Starting request: urn:request:7d129c5f-baf1-49d3-84dc-46aeb4a255ae (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:14,501 [536] [DEBUG] [app] Starting request: urn:request:7d129c5f-baf1-49d3-84dc-46aeb4a255ae (/c1/_internal_ping) 2020-03-31 09:05:14,502 [536] [DEBUG] [app] Ending request: urn:request:7d129c5f-baf1-49d3-84dc-46aeb4a255ae (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:14,502 [536] [DEBUG] [app] Ending request: urn:request:7d129c5f-baf1-49d3-84dc-46aeb4a255ae (/c1/_internal_ping) 2020-03-31 09:05:14,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:14,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:05:14,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:14,505 [527] [DEBUG] [app] Ending request: urn:request:316530f0-165b-4730-a01b-fc2edaafdcc0 (/health/instance) 2020-03-31 09:05:14,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:14,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:14,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:05:14,505 [527] [DEBUG] [app] Ending request: urn:request:316530f0-165b-4730-a01b-fc2edaafdcc0 (/health/instance) 2020-03-31 09:05:14,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:14,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:05:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:05:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:33.366791+00:00 (in 18.825954 seconds) 2020-03-31 09:05:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:10:14 UTC)" (scheduled at 2020-03-31 09:05:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:05:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:33.366791+00:00 (in 18.825954 seconds) 2020-03-31 09:05:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:10:14 UTC)" (scheduled at 2020-03-31 09:05:14.539408+00:00) 2020-03-31 09:05:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:05:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), u'exportactionlogs/%']) 2020-03-31 09:05:14,557 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), True, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:05:14,557 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), True, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), 0, u'exportactionlogs/%']) 2020-03-31 09:05:14,561 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), True, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:05:14,561 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), True, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 5, 14, 541755), u'exportactionlogs/%']) 2020-03-31 09:05:14,565 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:05:14,565 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:14,567 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:10:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:05:14,567 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:10:14 UTC)" executed successfully 2020-03-31 09:05:14,568 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:05:14,568 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:14,569 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:05:14,569 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:14,570 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:05:14,570 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:05:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:05:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:45.064514+00:00 (in 29.999081 seconds) 2020-03-31 09:05:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:45 UTC)" (scheduled at 2020-03-31 09:05:15.064514+00:00) gcworker stdout | 2020-03-31 09:05:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:45.064514+00:00 (in 29.999081 seconds) 2020-03-31 09:05:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:45 UTC)" (scheduled at 2020-03-31 09:05:15.064514+00:00) 2020-03-31 09:05:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435915, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:05:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435915, 1209600, 500, 1, 0]) 2020-03-31 09:05:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:05:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:05:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:05:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:45 UTC)" executed successfully 2020-03-31 09:05:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:26.924497+00:00 (in 9.998874 seconds) 2020-03-31 09:05:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:26 UTC)" (scheduled at 2020-03-31 09:05:16.924497+00:00) 2020-03-31 09:05:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:05:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:26.924497+00:00 (in 9.998874 seconds) 2020-03-31 09:05:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:26 UTC)" (scheduled at 2020-03-31 09:05:16.924497+00:00) 2020-03-31 09:05:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 16, 926379), True, datetime.datetime(2020, 3, 31, 9, 5, 16, 926379), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:05:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 16, 926379), True, datetime.datetime(2020, 3, 31, 9, 5, 16, 926379), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:05:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:05:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:05:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:26 UTC)" executed successfully 2020-03-31 09:05:18,273 [518] [DEBUG] [app] Starting request: urn:request:1928fcb0-095c-4842-9131-34dc9f214b14 (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:18,273 [518] [DEBUG] [app] Starting request: urn:request:1928fcb0-095c-4842-9131-34dc9f214b14 (/health/instance) 2020-03-31 09:05:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:18,281 [547] [DEBUG] [app] Starting request: urn:request:aadd2333-6a50-451e-aff7-ea60521ede70 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:18,281 [547] [DEBUG] [app] Starting request: urn:request:aadd2333-6a50-451e-aff7-ea60521ede70 (/v1/_internal_ping) 2020-03-31 09:05:18,282 [547] [DEBUG] [app] Ending request: urn:request:aadd2333-6a50-451e-aff7-ea60521ede70 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:18,282 [547] [DEBUG] [app] Ending request: urn:request:aadd2333-6a50-451e-aff7-ea60521ede70 (/v1/_internal_ping) 2020-03-31 09:05:18,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:05:18,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:05:18,286 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:05:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:05:18,286 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:18,287 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:18,287 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:18,288 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:18,288 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:18,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:18,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:18,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:18,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:18,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:18,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:18,312 [518] [DEBUG] [app] Starting request: urn:request:f820fea8-51ce-45d4-8703-a22c247f459e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:18,312 [518] [DEBUG] [app] Starting request: urn:request:f820fea8-51ce-45d4-8703-a22c247f459e (/_internal_ping) 2020-03-31 09:05:18,313 [518] [DEBUG] [app] Ending request: urn:request:f820fea8-51ce-45d4-8703-a22c247f459e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:18,313 [518] [DEBUG] [app] Ending request: urn:request:f820fea8-51ce-45d4-8703-a22c247f459e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:05:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:05:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:05:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:05:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:05:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:05:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:18,326 [536] [DEBUG] [app] Starting request: urn:request:79980a02-73d9-445d-8253-c46c0bf920b3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:18,326 [536] [DEBUG] [app] Starting request: urn:request:79980a02-73d9-445d-8253-c46c0bf920b3 (/c1/_internal_ping) 2020-03-31 09:05:18,327 [536] [DEBUG] [app] Ending request: urn:request:79980a02-73d9-445d-8253-c46c0bf920b3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:18,327 [536] [DEBUG] [app] Ending request: urn:request:79980a02-73d9-445d-8253-c46c0bf920b3 (/c1/_internal_ping) 2020-03-31 09:05:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:18,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:05:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:05:18,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:18,330 [518] [DEBUG] [app] Ending request: urn:request:1928fcb0-095c-4842-9131-34dc9f214b14 (/health/instance) 2020-03-31 09:05:18,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:18,330 [518] [DEBUG] [app] Ending request: urn:request:1928fcb0-095c-4842-9131-34dc9f214b14 (/health/instance) 2020-03-31 09:05:18,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:18,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:05:18,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:05:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:05:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:31 UTC)" (scheduled at 2020-03-31 09:05:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:05:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:31 UTC)" (scheduled at 2020-03-31 09:05:21.440707+00:00) 2020-03-31 09:05:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:05:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:05:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:31.440707+00:00 (in 9.998761 seconds) repositoryactioncounter stdout | 2020-03-31 09:05:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:31.440707+00:00 (in 9.998761 seconds) 2020-03-31 09:05:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:05:21,461 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:05:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:05:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:05:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:31 UTC)" executed successfully 2020-03-31 09:05:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:05:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:52.113221+00:00 (in 29.999177 seconds) 2020-03-31 09:05:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:52 UTC)" (scheduled at 2020-03-31 09:05:22.113221+00:00) securityworker stdout | 2020-03-31 09:05:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:52.113221+00:00 (in 29.999177 seconds) 2020-03-31 09:05:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:52 UTC)" (scheduled at 2020-03-31 09:05:22.113221+00:00) 2020-03-31 09:05:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:05:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:05:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:05:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:05:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:05:22,131 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:05:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:05:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:05:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:05:52 UTC)" executed successfully 2020-03-31 09:05:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:05:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:34.323352+00:00 (in 9.999101 seconds) 2020-03-31 09:05:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:34 UTC)" (scheduled at 2020-03-31 09:05:24.323352+00:00) 2020-03-31 09:05:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:05:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:34.323352+00:00 (in 9.999101 seconds) 2020-03-31 09:05:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:34 UTC)" (scheduled at 2020-03-31 09:05:24.323352+00:00) 2020-03-31 09:05:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 24, 324908), True, datetime.datetime(2020, 3, 31, 9, 5, 24, 324908), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:05:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 24, 324908), True, datetime.datetime(2020, 3, 31, 9, 5, 24, 324908), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:05:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:05:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:05:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:34 UTC)" executed successfully 2020-03-31 09:05:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:05:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:24 UTC)" (scheduled at 2020-03-31 09:05:24.410927+00:00) 2020-03-31 09:05:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:33.330112+00:00 (in 8.918233 seconds) security_notification_worker stdout | 2020-03-31 09:05:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:24 UTC)" (scheduled at 2020-03-31 09:05:24.410927+00:00) 2020-03-31 09:05:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:33.330112+00:00 (in 8.918233 seconds) 2020-03-31 09:05:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:05:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:24 UTC)" executed successfully 2020-03-31 09:05:24,446 [518] [DEBUG] [app] Starting request: urn:request:ba7b75d6-f408-45ee-bf19-d650626e7875 (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:24,446 [518] [DEBUG] [app] Starting request: urn:request:ba7b75d6-f408-45ee-bf19-d650626e7875 (/health/instance) 2020-03-31 09:05:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:24,455 [555] [DEBUG] [app] Starting request: urn:request:2a810b28-604f-43b0-9d9a-3c6d09877346 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:24,455 [555] [DEBUG] [app] Starting request: urn:request:2a810b28-604f-43b0-9d9a-3c6d09877346 (/v1/_internal_ping) 2020-03-31 09:05:24,457 [555] [DEBUG] [app] Ending request: urn:request:2a810b28-604f-43b0-9d9a-3c6d09877346 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:24,457 [555] [DEBUG] [app] Ending request: urn:request:2a810b28-604f-43b0-9d9a-3c6d09877346 (/v1/_internal_ping) 2020-03-31 09:05:24,457 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:05:24,457 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:24,460 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:24,462 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:05:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:05:24,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:24,460 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:05:24,462 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:24,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:24,488 [518] [DEBUG] [app] Starting request: urn:request:adaaeabd-fc94-41e0-91b6-8c1cbafd4a29 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:24,488 [518] [DEBUG] [app] Starting request: urn:request:adaaeabd-fc94-41e0-91b6-8c1cbafd4a29 (/_internal_ping) 2020-03-31 09:05:24,489 [518] [DEBUG] [app] Ending request: urn:request:adaaeabd-fc94-41e0-91b6-8c1cbafd4a29 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:24,489 [518] [DEBUG] [app] Ending request: urn:request:adaaeabd-fc94-41e0-91b6-8c1cbafd4a29 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:05:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:05:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:05:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:05:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:24,505 [543] [DEBUG] [app] Starting request: urn:request:25153dc7-97de-4d5e-a0bb-b510498b32f0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:24,505 [543] [DEBUG] [app] Starting request: urn:request:25153dc7-97de-4d5e-a0bb-b510498b32f0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:24,506 [543] [DEBUG] [app] Ending request: urn:request:25153dc7-97de-4d5e-a0bb-b510498b32f0 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:24,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:24,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:05:24,506 [543] [DEBUG] [app] Ending request: urn:request:25153dc7-97de-4d5e-a0bb-b510498b32f0 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:24,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:24,509 [518] [DEBUG] [app] Ending request: urn:request:ba7b75d6-f408-45ee-bf19-d650626e7875 (/health/instance) 2020-03-31 09:05:24,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:24,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:24,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:05:24,509 [518] [DEBUG] [app] Ending request: urn:request:ba7b75d6-f408-45ee-bf19-d650626e7875 (/health/instance) 2020-03-31 09:05:24,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:24,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:05:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:05:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:36.924497+00:00 (in 9.999319 seconds) 2020-03-31 09:05:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:36 UTC)" (scheduled at 2020-03-31 09:05:26.924497+00:00) 2020-03-31 09:05:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:05:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:36.924497+00:00 (in 9.999319 seconds) 2020-03-31 09:05:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:36 UTC)" (scheduled at 2020-03-31 09:05:26.924497+00:00) 2020-03-31 09:05:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 26, 925923), True, datetime.datetime(2020, 3, 31, 9, 5, 26, 925923), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:05:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 26, 925923), True, datetime.datetime(2020, 3, 31, 9, 5, 26, 925923), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:05:26,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:05:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:05:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:36 UTC)" executed successfully 2020-03-31 09:05:28,273 [518] [DEBUG] [app] Starting request: urn:request:c0e5b415-fcd1-428f-a10b-d872bc35840b (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:28,273 [518] [DEBUG] [app] Starting request: urn:request:c0e5b415-fcd1-428f-a10b-d872bc35840b (/health/instance) 2020-03-31 09:05:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:28,278 [546] [DEBUG] [app] Starting request: urn:request:2a24bb75-0bc3-4eb0-83d6-df0ed21b8d61 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:28,278 [546] [DEBUG] [app] Starting request: urn:request:2a24bb75-0bc3-4eb0-83d6-df0ed21b8d61 (/v1/_internal_ping) 2020-03-31 09:05:28,279 [546] [DEBUG] [app] Ending request: urn:request:2a24bb75-0bc3-4eb0-83d6-df0ed21b8d61 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:28,279 [546] [DEBUG] [app] Ending request: urn:request:2a24bb75-0bc3-4eb0-83d6-df0ed21b8d61 (/v1/_internal_ping) 2020-03-31 09:05:28,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:05:28,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:28,282 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:05:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:05:28,282 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:28,283 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:28,283 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:28,284 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:28,284 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:28,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:28,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:28,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:28,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:28,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:28,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:28,309 [518] [DEBUG] [app] Starting request: urn:request:aac7359a-4666-46ec-ad07-d6f1628e3b29 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:28,309 [518] [DEBUG] [app] Starting request: urn:request:aac7359a-4666-46ec-ad07-d6f1628e3b29 (/_internal_ping) 2020-03-31 09:05:28,310 [518] [DEBUG] [app] Ending request: urn:request:aac7359a-4666-46ec-ad07-d6f1628e3b29 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:28,310 [518] [DEBUG] [app] Ending request: urn:request:aac7359a-4666-46ec-ad07-d6f1628e3b29 (/_internal_ping) 2020-03-31 09:05:28,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:05:28,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:28,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:05:28,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:05:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:05:28,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:05:28,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:05:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:28,322 [543] [DEBUG] [app] Starting request: urn:request:00346759-0fac-4535-90c5-b4fd468ea725 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:28,322 [543] [DEBUG] [app] Starting request: urn:request:00346759-0fac-4535-90c5-b4fd468ea725 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:28,323 [543] [DEBUG] [app] Ending request: urn:request:00346759-0fac-4535-90c5-b4fd468ea725 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:28,324 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:05:28,323 [543] [DEBUG] [app] Ending request: urn:request:00346759-0fac-4535-90c5-b4fd468ea725 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:28,325 [518] [DEBUG] [app] Ending request: urn:request:c0e5b415-fcd1-428f-a10b-d872bc35840b (/health/instance) 2020-03-31 09:05:28,325 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:28,324 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:28,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:05:28,325 [518] [DEBUG] [app] Ending request: urn:request:c0e5b415-fcd1-428f-a10b-d872bc35840b (/health/instance) 2020-03-31 09:05:28,325 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:28,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:05:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:05:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:00.337699+00:00 (in 29.999204 seconds) 2020-03-31 09:05:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:00 UTC)" (scheduled at 2020-03-31 09:05:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:05:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:00.337699+00:00 (in 29.999204 seconds) 2020-03-31 09:05:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:00 UTC)" (scheduled at 2020-03-31 09:05:30.337699+00:00) 2020-03-31 09:05:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 5, 30, 339382), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:05:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 5, 30, 339382), False, 50, 1, 0]) 2020-03-31 09:05:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:05:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:05:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:05:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:00 UTC)" executed successfully 2020-03-31 09:05:31,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:05:31,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:31,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:41.440707+00:00 (in 9.995280 seconds) 2020-03-31 09:05:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:41 UTC)" (scheduled at 2020-03-31 09:05:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:05:31,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:41.440707+00:00 (in 9.995280 seconds) 2020-03-31 09:05:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:41 UTC)" (scheduled at 2020-03-31 09:05:31.440707+00:00) 2020-03-31 09:05:31,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:05:31,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:05:31,470 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:05:31,470 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:05:31,471 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:31,471 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:05:31,471 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:31,471 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:41 UTC)" executed successfully 2020-03-31 09:05:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:05:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:03.330112+00:00 (in 29.998688 seconds) 2020-03-31 09:05:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:03 UTC)" (scheduled at 2020-03-31 09:05:33.330112+00:00) 2020-03-31 09:05:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:05:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:03.330112+00:00 (in 29.998688 seconds) 2020-03-31 09:05:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:03 UTC)" (scheduled at 2020-03-31 09:05:33.330112+00:00) 2020-03-31 09:05:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 33, 332001), True, datetime.datetime(2020, 3, 31, 9, 5, 33, 332001), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:05:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 33, 332001), True, datetime.datetime(2020, 3, 31, 9, 5, 33, 332001), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:05:33,348 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:05:33,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:05:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:03 UTC)" executed successfully 2020-03-31 09:05:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:05:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:43.539999+00:00 (in 10.172297 seconds) 2020-03-31 09:05:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:33 UTC)" (scheduled at 2020-03-31 09:05:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:05:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:43.539999+00:00 (in 10.172297 seconds) 2020-03-31 09:05:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:33 UTC)" (scheduled at 2020-03-31 09:05:33.366791+00:00) 2020-03-31 09:05:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:05:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 33, 368698), True, datetime.datetime(2020, 3, 31, 9, 5, 33, 368698), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:05:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 33, 368698), True, datetime.datetime(2020, 3, 31, 9, 5, 33, 368698), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:05:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:05:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:05:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:33 UTC)" executed successfully 2020-03-31 09:05:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:05:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:44.323352+00:00 (in 9.999245 seconds) 2020-03-31 09:05:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:44 UTC)" (scheduled at 2020-03-31 09:05:34.323352+00:00) 2020-03-31 09:05:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:05:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:44.323352+00:00 (in 9.999245 seconds) 2020-03-31 09:05:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:44 UTC)" (scheduled at 2020-03-31 09:05:34.323352+00:00) 2020-03-31 09:05:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 34, 325077), True, datetime.datetime(2020, 3, 31, 9, 5, 34, 325077), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:05:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 34, 325077), True, datetime.datetime(2020, 3, 31, 9, 5, 34, 325077), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:05:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:05:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:05:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:44 UTC)" executed successfully 2020-03-31 09:05:34,446 [518] [DEBUG] [app] Starting request: urn:request:6980f499-17ac-48c9-8af6-46fea2ca0a2c (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:34,446 [518] [DEBUG] [app] Starting request: urn:request:6980f499-17ac-48c9-8af6-46fea2ca0a2c (/health/instance) 2020-03-31 09:05:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:34,454 [538] [DEBUG] [app] Starting request: urn:request:a3b2941c-4eee-4d04-8cb3-1b375dcdd6f9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:34,454 [538] [DEBUG] [app] Starting request: urn:request:a3b2941c-4eee-4d04-8cb3-1b375dcdd6f9 (/v1/_internal_ping) 2020-03-31 09:05:34,455 [538] [DEBUG] [app] Ending request: urn:request:a3b2941c-4eee-4d04-8cb3-1b375dcdd6f9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:34,455 [538] [DEBUG] [app] Ending request: urn:request:a3b2941c-4eee-4d04-8cb3-1b375dcdd6f9 (/v1/_internal_ping) 2020-03-31 09:05:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:05:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:05:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:05:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:34,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:34,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:34,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:34,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:34,487 [518] [DEBUG] [app] Starting request: urn:request:7cd0ad51-884e-4990-ba43-b8c758ef7a98 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:34,487 [518] [DEBUG] [app] Starting request: urn:request:7cd0ad51-884e-4990-ba43-b8c758ef7a98 (/_internal_ping) 2020-03-31 09:05:34,488 [518] [DEBUG] [app] Ending request: urn:request:7cd0ad51-884e-4990-ba43-b8c758ef7a98 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:34,488 [518] [DEBUG] [app] Ending request: urn:request:7cd0ad51-884e-4990-ba43-b8c758ef7a98 (/_internal_ping) 2020-03-31 09:05:34,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:34,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:05:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:05:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:05:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:05:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:34,504 [539] [DEBUG] [app] Starting request: urn:request:f22af43c-3852-4937-9700-6ec79eff2ed4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:34,504 [539] [DEBUG] [app] Starting request: urn:request:f22af43c-3852-4937-9700-6ec79eff2ed4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:34,505 [539] [DEBUG] [app] Ending request: urn:request:f22af43c-3852-4937-9700-6ec79eff2ed4 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:34,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:05:34,505 [539] [DEBUG] [app] Ending request: urn:request:f22af43c-3852-4937-9700-6ec79eff2ed4 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:34,506 [518] [DEBUG] [app] Ending request: urn:request:6980f499-17ac-48c9-8af6-46fea2ca0a2c (/health/instance) 2020-03-31 09:05:34,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:34,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:05:34,506 [518] [DEBUG] [app] Ending request: urn:request:6980f499-17ac-48c9-8af6-46fea2ca0a2c (/health/instance) 2020-03-31 09:05:34,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:05:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:05:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:38.010934+00:00 (in 1.085698 seconds) 2020-03-31 09:05:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:46 UTC)" (scheduled at 2020-03-31 09:05:36.924497+00:00) 2020-03-31 09:05:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:05:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:38.010934+00:00 (in 1.085698 seconds) 2020-03-31 09:05:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:46 UTC)" (scheduled at 2020-03-31 09:05:36.924497+00:00) 2020-03-31 09:05:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:36,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 36, 926551), True, datetime.datetime(2020, 3, 31, 9, 5, 36, 926551), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:05:36,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 36, 926551), True, datetime.datetime(2020, 3, 31, 9, 5, 36, 926551), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:05:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:05:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:05:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:46 UTC)" executed successfully 2020-03-31 09:05:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:05:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:46.924497+00:00 (in 8.912563 seconds) 2020-03-31 09:05:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:38 UTC)" (scheduled at 2020-03-31 09:05:38.010934+00:00) 2020-03-31 09:05:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:38,013 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:05:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:46.924497+00:00 (in 8.912563 seconds) 2020-03-31 09:05:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:38 UTC)" (scheduled at 2020-03-31 09:05:38.010934+00:00) 2020-03-31 09:05:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:38,013 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:38 UTC)" executed successfully 2020-03-31 09:05:38,273 [518] [DEBUG] [app] Starting request: urn:request:43fe8ab8-3c17-4c32-bcb7-c6996936938a (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:38,273 [518] [DEBUG] [app] Starting request: urn:request:43fe8ab8-3c17-4c32-bcb7-c6996936938a (/health/instance) 2020-03-31 09:05:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:38,278 [550] [DEBUG] [app] Starting request: urn:request:4379d1b2-f85e-40b7-956a-71ab372dbb4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:38,278 [550] [DEBUG] [app] Starting request: urn:request:4379d1b2-f85e-40b7-956a-71ab372dbb4c (/v1/_internal_ping) 2020-03-31 09:05:38,280 [550] [DEBUG] [app] Ending request: urn:request:4379d1b2-f85e-40b7-956a-71ab372dbb4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:38,280 [550] [DEBUG] [app] Ending request: urn:request:4379d1b2-f85e-40b7-956a-71ab372dbb4c (/v1/_internal_ping) 2020-03-31 09:05:38,281 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-registry stdout | 2020-03-31 09:05:38,281 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-web stdout | 2020-03-31 09:05:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:05:38,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:05:38,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:38,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:38,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:38,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:38,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:38,312 [518] [DEBUG] [app] Starting request: urn:request:32eaecd0-145f-4d94-88fe-3c2f97dd16e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:38,312 [518] [DEBUG] [app] Starting request: urn:request:32eaecd0-145f-4d94-88fe-3c2f97dd16e3 (/_internal_ping) 2020-03-31 09:05:38,313 [518] [DEBUG] [app] Ending request: urn:request:32eaecd0-145f-4d94-88fe-3c2f97dd16e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:38,313 [518] [DEBUG] [app] Ending request: urn:request:32eaecd0-145f-4d94-88fe-3c2f97dd16e3 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:05:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:05:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:05:38,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:38,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:38,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:05:38,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:38,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:38,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:38,327 [539] [DEBUG] [app] Starting request: urn:request:714b803f-2b13-4344-8c30-41e75d1e63c0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:38,327 [539] [DEBUG] [app] Starting request: urn:request:714b803f-2b13-4344-8c30-41e75d1e63c0 (/c1/_internal_ping) 2020-03-31 09:05:38,328 [539] [DEBUG] [app] Ending request: urn:request:714b803f-2b13-4344-8c30-41e75d1e63c0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:38,328 [539] [DEBUG] [app] Ending request: urn:request:714b803f-2b13-4344-8c30-41e75d1e63c0 (/c1/_internal_ping) 2020-03-31 09:05:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:38,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:05:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:38,330 [518] [DEBUG] [app] Ending request: urn:request:43fe8ab8-3c17-4c32-bcb7-c6996936938a (/health/instance) 2020-03-31 09:05:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:38,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:05:38,330 [518] [DEBUG] [app] Ending request: urn:request:43fe8ab8-3c17-4c32-bcb7-c6996936938a (/health/instance) 2020-03-31 09:05:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:05:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:05:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:51 UTC)" (scheduled at 2020-03-31 09:05:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:05:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:51 UTC)" (scheduled at 2020-03-31 09:05:41.440707+00:00) 2020-03-31 09:05:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:51.440707+00:00 (in 9.998665 seconds) 2020-03-31 09:05:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:05:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:51.440707+00:00 (in 9.998665 seconds) 2020-03-31 09:05:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:05:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:05:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:05:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:05:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:51 UTC)" executed successfully 2020-03-31 09:05:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:05:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:33.366791+00:00 (in 49.825963 seconds) 2020-03-31 09:05:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:43 UTC)" (scheduled at 2020-03-31 09:05:43.539999+00:00) 2020-03-31 09:05:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:05:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:33.366791+00:00 (in 49.825963 seconds) 2020-03-31 09:05:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:43 UTC)" (scheduled at 2020-03-31 09:05:43.539999+00:00) 2020-03-31 09:05:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:43 UTC)" executed successfully 2020-03-31 09:05:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:05:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:49.498966+00:00 (in 5.173237 seconds) 2020-03-31 09:05:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:54 UTC)" (scheduled at 2020-03-31 09:05:44.323352+00:00) 2020-03-31 09:05:44,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:05:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:49.498966+00:00 (in 5.173237 seconds) 2020-03-31 09:05:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:54 UTC)" (scheduled at 2020-03-31 09:05:44.323352+00:00) 2020-03-31 09:05:44,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 44, 326556), True, datetime.datetime(2020, 3, 31, 9, 5, 44, 326556), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:05:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 44, 326556), True, datetime.datetime(2020, 3, 31, 9, 5, 44, 326556), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:05:44,351 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:05:44,351 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:44,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:44,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:05:44,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:44,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:54 UTC)" executed successfully 2020-03-31 09:05:44,446 [518] [DEBUG] [app] Starting request: urn:request:2eda8267-8abb-4535-8de3-f573958d7c4b (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:44,446 [518] [DEBUG] [app] Starting request: urn:request:2eda8267-8abb-4535-8de3-f573958d7c4b (/health/instance) 2020-03-31 09:05:44,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:44,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:44,455 [553] [DEBUG] [app] Starting request: urn:request:e02db247-cc68-4f88-a9e9-569c4fb8b4c1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:44,455 [553] [DEBUG] [app] Starting request: urn:request:e02db247-cc68-4f88-a9e9-569c4fb8b4c1 (/v1/_internal_ping) 2020-03-31 09:05:44,456 [553] [DEBUG] [app] Ending request: urn:request:e02db247-cc68-4f88-a9e9-569c4fb8b4c1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:44,456 [553] [DEBUG] [app] Ending request: urn:request:e02db247-cc68-4f88-a9e9-569c4fb8b4c1 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:44,457 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:05:44,457 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:05:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:05:44,460 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:05:44,460 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:44,462 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:44,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:44,462 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:44,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:44,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:44,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:44,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:44,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:44,488 [518] [DEBUG] [app] Starting request: urn:request:b130949d-1ed3-4b65-ad56-73b38c816fee (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:44,488 [518] [DEBUG] [app] Starting request: urn:request:b130949d-1ed3-4b65-ad56-73b38c816fee (/_internal_ping) 2020-03-31 09:05:44,489 [518] [DEBUG] [app] Ending request: urn:request:b130949d-1ed3-4b65-ad56-73b38c816fee (/_internal_ping) 2020-03-31 09:05:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:44,489 [518] [DEBUG] [app] Ending request: urn:request:b130949d-1ed3-4b65-ad56-73b38c816fee (/_internal_ping) 2020-03-31 09:05:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:05:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:05:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:44,504 [543] [DEBUG] [app] Starting request: urn:request:634752c0-c20e-4574-9704-13ff06337c02 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:44,504 [543] [DEBUG] [app] Starting request: urn:request:634752c0-c20e-4574-9704-13ff06337c02 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:44,504 [543] [DEBUG] [app] Ending request: urn:request:634752c0-c20e-4574-9704-13ff06337c02 (/c1/_internal_ping) 2020-03-31 09:05:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:44,505 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:05:44,504 [543] [DEBUG] [app] Ending request: urn:request:634752c0-c20e-4574-9704-13ff06337c02 (/c1/_internal_ping) 2020-03-31 09:05:44,507 [518] [DEBUG] [app] Ending request: urn:request:2eda8267-8abb-4535-8de3-f573958d7c4b (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:44,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:05:44,505 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:05:44,507 [518] [DEBUG] [app] Ending request: urn:request:2eda8267-8abb-4535-8de3-f573958d7c4b (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:44,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:05:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:05:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:15.064514+00:00 (in 29.999050 seconds) 2020-03-31 09:05:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:15 UTC)" (scheduled at 2020-03-31 09:05:45.064514+00:00) gcworker stdout | 2020-03-31 09:05:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:15.064514+00:00 (in 29.999050 seconds) 2020-03-31 09:05:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:15 UTC)" (scheduled at 2020-03-31 09:05:45.064514+00:00) 2020-03-31 09:05:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435945, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:05:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435945, 1209600, 500, 1, 0]) 2020-03-31 09:05:45,093 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:05:45,093 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:05:45,093 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:45,094 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:05:45,093 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:45,094 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:15 UTC)" executed successfully 2020-03-31 09:05:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:05:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:56.924497+00:00 (in 9.999146 seconds) 2020-03-31 09:05:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:56 UTC)" (scheduled at 2020-03-31 09:05:46.924497+00:00) 2020-03-31 09:05:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:05:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:56.924497+00:00 (in 9.999146 seconds) 2020-03-31 09:05:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:56 UTC)" (scheduled at 2020-03-31 09:05:46.924497+00:00) 2020-03-31 09:05:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 46, 926273), True, datetime.datetime(2020, 3, 31, 9, 5, 46, 926273), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:05:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 46, 926273), True, datetime.datetime(2020, 3, 31, 9, 5, 46, 926273), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:05:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:05:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:05:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:05:56 UTC)" executed successfully 2020-03-31 09:05:48,273 [518] [DEBUG] [app] Starting request: urn:request:dae771f1-7410-4d47-8966-2de840ee22ac (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:48,273 [518] [DEBUG] [app] Starting request: urn:request:dae771f1-7410-4d47-8966-2de840ee22ac (/health/instance) 2020-03-31 09:05:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:48,279 [538] [DEBUG] [app] Starting request: urn:request:80a5bf0f-8207-463f-81a7-876ea92ce99c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:48,279 [538] [DEBUG] [app] Starting request: urn:request:80a5bf0f-8207-463f-81a7-876ea92ce99c (/v1/_internal_ping) 2020-03-31 09:05:48,280 [538] [DEBUG] [app] Ending request: urn:request:80a5bf0f-8207-463f-81a7-876ea92ce99c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:48,280 [538] [DEBUG] [app] Ending request: urn:request:80a5bf0f-8207-463f-81a7-876ea92ce99c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:48,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:05:48,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:05:48,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:05:48,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:48,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:48,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:48,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:48,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:48,310 [518] [DEBUG] [app] Starting request: urn:request:6f03ee0c-f5ed-4a39-85d8-440e118c351e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:48,310 [518] [DEBUG] [app] Starting request: urn:request:6f03ee0c-f5ed-4a39-85d8-440e118c351e (/_internal_ping) 2020-03-31 09:05:48,311 [518] [DEBUG] [app] Ending request: urn:request:6f03ee0c-f5ed-4a39-85d8-440e118c351e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:48,311 [518] [DEBUG] [app] Ending request: urn:request:6f03ee0c-f5ed-4a39-85d8-440e118c351e (/_internal_ping) 2020-03-31 09:05:48,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:05:48,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:05:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:05:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:05:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:48,325 [543] [DEBUG] [app] Starting request: urn:request:eea9e301-492b-43f1-8aaf-156681a5e7f0 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:05:48,325 [543] [DEBUG] [app] Starting request: urn:request:eea9e301-492b-43f1-8aaf-156681a5e7f0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:48,326 [543] [DEBUG] [app] Ending request: urn:request:eea9e301-492b-43f1-8aaf-156681a5e7f0 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:05:48,326 [543] [DEBUG] [app] Ending request: urn:request:eea9e301-492b-43f1-8aaf-156681a5e7f0 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:05:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:48,328 [518] [DEBUG] [app] Ending request: urn:request:dae771f1-7410-4d47-8966-2de840ee22ac (/health/instance) 2020-03-31 09:05:48,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:48,328 [518] [DEBUG] [app] Ending request: urn:request:dae771f1-7410-4d47-8966-2de840ee22ac (/health/instance) 2020-03-31 09:05:48,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:05:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:05:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:05:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:54.323352+00:00 (in 4.823704 seconds) 2020-03-31 09:05:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:49 UTC)" (scheduled at 2020-03-31 09:05:49.498966+00:00) 2020-03-31 09:05:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:05:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:54.323352+00:00 (in 4.823704 seconds) 2020-03-31 09:05:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:49 UTC)" (scheduled at 2020-03-31 09:05:49.498966+00:00) 2020-03-31 09:05:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:49 UTC)" executed successfully 2020-03-31 09:05:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:05:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:51 UTC)" (scheduled at 2020-03-31 09:05:51.110610+00:00) 2020-03-31 09:05:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:05:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:51 UTC)" (scheduled at 2020-03-31 09:05:51.110610+00:00) 2020-03-31 09:05:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:59.560326+00:00 (in 8.448393 seconds) namespacegcworker stdout | 2020-03-31 09:05:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:05:59.560326+00:00 (in 8.448393 seconds) 2020-03-31 09:05:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 51, 112673), True, datetime.datetime(2020, 3, 31, 9, 5, 51, 112673), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:05:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 51, 112673), True, datetime.datetime(2020, 3, 31, 9, 5, 51, 112673), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:05:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:05:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:05:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:51 UTC)" executed successfully 2020-03-31 09:05:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:05:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:01.440707+00:00 (in 9.995692 seconds) 2020-03-31 09:05:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:01 UTC)" (scheduled at 2020-03-31 09:05:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:05:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:01.440707+00:00 (in 9.995692 seconds) 2020-03-31 09:05:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:01 UTC)" (scheduled at 2020-03-31 09:05:51.440707+00:00) 2020-03-31 09:05:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:05:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:05:51,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:05:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:05:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:05:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:01 UTC)" executed successfully 2020-03-31 09:05:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:05:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:22 UTC)" (scheduled at 2020-03-31 09:05:52.113221+00:00) securityworker stdout | 2020-03-31 09:05:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:22 UTC)" (scheduled at 2020-03-31 09:05:52.113221+00:00) 2020-03-31 09:05:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:05:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:22.113221+00:00 (in 29.998723 seconds) securityworker stdout | 2020-03-31 09:05:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:05:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:22.113221+00:00 (in 29.998723 seconds) 2020-03-31 09:05:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:05:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:05:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:05:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:05:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:05:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:22 UTC)" executed successfully 2020-03-31 09:05:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:05:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:04.323352+00:00 (in 9.998828 seconds) 2020-03-31 09:05:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:04 UTC)" (scheduled at 2020-03-31 09:05:54.323352+00:00) 2020-03-31 09:05:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:05:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:04.323352+00:00 (in 9.998828 seconds) 2020-03-31 09:05:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:04 UTC)" (scheduled at 2020-03-31 09:05:54.323352+00:00) 2020-03-31 09:05:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 54, 325436), True, datetime.datetime(2020, 3, 31, 9, 5, 54, 325436), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:05:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 54, 325436), True, datetime.datetime(2020, 3, 31, 9, 5, 54, 325436), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:05:54,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:54,340 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:05:54,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:54,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:05:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:04 UTC)" executed successfully 2020-03-31 09:05:54,446 [518] [DEBUG] [app] Starting request: urn:request:b1151bf4-fe0a-4188-8a88-a7dd2a6572e4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:54,446 [518] [DEBUG] [app] Starting request: urn:request:b1151bf4-fe0a-4188-8a88-a7dd2a6572e4 (/health/instance) 2020-03-31 09:05:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:54,453 [554] [DEBUG] [app] Starting request: urn:request:09a714d6-400d-4263-8644-c8c7e8cde0af (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:54,453 [554] [DEBUG] [app] Starting request: urn:request:09a714d6-400d-4263-8644-c8c7e8cde0af (/v1/_internal_ping) 2020-03-31 09:05:54,454 [554] [DEBUG] [app] Ending request: urn:request:09a714d6-400d-4263-8644-c8c7e8cde0af (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:54,454 [554] [DEBUG] [app] Ending request: urn:request:09a714d6-400d-4263-8644-c8c7e8cde0af (/v1/_internal_ping) 2020-03-31 09:05:54,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:05:54,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:54,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:05:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:05:54,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:54,458 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:05:54,458 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:54,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:54,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:54,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:54,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:54,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:54,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:54,488 [518] [DEBUG] [app] Starting request: urn:request:33c9940d-c336-49e5-9cf2-3cb58cb38952 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:54,488 [518] [DEBUG] [app] Starting request: urn:request:33c9940d-c336-49e5-9cf2-3cb58cb38952 (/_internal_ping) 2020-03-31 09:05:54,489 [518] [DEBUG] [app] Ending request: urn:request:33c9940d-c336-49e5-9cf2-3cb58cb38952 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:54,489 [518] [DEBUG] [app] Ending request: urn:request:33c9940d-c336-49e5-9cf2-3cb58cb38952 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:05:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:05:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:05:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:54,502 [543] [DEBUG] [app] Starting request: urn:request:79698e53-1b5a-48e3-bb63-335ef2a187c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:54,502 [543] [DEBUG] [app] Starting request: urn:request:79698e53-1b5a-48e3-bb63-335ef2a187c7 (/c1/_internal_ping) 2020-03-31 09:05:54,502 [543] [DEBUG] [app] Ending request: urn:request:79698e53-1b5a-48e3-bb63-335ef2a187c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:54,502 [543] [DEBUG] [app] Ending request: urn:request:79698e53-1b5a-48e3-bb63-335ef2a187c7 (/c1/_internal_ping) 2020-03-31 09:05:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:54,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:05:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:05:54,505 [518] [DEBUG] [app] Ending request: urn:request:b1151bf4-fe0a-4188-8a88-a7dd2a6572e4 (/health/instance) 2020-03-31 09:05:54,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:05:54,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:54,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:05:54,505 [518] [DEBUG] [app] Ending request: urn:request:b1151bf4-fe0a-4188-8a88-a7dd2a6572e4 (/health/instance) 2020-03-31 09:05:54,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:54,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:05:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:05:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:06.924497+00:00 (in 9.999260 seconds) 2020-03-31 09:05:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:06 UTC)" (scheduled at 2020-03-31 09:05:56.924497+00:00) 2020-03-31 09:05:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:05:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:06.924497+00:00 (in 9.999260 seconds) 2020-03-31 09:05:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:06 UTC)" (scheduled at 2020-03-31 09:05:56.924497+00:00) 2020-03-31 09:05:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:05:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 56, 925994), True, datetime.datetime(2020, 3, 31, 9, 5, 56, 925994), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:05:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 5, 56, 925994), True, datetime.datetime(2020, 3, 31, 9, 5, 56, 925994), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:05:56,966 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:56,966 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:05:56,966 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:05:56,966 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:56,966 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:05:56,966 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:06 UTC)" executed successfully 2020-03-31 09:05:58,273 [518] [DEBUG] [app] Starting request: urn:request:1fa2f91f-6c3b-4fbe-8edb-fb9374c75777 (/health/instance) gunicorn-web stdout | 2020-03-31 09:05:58,273 [518] [DEBUG] [app] Starting request: urn:request:1fa2f91f-6c3b-4fbe-8edb-fb9374c75777 (/health/instance) 2020-03-31 09:05:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:58,279 [556] [DEBUG] [app] Starting request: urn:request:bc376982-c0c2-4ac5-a5cc-6d364a0781e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:58,279 [556] [DEBUG] [app] Starting request: urn:request:bc376982-c0c2-4ac5-a5cc-6d364a0781e9 (/v1/_internal_ping) 2020-03-31 09:05:58,280 [556] [DEBUG] [app] Ending request: urn:request:bc376982-c0c2-4ac5-a5cc-6d364a0781e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:05:58,280 [556] [DEBUG] [app] Ending request: urn:request:bc376982-c0c2-4ac5-a5cc-6d364a0781e9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:58,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:05:58,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:05:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:58,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:05:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:05:58,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:05:58,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:58,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:05:58,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:05:58,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:05:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:05:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:05:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:05:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:05:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:58,310 [521] [DEBUG] [app] Starting request: urn:request:727ce348-157a-4abf-8ab0-c1770add84ca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:58,310 [521] [DEBUG] [app] Starting request: urn:request:727ce348-157a-4abf-8ab0-c1770add84ca (/_internal_ping) 2020-03-31 09:05:58,311 [521] [DEBUG] [app] Ending request: urn:request:727ce348-157a-4abf-8ab0-c1770add84ca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:05:58,311 [521] [DEBUG] [app] Ending request: urn:request:727ce348-157a-4abf-8ab0-c1770add84ca (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:05:58,311 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:58,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:05:58,311 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:58,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:05:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:05:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:05:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:05:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:05:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:05:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:05:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:05:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:05:58,323 [543] [DEBUG] [app] Starting request: urn:request:e9d749eb-7b12-440f-a1d8-6f92e28b4a07 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:58,323 [543] [DEBUG] [app] Starting request: urn:request:e9d749eb-7b12-440f-a1d8-6f92e28b4a07 (/c1/_internal_ping) 2020-03-31 09:05:58,325 [543] [DEBUG] [app] Ending request: urn:request:e9d749eb-7b12-440f-a1d8-6f92e28b4a07 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:05:58,325 [543] [DEBUG] [app] Ending request: urn:request:e9d749eb-7b12-440f-a1d8-6f92e28b4a07 (/c1/_internal_ping) 2020-03-31 09:05:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:05:58,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:05:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:05:58,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:05:58,327 [518] [DEBUG] [app] Ending request: urn:request:1fa2f91f-6c3b-4fbe-8edb-fb9374c75777 (/health/instance) 2020-03-31 09:05:58,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:05:58,327 [518] [DEBUG] [app] Ending request: urn:request:1fa2f91f-6c3b-4fbe-8edb-fb9374c75777 (/health/instance) 2020-03-31 09:05:58,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:05:58,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:05:58,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:05:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:05:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:05:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:05:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:05:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:51.110610+00:00 (in 51.547521 seconds) 2020-03-31 09:05:59,563 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:59 UTC)" (scheduled at 2020-03-31 09:05:59.560326+00:00) 2020-03-31 09:05:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:05:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:51.110610+00:00 (in 51.547521 seconds) 2020-03-31 09:05:59,563 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:59 UTC)" (scheduled at 2020-03-31 09:05:59.560326+00:00) 2020-03-31 09:05:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:05:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:06:59 UTC)" executed successfully 2020-03-31 09:06:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:06:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:30.337699+00:00 (in 29.999109 seconds) 2020-03-31 09:06:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:30 UTC)" (scheduled at 2020-03-31 09:06:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:06:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:30.337699+00:00 (in 29.999109 seconds) 2020-03-31 09:06:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:30 UTC)" (scheduled at 2020-03-31 09:06:00.337699+00:00) 2020-03-31 09:06:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 6, 0, 339351), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:06:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 6, 0, 339351), False, 50, 1, 0]) 2020-03-31 09:06:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:06:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:06:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:06:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:30 UTC)" executed successfully 2020-03-31 09:06:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:06:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:11.440707+00:00 (in 9.999282 seconds) 2020-03-31 09:06:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:11 UTC)" (scheduled at 2020-03-31 09:06:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:06:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:11.440707+00:00 (in 9.999282 seconds) 2020-03-31 09:06:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:11 UTC)" (scheduled at 2020-03-31 09:06:01.440707+00:00) 2020-03-31 09:06:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:06:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:06:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:06:01,458 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:06:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:06:01,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:01,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:06:01,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:11 UTC)" executed successfully 2020-03-31 09:06:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:06:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:24.410927+00:00 (in 21.080075 seconds) 2020-03-31 09:06:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:33 UTC)" (scheduled at 2020-03-31 09:06:03.330112+00:00) 2020-03-31 09:06:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:06:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:24.410927+00:00 (in 21.080075 seconds) 2020-03-31 09:06:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:33 UTC)" (scheduled at 2020-03-31 09:06:03.330112+00:00) 2020-03-31 09:06:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 3, 331857), True, datetime.datetime(2020, 3, 31, 9, 6, 3, 331857), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:06:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 3, 331857), True, datetime.datetime(2020, 3, 31, 9, 6, 3, 331857), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:06:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:06:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:06:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:33 UTC)" executed successfully 2020-03-31 09:06:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:06:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:14.323352+00:00 (in 9.999113 seconds) 2020-03-31 09:06:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:14 UTC)" (scheduled at 2020-03-31 09:06:04.323352+00:00) 2020-03-31 09:06:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:06:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:14.323352+00:00 (in 9.999113 seconds) 2020-03-31 09:06:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:14 UTC)" (scheduled at 2020-03-31 09:06:04.323352+00:00) 2020-03-31 09:06:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 4, 325331), True, datetime.datetime(2020, 3, 31, 9, 6, 4, 325331), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:06:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 4, 325331), True, datetime.datetime(2020, 3, 31, 9, 6, 4, 325331), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:06:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:06:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:06:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:14 UTC)" executed successfully 2020-03-31 09:06:04,445 [518] [DEBUG] [app] Starting request: urn:request:f5115261-455e-450b-b65a-086cdc086a1f (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:04,445 [518] [DEBUG] [app] Starting request: urn:request:f5115261-455e-450b-b65a-086cdc086a1f (/health/instance) 2020-03-31 09:06:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:04,453 [551] [DEBUG] [app] Starting request: urn:request:7b6d0dae-325a-4da8-b736-924cae70b873 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:04,453 [551] [DEBUG] [app] Starting request: urn:request:7b6d0dae-325a-4da8-b736-924cae70b873 (/v1/_internal_ping) 2020-03-31 09:06:04,454 [551] [DEBUG] [app] Ending request: urn:request:7b6d0dae-325a-4da8-b736-924cae70b873 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:04,454 [551] [DEBUG] [app] Ending request: urn:request:7b6d0dae-325a-4da8-b736-924cae70b873 (/v1/_internal_ping) 2020-03-31 09:06:04,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:06:04,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:04,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:06:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:06:04,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:04,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:04,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:04,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:04,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:04,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:04,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:04,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:04,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:04,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:04,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:04,483 [520] [DEBUG] [app] Starting request: urn:request:5aa6951c-1fd7-4960-a15b-b684808ca7df (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:04,483 [520] [DEBUG] [app] Starting request: urn:request:5aa6951c-1fd7-4960-a15b-b684808ca7df (/_internal_ping) 2020-03-31 09:06:04,484 [520] [DEBUG] [app] Ending request: urn:request:5aa6951c-1fd7-4960-a15b-b684808ca7df (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:04,484 [520] [DEBUG] [app] Ending request: urn:request:5aa6951c-1fd7-4960-a15b-b684808ca7df (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:06:04,485 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:04,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:04,485 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:04,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:06:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:06:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:06:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:04,499 [543] [DEBUG] [app] Starting request: urn:request:fd373faa-d781-46e1-adf7-c779fe4c26b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:04,499 [543] [DEBUG] [app] Starting request: urn:request:fd373faa-d781-46e1-adf7-c779fe4c26b1 (/c1/_internal_ping) 2020-03-31 09:06:04,502 [543] [DEBUG] [app] Ending request: urn:request:fd373faa-d781-46e1-adf7-c779fe4c26b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:04,502 [543] [DEBUG] [app] Ending request: urn:request:fd373faa-d781-46e1-adf7-c779fe4c26b1 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:06:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:04,504 [518] [DEBUG] [app] Ending request: urn:request:f5115261-455e-450b-b65a-086cdc086a1f (/health/instance) 2020-03-31 09:06:04,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:04,504 [518] [DEBUG] [app] Ending request: urn:request:f5115261-455e-450b-b65a-086cdc086a1f (/health/instance) 2020-03-31 09:06:04,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:06:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:06:04,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-verbs stdout | 2020-03-31 09:06:04,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:16.924497+00:00 (in 9.999171 seconds) 2020-03-31 09:06:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:16 UTC)" (scheduled at 2020-03-31 09:06:06.924497+00:00) 2020-03-31 09:06:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 6, 926074), True, datetime.datetime(2020, 3, 31, 9, 6, 6, 926074), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:06:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:16.924497+00:00 (in 9.999171 seconds) 2020-03-31 09:06:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:16 UTC)" (scheduled at 2020-03-31 09:06:06.924497+00:00) 2020-03-31 09:06:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 6, 926074), True, datetime.datetime(2020, 3, 31, 9, 6, 6, 926074), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:06:06,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:06:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:06:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:16 UTC)" executed successfully 2020-03-31 09:06:08,273 [527] [DEBUG] [app] Starting request: urn:request:94f93ee9-cdbd-401a-aba2-182a410394ab (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:08,273 [527] [DEBUG] [app] Starting request: urn:request:94f93ee9-cdbd-401a-aba2-182a410394ab (/health/instance) 2020-03-31 09:06:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:08,280 [538] [DEBUG] [app] Starting request: urn:request:86e6294a-89fa-4290-93ed-b8f1ea69c770 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:08,280 [538] [DEBUG] [app] Starting request: urn:request:86e6294a-89fa-4290-93ed-b8f1ea69c770 (/v1/_internal_ping) 2020-03-31 09:06:08,281 [538] [DEBUG] [app] Ending request: urn:request:86e6294a-89fa-4290-93ed-b8f1ea69c770 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:08,281 [538] [DEBUG] [app] Ending request: urn:request:86e6294a-89fa-4290-93ed-b8f1ea69c770 (/v1/_internal_ping) 2020-03-31 09:06:08,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:06:08,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:08,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:08,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:08,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:08,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:08,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:08,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:08,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:08,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:08,314 [527] [DEBUG] [app] Starting request: urn:request:7ce80a95-463e-4a3a-a349-5c07ef18faba (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:08,314 [527] [DEBUG] [app] Starting request: urn:request:7ce80a95-463e-4a3a-a349-5c07ef18faba (/_internal_ping) 2020-03-31 09:06:08,315 [527] [DEBUG] [app] Ending request: urn:request:7ce80a95-463e-4a3a-a349-5c07ef18faba (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:08,315 [527] [DEBUG] [app] Ending request: urn:request:7ce80a95-463e-4a3a-a349-5c07ef18faba (/_internal_ping) 2020-03-31 09:06:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:08,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:08,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:06:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:06:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:06:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:08,329 [543] [DEBUG] [app] Starting request: urn:request:99131cfa-9fbc-47f1-99cb-22b075a81a47 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:08,329 [543] [DEBUG] [app] Starting request: urn:request:99131cfa-9fbc-47f1-99cb-22b075a81a47 (/c1/_internal_ping) 2020-03-31 09:06:08,330 [543] [DEBUG] [app] Ending request: urn:request:99131cfa-9fbc-47f1-99cb-22b075a81a47 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:08,330 [543] [DEBUG] [app] Ending request: urn:request:99131cfa-9fbc-47f1-99cb-22b075a81a47 (/c1/_internal_ping) 2020-03-31 09:06:08,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:08,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:08,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:06:08,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:08,332 [527] [DEBUG] [app] Ending request: urn:request:94f93ee9-cdbd-401a-aba2-182a410394ab (/health/instance) 2020-03-31 09:06:08,333 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:06:08,332 [527] [DEBUG] [app] Ending request: urn:request:94f93ee9-cdbd-401a-aba2-182a410394ab (/health/instance) 2020-03-31 09:06:08,333 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:08,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:06:08,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:06:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:06:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:21.440707+00:00 (in 9.998892 seconds) 2020-03-31 09:06:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:21 UTC)" (scheduled at 2020-03-31 09:06:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:06:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:21.440707+00:00 (in 9.998892 seconds) 2020-03-31 09:06:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:21 UTC)" (scheduled at 2020-03-31 09:06:11.440707+00:00) 2020-03-31 09:06:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:06:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:06:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:06:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:06:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:06:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:21 UTC)" executed successfully 2020-03-31 09:06:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:06:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:24.323352+00:00 (in 9.995531 seconds) 2020-03-31 09:06:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:24 UTC)" (scheduled at 2020-03-31 09:06:14.323352+00:00) 2020-03-31 09:06:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:06:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:24.323352+00:00 (in 9.995531 seconds) 2020-03-31 09:06:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:24 UTC)" (scheduled at 2020-03-31 09:06:14.323352+00:00) 2020-03-31 09:06:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 14, 328883), True, datetime.datetime(2020, 3, 31, 9, 6, 14, 328883), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:06:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 14, 328883), True, datetime.datetime(2020, 3, 31, 9, 6, 14, 328883), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:06:14,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:06:14,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:14,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:14,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:06:14,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:14,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:24 UTC)" executed successfully 2020-03-31 09:06:14,446 [526] [DEBUG] [app] Starting request: urn:request:c929b8c4-d324-4987-bcc5-23e9640dc7c7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:14,446 [526] [DEBUG] [app] Starting request: urn:request:c929b8c4-d324-4987-bcc5-23e9640dc7c7 (/health/instance) 2020-03-31 09:06:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:14,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:14,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:14,455 [544] [DEBUG] [app] Starting request: urn:request:206068b7-c637-41f2-b898-230e2119ea56 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:14,455 [544] [DEBUG] [app] Starting request: urn:request:206068b7-c637-41f2-b898-230e2119ea56 (/v1/_internal_ping) 2020-03-31 09:06:14,456 [544] [DEBUG] [app] Ending request: urn:request:206068b7-c637-41f2-b898-230e2119ea56 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:14,456 [544] [DEBUG] [app] Ending request: urn:request:206068b7-c637-41f2-b898-230e2119ea56 (/v1/_internal_ping) 2020-03-31 09:06:14,457 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:14,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:06:14,457 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:14,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:14,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:06:14,460 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:06:14,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:06:14,460 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:14,461 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:14,461 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:14,462 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:14,462 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:14,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:14,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:14,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:14,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:14,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:14,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:14,491 [526] [DEBUG] [app] Starting request: urn:request:88df1a42-51b8-4deb-97f8-cfa5257fba03 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:14,491 [526] [DEBUG] [app] Starting request: urn:request:88df1a42-51b8-4deb-97f8-cfa5257fba03 (/_internal_ping) 2020-03-31 09:06:14,491 [526] [DEBUG] [app] Ending request: urn:request:88df1a42-51b8-4deb-97f8-cfa5257fba03 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:14,491 [526] [DEBUG] [app] Ending request: urn:request:88df1a42-51b8-4deb-97f8-cfa5257fba03 (/_internal_ping) 2020-03-31 09:06:14,492 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:14,492 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:14,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:14,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:06:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:06:14,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:14,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:14,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:06:14,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:06:14,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:14,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:14,504 [543] [DEBUG] [app] Starting request: urn:request:ba4d07dc-3c5e-4f82-8b6f-39825c7e595d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:14,504 [543] [DEBUG] [app] Starting request: urn:request:ba4d07dc-3c5e-4f82-8b6f-39825c7e595d (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:14,505 [543] [DEBUG] [app] Ending request: urn:request:ba4d07dc-3c5e-4f82-8b6f-39825c7e595d (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:14,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:14,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:06:14,505 [543] [DEBUG] [app] Ending request: urn:request:ba4d07dc-3c5e-4f82-8b6f-39825c7e595d (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:14,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:14,508 [526] [DEBUG] [app] Ending request: urn:request:c929b8c4-d324-4987-bcc5-23e9640dc7c7 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:06:14,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:14,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:14,508 [526] [DEBUG] [app] Ending request: urn:request:c929b8c4-d324-4987-bcc5-23e9640dc7c7 (/health/instance) 2020-03-31 09:06:14,508 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:06:14,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:14,508 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:06:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:06:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:45.064514+00:00 (in 29.999076 seconds) 2020-03-31 09:06:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:45 UTC)" (scheduled at 2020-03-31 09:06:15.064514+00:00) gcworker stdout | 2020-03-31 09:06:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:45.064514+00:00 (in 29.999076 seconds) 2020-03-31 09:06:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:45 UTC)" (scheduled at 2020-03-31 09:06:15.064514+00:00) 2020-03-31 09:06:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435975, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:06:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584435975, 1209600, 500, 1, 0]) 2020-03-31 09:06:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:06:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:06:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:06:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:45 UTC)" executed successfully 2020-03-31 09:06:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:06:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:26.924497+00:00 (in 9.999195 seconds) 2020-03-31 09:06:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:26 UTC)" (scheduled at 2020-03-31 09:06:16.924497+00:00) 2020-03-31 09:06:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:06:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:26.924497+00:00 (in 9.999195 seconds) 2020-03-31 09:06:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:26 UTC)" (scheduled at 2020-03-31 09:06:16.924497+00:00) 2020-03-31 09:06:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 16, 926200), True, datetime.datetime(2020, 3, 31, 9, 6, 16, 926200), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:06:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 16, 926200), True, datetime.datetime(2020, 3, 31, 9, 6, 16, 926200), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:06:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:06:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:06:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:26 UTC)" executed successfully 2020-03-31 09:06:18,272 [518] [DEBUG] [app] Starting request: urn:request:2c51ba5d-9399-4031-bdc4-798e0e6f824c (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:18,272 [518] [DEBUG] [app] Starting request: urn:request:2c51ba5d-9399-4031-bdc4-798e0e6f824c (/health/instance) 2020-03-31 09:06:18,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:18,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:18,280 [538] [DEBUG] [app] Starting request: urn:request:65f4707e-a6a6-4219-bb23-99028a53ae54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:18,280 [538] [DEBUG] [app] Starting request: urn:request:65f4707e-a6a6-4219-bb23-99028a53ae54 (/v1/_internal_ping) 2020-03-31 09:06:18,281 [538] [DEBUG] [app] Ending request: urn:request:65f4707e-a6a6-4219-bb23-99028a53ae54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:18,281 [538] [DEBUG] [app] Ending request: urn:request:65f4707e-a6a6-4219-bb23-99028a53ae54 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:06:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:18,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:18,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:18,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:18,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:18,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:18,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:18,310 [518] [DEBUG] [app] Starting request: urn:request:f31f5108-dd9e-4b4a-8566-2777a0b9638c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:18,310 [518] [DEBUG] [app] Starting request: urn:request:f31f5108-dd9e-4b4a-8566-2777a0b9638c (/_internal_ping) 2020-03-31 09:06:18,311 [518] [DEBUG] [app] Ending request: urn:request:f31f5108-dd9e-4b4a-8566-2777a0b9638c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:18,311 [518] [DEBUG] [app] Ending request: urn:request:f31f5108-dd9e-4b4a-8566-2777a0b9638c (/_internal_ping) 2020-03-31 09:06:18,311 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:18,311 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:06:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:06:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:06:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:18,323 [543] [DEBUG] [app] Starting request: urn:request:2d4260d7-5c7e-44c0-9981-e3b43a765f05 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:18,323 [543] [DEBUG] [app] Starting request: urn:request:2d4260d7-5c7e-44c0-9981-e3b43a765f05 (/c1/_internal_ping) 2020-03-31 09:06:18,324 [543] [DEBUG] [app] Ending request: urn:request:2d4260d7-5c7e-44c0-9981-e3b43a765f05 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:18,324 [543] [DEBUG] [app] Ending request: urn:request:2d4260d7-5c7e-44c0-9981-e3b43a765f05 (/c1/_internal_ping) 2020-03-31 09:06:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:18,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:18,327 [518] [DEBUG] [app] Ending request: urn:request:2c51ba5d-9399-4031-bdc4-798e0e6f824c (/health/instance) 2020-03-31 09:06:18,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:06:18,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:06:18,327 [518] [DEBUG] [app] Ending request: urn:request:2c51ba5d-9399-4031-bdc4-798e0e6f824c (/health/instance) 2020-03-31 09:06:18,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:06:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:06:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:31.440707+00:00 (in 9.998718 seconds) 2020-03-31 09:06:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:31 UTC)" (scheduled at 2020-03-31 09:06:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:06:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:31.440707+00:00 (in 9.998718 seconds) 2020-03-31 09:06:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:31 UTC)" (scheduled at 2020-03-31 09:06:21.440707+00:00) 2020-03-31 09:06:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:06:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:06:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:06:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:06:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:06:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:31 UTC)" executed successfully 2020-03-31 09:06:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:06:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:52.113221+00:00 (in 29.997742 seconds) 2020-03-31 09:06:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:52 UTC)" (scheduled at 2020-03-31 09:06:22.113221+00:00) securityworker stdout | 2020-03-31 09:06:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:52.113221+00:00 (in 29.997742 seconds) 2020-03-31 09:06:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:52 UTC)" (scheduled at 2020-03-31 09:06:22.113221+00:00) 2020-03-31 09:06:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:06:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:06:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:06:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:06:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:06:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:06:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:06:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:06:52 UTC)" executed successfully 2020-03-31 09:06:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:06:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:34.323352+00:00 (in 9.998881 seconds) 2020-03-31 09:06:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:34 UTC)" (scheduled at 2020-03-31 09:06:24.323352+00:00) 2020-03-31 09:06:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:06:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:34.323352+00:00 (in 9.998881 seconds) 2020-03-31 09:06:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:34 UTC)" (scheduled at 2020-03-31 09:06:24.323352+00:00) 2020-03-31 09:06:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 24, 325267), True, datetime.datetime(2020, 3, 31, 9, 6, 24, 325267), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:06:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 24, 325267), True, datetime.datetime(2020, 3, 31, 9, 6, 24, 325267), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:06:24,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:06:24,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:06:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:34 UTC)" executed successfully 2020-03-31 09:06:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:06:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:33.330112+00:00 (in 8.918055 seconds) 2020-03-31 09:06:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:24 UTC)" (scheduled at 2020-03-31 09:06:24.410927+00:00) 2020-03-31 09:06:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:06:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:33.330112+00:00 (in 8.918055 seconds) 2020-03-31 09:06:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:24 UTC)" (scheduled at 2020-03-31 09:06:24.410927+00:00) 2020-03-31 09:06:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:24 UTC)" executed successfully 2020-03-31 09:06:24,446 [518] [DEBUG] [app] Starting request: urn:request:ab2a29db-f8c2-4aa3-81cd-ef5f26226318 (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:24,446 [518] [DEBUG] [app] Starting request: urn:request:ab2a29db-f8c2-4aa3-81cd-ef5f26226318 (/health/instance) 2020-03-31 09:06:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:24,455 [538] [DEBUG] [app] Starting request: urn:request:2a3a2f04-b321-4a45-8527-4c4328a1577e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:24,455 [538] [DEBUG] [app] Starting request: urn:request:2a3a2f04-b321-4a45-8527-4c4328a1577e (/v1/_internal_ping) 2020-03-31 09:06:24,456 [538] [DEBUG] [app] Ending request: urn:request:2a3a2f04-b321-4a45-8527-4c4328a1577e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:24,456 [538] [DEBUG] [app] Ending request: urn:request:2a3a2f04-b321-4a45-8527-4c4328a1577e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:06:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:24,459 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:06:24,459 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:24,462 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:24,462 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:24,463 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:24,463 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:24,464 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:24,464 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:24,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:24,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:24,487 [518] [DEBUG] [app] Starting request: urn:request:9f00d318-7f5c-4763-990d-23eafc645625 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:24,487 [518] [DEBUG] [app] Starting request: urn:request:9f00d318-7f5c-4763-990d-23eafc645625 (/_internal_ping) 2020-03-31 09:06:24,488 [518] [DEBUG] [app] Ending request: urn:request:9f00d318-7f5c-4763-990d-23eafc645625 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:24,488 [518] [DEBUG] [app] Ending request: urn:request:9f00d318-7f5c-4763-990d-23eafc645625 (/_internal_ping) 2020-03-31 09:06:24,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:24,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:06:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:06:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:06:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:06:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:06:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:24,501 [543] [DEBUG] [app] Starting request: urn:request:fa9962af-4f6c-4df4-a761-83a9ea0e2fc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:24,501 [543] [DEBUG] [app] Starting request: urn:request:fa9962af-4f6c-4df4-a761-83a9ea0e2fc2 (/c1/_internal_ping) 2020-03-31 09:06:24,502 [543] [DEBUG] [app] Ending request: urn:request:fa9962af-4f6c-4df4-a761-83a9ea0e2fc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:24,502 [543] [DEBUG] [app] Ending request: urn:request:fa9962af-4f6c-4df4-a761-83a9ea0e2fc2 (/c1/_internal_ping) 2020-03-31 09:06:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:24,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:24,505 [518] [DEBUG] [app] Ending request: urn:request:ab2a29db-f8c2-4aa3-81cd-ef5f26226318 (/health/instance) 2020-03-31 09:06:24,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:06:24,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:06:24,505 [518] [DEBUG] [app] Ending request: urn:request:ab2a29db-f8c2-4aa3-81cd-ef5f26226318 (/health/instance) 2020-03-31 09:06:24,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:06:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:06:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:36.924497+00:00 (in 9.999235 seconds) storagereplication stdout | 2020-03-31 09:06:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:36.924497+00:00 (in 9.999235 seconds) 2020-03-31 09:06:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:36 UTC)" (scheduled at 2020-03-31 09:06:26.924497+00:00) 2020-03-31 09:06:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:06:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:36 UTC)" (scheduled at 2020-03-31 09:06:26.924497+00:00) 2020-03-31 09:06:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 26, 927536), True, datetime.datetime(2020, 3, 31, 9, 6, 26, 927536), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:06:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 26, 927536), True, datetime.datetime(2020, 3, 31, 9, 6, 26, 927536), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:06:26,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:06:26,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:06:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:36 UTC)" executed successfully 2020-03-31 09:06:28,273 [518] [DEBUG] [app] Starting request: urn:request:8fd893c3-25e7-4c45-94df-a2c72d591380 (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:28,273 [518] [DEBUG] [app] Starting request: urn:request:8fd893c3-25e7-4c45-94df-a2c72d591380 (/health/instance) 2020-03-31 09:06:28,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:28,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:28,281 [538] [DEBUG] [app] Starting request: urn:request:4eb41800-69e0-41dc-89bd-eae92c64aac7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:28,281 [538] [DEBUG] [app] Starting request: urn:request:4eb41800-69e0-41dc-89bd-eae92c64aac7 (/v1/_internal_ping) 2020-03-31 09:06:28,283 [538] [DEBUG] [app] Ending request: urn:request:4eb41800-69e0-41dc-89bd-eae92c64aac7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:06:28,283 [538] [DEBUG] [app] Ending request: urn:request:4eb41800-69e0-41dc-89bd-eae92c64aac7 (/v1/_internal_ping) 2020-03-31 09:06:28,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:28,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:06:28,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:28,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:06:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:28,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:28,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:28,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:28,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:28,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:28,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:28,319 [518] [DEBUG] [app] Starting request: urn:request:21da7bf0-a693-49c9-a996-90209515043c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:28,319 [518] [DEBUG] [app] Starting request: urn:request:21da7bf0-a693-49c9-a996-90209515043c (/_internal_ping) 2020-03-31 09:06:28,320 [518] [DEBUG] [app] Ending request: urn:request:21da7bf0-a693-49c9-a996-90209515043c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:28,320 [518] [DEBUG] [app] Ending request: urn:request:21da7bf0-a693-49c9-a996-90209515043c (/_internal_ping) 2020-03-31 09:06:28,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:06:28,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:06:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:06:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:06:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:06:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:06:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:28,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:28,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:28,336 [543] [DEBUG] [app] Starting request: urn:request:520ef753-e8a1-4301-a8ea-934f68500200 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:28,336 [543] [DEBUG] [app] Starting request: urn:request:520ef753-e8a1-4301-a8ea-934f68500200 (/c1/_internal_ping) 2020-03-31 09:06:28,337 [543] [DEBUG] [app] Ending request: urn:request:520ef753-e8a1-4301-a8ea-934f68500200 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:28,337 [543] [DEBUG] [app] Ending request: urn:request:520ef753-e8a1-4301-a8ea-934f68500200 (/c1/_internal_ping) 2020-03-31 09:06:28,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:28,338 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:28,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:06:28,338 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:28,341 [518] [DEBUG] [app] Ending request: urn:request:8fd893c3-25e7-4c45-94df-a2c72d591380 (/health/instance) 2020-03-31 09:06:28,341 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:28,341 [518] [DEBUG] [app] Ending request: urn:request:8fd893c3-25e7-4c45-94df-a2c72d591380 (/health/instance) 2020-03-31 09:06:28,341 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:06:28,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:06:28,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:06:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:06:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:00.337699+00:00 (in 29.998995 seconds) 2020-03-31 09:06:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:00 UTC)" (scheduled at 2020-03-31 09:06:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:06:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:00.337699+00:00 (in 29.998995 seconds) 2020-03-31 09:06:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:00 UTC)" (scheduled at 2020-03-31 09:06:30.337699+00:00) 2020-03-31 09:06:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 6, 30, 339448), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:06:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 6, 30, 339448), False, 50, 1, 0]) 2020-03-31 09:06:30,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:06:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:06:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:06:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:00 UTC)" executed successfully 2020-03-31 09:06:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:06:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:41.440707+00:00 (in 9.998795 seconds) 2020-03-31 09:06:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:41 UTC)" (scheduled at 2020-03-31 09:06:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:06:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:41.440707+00:00 (in 9.998795 seconds) 2020-03-31 09:06:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:41 UTC)" (scheduled at 2020-03-31 09:06:31.440707+00:00) 2020-03-31 09:06:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:06:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:06:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:06:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:06:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:06:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:41 UTC)" executed successfully 2020-03-31 09:06:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:03 UTC)" (scheduled at 2020-03-31 09:06:33.330112+00:00) 2020-03-31 09:06:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 33, 331600), True, datetime.datetime(2020, 3, 31, 9, 6, 33, 331600), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:06:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:33,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:03.330112+00:00 (in 29.999230 seconds) security_notification_worker stdout | 2020-03-31 09:06:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:03 UTC)" (scheduled at 2020-03-31 09:06:33.330112+00:00) 2020-03-31 09:06:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:06:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 33, 331600), True, datetime.datetime(2020, 3, 31, 9, 6, 33, 331600), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:06:33,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:03.330112+00:00 (in 29.999230 seconds) 2020-03-31 09:06:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:06:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:06:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:03 UTC)" executed successfully 2020-03-31 09:06:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:06:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:43.539999+00:00 (in 10.172335 seconds) 2020-03-31 09:06:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:33 UTC)" (scheduled at 2020-03-31 09:06:33.366791+00:00) 2020-03-31 09:06:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:06:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:43.539999+00:00 (in 10.172335 seconds) 2020-03-31 09:06:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:33 UTC)" (scheduled at 2020-03-31 09:06:33.366791+00:00) 2020-03-31 09:06:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 33, 368395), True, datetime.datetime(2020, 3, 31, 9, 6, 33, 368395), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:06:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 33, 368395), True, datetime.datetime(2020, 3, 31, 9, 6, 33, 368395), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:06:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:06:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:06:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:33 UTC)" executed successfully 2020-03-31 09:06:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:06:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:44.323352+00:00 (in 9.999211 seconds) 2020-03-31 09:06:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:44 UTC)" (scheduled at 2020-03-31 09:06:34.323352+00:00) 2020-03-31 09:06:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:06:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:44.323352+00:00 (in 9.999211 seconds) 2020-03-31 09:06:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:44 UTC)" (scheduled at 2020-03-31 09:06:34.323352+00:00) 2020-03-31 09:06:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 34, 325292), True, datetime.datetime(2020, 3, 31, 9, 6, 34, 325292), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:06:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 34, 325292), True, datetime.datetime(2020, 3, 31, 9, 6, 34, 325292), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:06:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:06:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:06:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:44 UTC)" executed successfully 2020-03-31 09:06:34,445 [526] [DEBUG] [app] Starting request: urn:request:270706dd-b14a-439b-b7c3-09968b784c2c (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:34,445 [526] [DEBUG] [app] Starting request: urn:request:270706dd-b14a-439b-b7c3-09968b784c2c (/health/instance) 2020-03-31 09:06:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:34,453 [544] [DEBUG] [app] Starting request: urn:request:27e2a0c1-e648-4eab-a47f-2a0d75e5883c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:34,453 [544] [DEBUG] [app] Starting request: urn:request:27e2a0c1-e648-4eab-a47f-2a0d75e5883c (/v1/_internal_ping) 2020-03-31 09:06:34,454 [544] [DEBUG] [app] Ending request: urn:request:27e2a0c1-e648-4eab-a47f-2a0d75e5883c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:34,454 [544] [DEBUG] [app] Ending request: urn:request:27e2a0c1-e648-4eab-a47f-2a0d75e5883c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:34,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:06:34,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:34,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:34,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:34,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:34,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:34,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:34,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:34,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:34,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:34,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:34,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:34,483 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:34,483 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:34,485 [518] [DEBUG] [app] Starting request: urn:request:97fce61c-9743-49d3-97d2-8c14d4ecde5a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:34,485 [518] [DEBUG] [app] Starting request: urn:request:97fce61c-9743-49d3-97d2-8c14d4ecde5a (/_internal_ping) 2020-03-31 09:06:34,486 [518] [DEBUG] [app] Ending request: urn:request:97fce61c-9743-49d3-97d2-8c14d4ecde5a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:34,486 [518] [DEBUG] [app] Ending request: urn:request:97fce61c-9743-49d3-97d2-8c14d4ecde5a (/_internal_ping) 2020-03-31 09:06:34,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:34,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:34,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:34,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:34,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:34,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:06:34,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:34,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:34,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:34,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:06:34,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:06:34,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:34,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:34,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:34,499 [536] [DEBUG] [app] Starting request: urn:request:a122e8e8-c8c1-4ce1-be63-6fb8ccb7d415 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:34,499 [536] [DEBUG] [app] Starting request: urn:request:a122e8e8-c8c1-4ce1-be63-6fb8ccb7d415 (/c1/_internal_ping) 2020-03-31 09:06:34,500 [536] [DEBUG] [app] Ending request: urn:request:a122e8e8-c8c1-4ce1-be63-6fb8ccb7d415 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:34,500 [536] [DEBUG] [app] Ending request: urn:request:a122e8e8-c8c1-4ce1-be63-6fb8ccb7d415 (/c1/_internal_ping) 2020-03-31 09:06:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:34,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:06:34,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:34,502 [526] [DEBUG] [app] Ending request: urn:request:270706dd-b14a-439b-b7c3-09968b784c2c (/health/instance) 2020-03-31 09:06:34,502 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:34,502 [526] [DEBUG] [app] Ending request: urn:request:270706dd-b14a-439b-b7c3-09968b784c2c (/health/instance) 2020-03-31 09:06:34,502 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:34,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:06:34,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:06:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:06:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:38.010934+00:00 (in 1.085724 seconds) 2020-03-31 09:06:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:46 UTC)" (scheduled at 2020-03-31 09:06:36.924497+00:00) 2020-03-31 09:06:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:06:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:38.010934+00:00 (in 1.085724 seconds) 2020-03-31 09:06:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:46 UTC)" (scheduled at 2020-03-31 09:06:36.924497+00:00) 2020-03-31 09:06:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 36, 925956), True, datetime.datetime(2020, 3, 31, 9, 6, 36, 925956), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:06:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 36, 925956), True, datetime.datetime(2020, 3, 31, 9, 6, 36, 925956), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:06:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:06:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:06:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:46 UTC)" executed successfully 2020-03-31 09:06:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:06:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:46.924497+00:00 (in 8.912849 seconds) 2020-03-31 09:06:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:38 UTC)" (scheduled at 2020-03-31 09:06:38.010934+00:00) 2020-03-31 09:06:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:06:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:46.924497+00:00 (in 8.912849 seconds) 2020-03-31 09:06:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:38 UTC)" (scheduled at 2020-03-31 09:06:38.010934+00:00) 2020-03-31 09:06:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:38 UTC)" executed successfully 2020-03-31 09:06:38,272 [527] [DEBUG] [app] Starting request: urn:request:c1ee0244-43b9-402b-904d-07a126ba4186 (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:38,272 [527] [DEBUG] [app] Starting request: urn:request:c1ee0244-43b9-402b-904d-07a126ba4186 (/health/instance) 2020-03-31 09:06:38,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:38,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:38,280 [556] [DEBUG] [app] Starting request: urn:request:c2c59e83-bd85-421f-9466-bed90d750b7a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:38,280 [556] [DEBUG] [app] Starting request: urn:request:c2c59e83-bd85-421f-9466-bed90d750b7a (/v1/_internal_ping) 2020-03-31 09:06:38,281 [556] [DEBUG] [app] Ending request: urn:request:c2c59e83-bd85-421f-9466-bed90d750b7a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:38,281 [556] [DEBUG] [app] Ending request: urn:request:c2c59e83-bd85-421f-9466-bed90d750b7a (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:38,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:06:38,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:38,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:38,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:38,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:38,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:38,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:38,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:38,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:38,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:38,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:38,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:38,310 [527] [DEBUG] [app] Starting request: urn:request:241aaede-b092-412f-87e2-0776afeb85b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:38,310 [527] [DEBUG] [app] Starting request: urn:request:241aaede-b092-412f-87e2-0776afeb85b2 (/_internal_ping) 2020-03-31 09:06:38,311 [527] [DEBUG] [app] Ending request: urn:request:241aaede-b092-412f-87e2-0776afeb85b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:38,311 [527] [DEBUG] [app] Ending request: urn:request:241aaede-b092-412f-87e2-0776afeb85b2 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:06:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:38,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:38,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:06:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:06:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.002 153 0.002) gunicorn-web stdout | 2020-03-31 09:06:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.002 153 0.002) 2020-03-31 09:06:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:38,326 [543] [DEBUG] [app] Starting request: urn:request:773eb133-14c1-47c1-864d-1153a863ee5a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:38,326 [543] [DEBUG] [app] Starting request: urn:request:773eb133-14c1-47c1-864d-1153a863ee5a (/c1/_internal_ping) 2020-03-31 09:06:38,327 [543] [DEBUG] [app] Ending request: urn:request:773eb133-14c1-47c1-864d-1153a863ee5a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:38,327 [543] [DEBUG] [app] Ending request: urn:request:773eb133-14c1-47c1-864d-1153a863ee5a (/c1/_internal_ping) 2020-03-31 09:06:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:38,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:38,330 [527] [DEBUG] [app] Ending request: urn:request:c1ee0244-43b9-402b-904d-07a126ba4186 (/health/instance) 2020-03-31 09:06:38,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:06:38,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:06:38,330 [527] [DEBUG] [app] Ending request: urn:request:c1ee0244-43b9-402b-904d-07a126ba4186 (/health/instance) 2020-03-31 09:06:38,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:06:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:06:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:51.440707+00:00 (in 9.999058 seconds) 2020-03-31 09:06:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:51 UTC)" (scheduled at 2020-03-31 09:06:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:06:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:51.440707+00:00 (in 9.999058 seconds) 2020-03-31 09:06:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:51 UTC)" (scheduled at 2020-03-31 09:06:41.440707+00:00) 2020-03-31 09:06:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:06:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:06:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:06:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:06:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:06:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:51 UTC)" executed successfully 2020-03-31 09:06:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:06:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:33.366791+00:00 (in 49.826044 seconds) 2020-03-31 09:06:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:43 UTC)" (scheduled at 2020-03-31 09:06:43.539999+00:00) 2020-03-31 09:06:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:06:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:33.366791+00:00 (in 49.826044 seconds) 2020-03-31 09:06:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:43 UTC)" (scheduled at 2020-03-31 09:06:43.539999+00:00) 2020-03-31 09:06:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:43 UTC)" executed successfully 2020-03-31 09:06:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:06:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:49.498966+00:00 (in 5.174221 seconds) 2020-03-31 09:06:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:54 UTC)" (scheduled at 2020-03-31 09:06:44.323352+00:00) 2020-03-31 09:06:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:06:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:49.498966+00:00 (in 5.174221 seconds) 2020-03-31 09:06:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:54 UTC)" (scheduled at 2020-03-31 09:06:44.323352+00:00) 2020-03-31 09:06:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 44, 325709), True, datetime.datetime(2020, 3, 31, 9, 6, 44, 325709), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:06:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 44, 325709), True, datetime.datetime(2020, 3, 31, 9, 6, 44, 325709), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:06:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:06:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:06:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:54 UTC)" executed successfully 2020-03-31 09:06:44,446 [518] [DEBUG] [app] Starting request: urn:request:56926f59-7bff-40e3-a208-25b339ba763f (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:44,446 [518] [DEBUG] [app] Starting request: urn:request:56926f59-7bff-40e3-a208-25b339ba763f (/health/instance) 2020-03-31 09:06:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:44,454 [551] [DEBUG] [app] Starting request: urn:request:187ef890-4518-4676-98bc-70f0dfb6a3da (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:44,454 [551] [DEBUG] [app] Starting request: urn:request:187ef890-4518-4676-98bc-70f0dfb6a3da (/v1/_internal_ping) 2020-03-31 09:06:44,455 [551] [DEBUG] [app] Ending request: urn:request:187ef890-4518-4676-98bc-70f0dfb6a3da (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:44,455 [551] [DEBUG] [app] Ending request: urn:request:187ef890-4518-4676-98bc-70f0dfb6a3da (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:06:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:44,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:44,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:44,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:44,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:44,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:44,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:44,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:44,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:44,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:44,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:44,484 [518] [DEBUG] [app] Starting request: urn:request:1db13d3d-0bd4-4369-86a9-69f27014b909 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:44,484 [518] [DEBUG] [app] Starting request: urn:request:1db13d3d-0bd4-4369-86a9-69f27014b909 (/_internal_ping) 2020-03-31 09:06:44,485 [518] [DEBUG] [app] Ending request: urn:request:1db13d3d-0bd4-4369-86a9-69f27014b909 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:44,485 [518] [DEBUG] [app] Ending request: urn:request:1db13d3d-0bd4-4369-86a9-69f27014b909 (/_internal_ping) 2020-03-31 09:06:44,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:44,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:44,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:44,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:06:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:06:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:06:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:06:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:44,499 [536] [DEBUG] [app] Starting request: urn:request:82996711-1778-4beb-8fca-e03aa5a8cb56 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:44,499 [536] [DEBUG] [app] Starting request: urn:request:82996711-1778-4beb-8fca-e03aa5a8cb56 (/c1/_internal_ping) 2020-03-31 09:06:44,500 [536] [DEBUG] [app] Ending request: urn:request:82996711-1778-4beb-8fca-e03aa5a8cb56 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:44,500 [536] [DEBUG] [app] Ending request: urn:request:82996711-1778-4beb-8fca-e03aa5a8cb56 (/c1/_internal_ping) 2020-03-31 09:06:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:44,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:44,504 [518] [DEBUG] [app] Ending request: urn:request:56926f59-7bff-40e3-a208-25b339ba763f (/health/instance) 2020-03-31 09:06:44,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:06:44,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:44,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:06:44,504 [518] [DEBUG] [app] Ending request: urn:request:56926f59-7bff-40e3-a208-25b339ba763f (/health/instance) 2020-03-31 09:06:44,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:44,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:06:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:06:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:15.064514+00:00 (in 29.999180 seconds) 2020-03-31 09:06:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:15 UTC)" (scheduled at 2020-03-31 09:06:45.064514+00:00) gcworker stdout | 2020-03-31 09:06:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:15.064514+00:00 (in 29.999180 seconds) 2020-03-31 09:06:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:15 UTC)" (scheduled at 2020-03-31 09:06:45.064514+00:00) 2020-03-31 09:06:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436005, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:06:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436005, 1209600, 500, 1, 0]) 2020-03-31 09:06:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:06:45,083 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:06:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:06:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:06:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:15 UTC)" executed successfully 2020-03-31 09:06:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:06:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:56.924497+00:00 (in 9.998881 seconds) 2020-03-31 09:06:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:56 UTC)" (scheduled at 2020-03-31 09:06:46.924497+00:00) 2020-03-31 09:06:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:06:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:56.924497+00:00 (in 9.998881 seconds) 2020-03-31 09:06:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:56 UTC)" (scheduled at 2020-03-31 09:06:46.924497+00:00) 2020-03-31 09:06:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 46, 926361), True, datetime.datetime(2020, 3, 31, 9, 6, 46, 926361), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:06:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 46, 926361), True, datetime.datetime(2020, 3, 31, 9, 6, 46, 926361), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:06:46,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:06:46,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:46,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:46,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:06:46,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:46,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:56 UTC)" executed successfully 2020-03-31 09:06:48,272 [518] [DEBUG] [app] Starting request: urn:request:0e737b04-da61-4f49-bcda-8fb9feec91c3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:48,272 [518] [DEBUG] [app] Starting request: urn:request:0e737b04-da61-4f49-bcda-8fb9feec91c3 (/health/instance) 2020-03-31 09:06:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:48,278 [538] [DEBUG] [app] Starting request: urn:request:f2b76ded-1408-4d77-b54d-d7fd83a2641e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:48,278 [538] [DEBUG] [app] Starting request: urn:request:f2b76ded-1408-4d77-b54d-d7fd83a2641e (/v1/_internal_ping) 2020-03-31 09:06:48,279 [538] [DEBUG] [app] Ending request: urn:request:f2b76ded-1408-4d77-b54d-d7fd83a2641e (/v1/_internal_ping) 2020-03-31 09:06:48,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:48,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:06:48,279 [538] [DEBUG] [app] Ending request: urn:request:f2b76ded-1408-4d77-b54d-d7fd83a2641e (/v1/_internal_ping) 2020-03-31 09:06:48,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:48,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:48,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:06:48,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:48,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:48,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:06:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:06:48,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:48,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:48,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:48,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:48,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:48,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:48,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:48,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:48,310 [525] [DEBUG] [app] Starting request: urn:request:0b2381cd-4375-460a-a39f-fb305944bdd2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:48,310 [525] [DEBUG] [app] Starting request: urn:request:0b2381cd-4375-460a-a39f-fb305944bdd2 (/_internal_ping) 2020-03-31 09:06:48,311 [525] [DEBUG] [app] Ending request: urn:request:0b2381cd-4375-460a-a39f-fb305944bdd2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:48,311 [525] [DEBUG] [app] Ending request: urn:request:0b2381cd-4375-460a-a39f-fb305944bdd2 (/_internal_ping) 2020-03-31 09:06:48,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:48,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:06:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:06:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:06:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:06:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:06:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:48,325 [543] [DEBUG] [app] Starting request: urn:request:ca256c45-ec4d-4eef-9d41-df756205b37d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:48,325 [543] [DEBUG] [app] Starting request: urn:request:ca256c45-ec4d-4eef-9d41-df756205b37d (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:48,326 [543] [DEBUG] [app] Ending request: urn:request:ca256c45-ec4d-4eef-9d41-df756205b37d (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:06:48,326 [543] [DEBUG] [app] Ending request: urn:request:ca256c45-ec4d-4eef-9d41-df756205b37d (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:48,328 [518] [DEBUG] [app] Ending request: urn:request:0e737b04-da61-4f49-bcda-8fb9feec91c3 (/health/instance) 2020-03-31 09:06:48,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:06:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:06:48,328 [518] [DEBUG] [app] Ending request: urn:request:0e737b04-da61-4f49-bcda-8fb9feec91c3 (/health/instance) 2020-03-31 09:06:48,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:06:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:06:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:54.323352+00:00 (in 4.823319 seconds) 2020-03-31 09:06:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:49 UTC)" (scheduled at 2020-03-31 09:06:49.498966+00:00) 2020-03-31 09:06:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:06:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:54.323352+00:00 (in 4.823319 seconds) 2020-03-31 09:06:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:49 UTC)" (scheduled at 2020-03-31 09:06:49.498966+00:00) 2020-03-31 09:06:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:49 UTC)" executed successfully 2020-03-31 09:06:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:06:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:59.560326+00:00 (in 8.448756 seconds) 2020-03-31 09:06:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:51 UTC)" (scheduled at 2020-03-31 09:06:51.110610+00:00) 2020-03-31 09:06:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:06:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:06:59.560326+00:00 (in 8.448756 seconds) 2020-03-31 09:06:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:51 UTC)" (scheduled at 2020-03-31 09:06:51.110610+00:00) 2020-03-31 09:06:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 51, 112450), True, datetime.datetime(2020, 3, 31, 9, 6, 51, 112450), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:06:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 51, 112450), True, datetime.datetime(2020, 3, 31, 9, 6, 51, 112450), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:06:51,128 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:06:51,128 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:51,128 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:06:51,128 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:51 UTC)" executed successfully 2020-03-31 09:06:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:06:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:01.440707+00:00 (in 9.995945 seconds) 2020-03-31 09:06:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:01 UTC)" (scheduled at 2020-03-31 09:06:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:06:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:01.440707+00:00 (in 9.995945 seconds) 2020-03-31 09:06:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:01 UTC)" (scheduled at 2020-03-31 09:06:51.440707+00:00) 2020-03-31 09:06:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:06:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:06:51,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:06:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:06:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:06:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:01 UTC)" executed successfully 2020-03-31 09:06:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:06:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:22.113221+00:00 (in 29.998871 seconds) 2020-03-31 09:06:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:22 UTC)" (scheduled at 2020-03-31 09:06:52.113221+00:00) securityworker stdout | 2020-03-31 09:06:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:22.113221+00:00 (in 29.998871 seconds) 2020-03-31 09:06:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:22 UTC)" (scheduled at 2020-03-31 09:06:52.113221+00:00) 2020-03-31 09:06:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:06:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:06:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:06:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:06:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:06:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:06:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:06:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:22 UTC)" executed successfully 2020-03-31 09:06:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:06:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:04.323352+00:00 (in 9.999013 seconds) 2020-03-31 09:06:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:04 UTC)" (scheduled at 2020-03-31 09:06:54.323352+00:00) 2020-03-31 09:06:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:06:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:04.323352+00:00 (in 9.999013 seconds) 2020-03-31 09:06:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:04 UTC)" (scheduled at 2020-03-31 09:06:54.323352+00:00) 2020-03-31 09:06:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 54, 325241), True, datetime.datetime(2020, 3, 31, 9, 6, 54, 325241), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:06:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 54, 325241), True, datetime.datetime(2020, 3, 31, 9, 6, 54, 325241), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:06:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:06:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:06:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:04 UTC)" executed successfully 2020-03-31 09:06:54,446 [527] [DEBUG] [app] Starting request: urn:request:9b5a15ea-8dcd-4755-87e4-5ccf45283dfb (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:54,446 [527] [DEBUG] [app] Starting request: urn:request:9b5a15ea-8dcd-4755-87e4-5ccf45283dfb (/health/instance) 2020-03-31 09:06:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:54,453 [538] [DEBUG] [app] Starting request: urn:request:c26f3418-127e-4347-983a-6b91fbe36d40 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:54,453 [538] [DEBUG] [app] Starting request: urn:request:c26f3418-127e-4347-983a-6b91fbe36d40 (/v1/_internal_ping) 2020-03-31 09:06:54,454 [538] [DEBUG] [app] Ending request: urn:request:c26f3418-127e-4347-983a-6b91fbe36d40 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:54,454 [538] [DEBUG] [app] Ending request: urn:request:c26f3418-127e-4347-983a-6b91fbe36d40 (/v1/_internal_ping) 2020-03-31 09:06:54,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:06:54,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:06:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:06:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:06:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:54,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:54,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:54,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:54,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:54,488 [527] [DEBUG] [app] Starting request: urn:request:910f5353-6b37-4b6f-a340-3978c8824856 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:54,488 [527] [DEBUG] [app] Starting request: urn:request:910f5353-6b37-4b6f-a340-3978c8824856 (/_internal_ping) 2020-03-31 09:06:54,489 [527] [DEBUG] [app] Ending request: urn:request:910f5353-6b37-4b6f-a340-3978c8824856 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:54,489 [527] [DEBUG] [app] Ending request: urn:request:910f5353-6b37-4b6f-a340-3978c8824856 (/_internal_ping) 2020-03-31 09:06:54,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:06:54,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:06:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:06:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:06:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:06:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:06:54,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:54,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:54,503 [543] [DEBUG] [app] Starting request: urn:request:7c036cba-330a-4f03-95c9-dd13595d7b65 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:54,503 [543] [DEBUG] [app] Starting request: urn:request:7c036cba-330a-4f03-95c9-dd13595d7b65 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:54,504 [543] [DEBUG] [app] Ending request: urn:request:7c036cba-330a-4f03-95c9-dd13595d7b65 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:54,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:54,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:06:54,504 [543] [DEBUG] [app] Ending request: urn:request:7c036cba-330a-4f03-95c9-dd13595d7b65 (/c1/_internal_ping) 2020-03-31 09:06:54,505 [527] [DEBUG] [app] Ending request: urn:request:9b5a15ea-8dcd-4755-87e4-5ccf45283dfb (/health/instance) 2020-03-31 09:06:54,506 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:54,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:06:54,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:06:54,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:54,505 [527] [DEBUG] [app] Ending request: urn:request:9b5a15ea-8dcd-4755-87e4-5ccf45283dfb (/health/instance) 2020-03-31 09:06:54,506 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:54,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:06:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:06:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:56 UTC)" (scheduled at 2020-03-31 09:06:56.924497+00:00) 2020-03-31 09:06:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:06:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:06:56 UTC)" (scheduled at 2020-03-31 09:06:56.924497+00:00) 2020-03-31 09:06:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:06:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 56, 925958), True, datetime.datetime(2020, 3, 31, 9, 6, 56, 925958), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:06:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 6, 56, 925958), True, datetime.datetime(2020, 3, 31, 9, 6, 56, 925958), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:06:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:03.010318+00:00 (in 6.085142 seconds) storagereplication stdout | 2020-03-31 09:06:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:03.010318+00:00 (in 6.085142 seconds) 2020-03-31 09:06:56,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:06:56,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:06:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:06:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:06 UTC)" executed successfully 2020-03-31 09:06:58,273 [527] [DEBUG] [app] Starting request: urn:request:be77ab32-5ce7-414a-92c7-4eab8e073208 (/health/instance) gunicorn-web stdout | 2020-03-31 09:06:58,273 [527] [DEBUG] [app] Starting request: urn:request:be77ab32-5ce7-414a-92c7-4eab8e073208 (/health/instance) 2020-03-31 09:06:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:58,278 [556] [DEBUG] [app] Starting request: urn:request:b2f7a7c2-0d41-40e2-ba0b-6f751a9ce2ae (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:58,278 [556] [DEBUG] [app] Starting request: urn:request:b2f7a7c2-0d41-40e2-ba0b-6f751a9ce2ae (/v1/_internal_ping) 2020-03-31 09:06:58,279 [556] [DEBUG] [app] Ending request: urn:request:b2f7a7c2-0d41-40e2-ba0b-6f751a9ce2ae (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:06:58,279 [556] [DEBUG] [app] Ending request: urn:request:b2f7a7c2-0d41-40e2-ba0b-6f751a9ce2ae (/v1/_internal_ping) 2020-03-31 09:06:58,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:06:58,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:06:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:06:58,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:06:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:06:58,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:06:58,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:06:58,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:06:58,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:06:58,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:06:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:06:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:06:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:06:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:06:58,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:58,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:58,313 [527] [DEBUG] [app] Starting request: urn:request:a48d37b3-2d53-4c40-afe5-a2689f52ee2d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:58,313 [527] [DEBUG] [app] Starting request: urn:request:a48d37b3-2d53-4c40-afe5-a2689f52ee2d (/_internal_ping) 2020-03-31 09:06:58,314 [527] [DEBUG] [app] Ending request: urn:request:a48d37b3-2d53-4c40-afe5-a2689f52ee2d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:06:58,314 [527] [DEBUG] [app] Ending request: urn:request:a48d37b3-2d53-4c40-afe5-a2689f52ee2d (/_internal_ping) 2020-03-31 09:06:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:06:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:06:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:06:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:06:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:06:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:58,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:06:58,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:06:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:06:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:06:58,326 [536] [DEBUG] [app] Starting request: urn:request:a53022dc-0926-4b21-984b-5db8c7c16a98 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:06:58,326 [536] [DEBUG] [app] Starting request: urn:request:a53022dc-0926-4b21-984b-5db8c7c16a98 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:58,327 [536] [DEBUG] [app] Ending request: urn:request:a53022dc-0926-4b21-984b-5db8c7c16a98 (/c1/_internal_ping) 2020-03-31 09:06:58,327 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:06:58,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:06:58,327 [536] [DEBUG] [app] Ending request: urn:request:a53022dc-0926-4b21-984b-5db8c7c16a98 (/c1/_internal_ping) 2020-03-31 09:06:58,327 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:06:58,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:06:58,329 [527] [DEBUG] [app] Ending request: urn:request:be77ab32-5ce7-414a-92c7-4eab8e073208 (/health/instance) 2020-03-31 09:06:58,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:06:58,329 [527] [DEBUG] [app] Ending request: urn:request:be77ab32-5ce7-414a-92c7-4eab8e073208 (/health/instance) 2020-03-31 09:06:58,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:06:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:06:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:06:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:06:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:06:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:06:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:06:59,563 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:59 UTC)" (scheduled at 2020-03-31 09:06:59.560326+00:00) 2020-03-31 09:06:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:06:59,563 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:59 UTC)" (scheduled at 2020-03-31 09:06:59.560326+00:00) 2020-03-31 09:06:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:06:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:07:59 UTC)" executed successfully 2020-03-31 09:06:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:51.110610+00:00 (in 51.547701 seconds) namespacegcworker stdout | 2020-03-31 09:06:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:51.110610+00:00 (in 51.547701 seconds) 2020-03-31 09:07:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:07:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:30.337699+00:00 (in 29.999074 seconds) 2020-03-31 09:07:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:30 UTC)" (scheduled at 2020-03-31 09:07:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:07:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:30.337699+00:00 (in 29.999074 seconds) 2020-03-31 09:07:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:30 UTC)" (scheduled at 2020-03-31 09:07:00.337699+00:00) 2020-03-31 09:07:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 7, 0, 339473), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:07:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 7, 0, 339473), False, 50, 1, 0]) 2020-03-31 09:07:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:07:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:07:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:07:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:30 UTC)" executed successfully 2020-03-31 09:07:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:07:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:11.440707+00:00 (in 9.999065 seconds) 2020-03-31 09:07:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:11 UTC)" (scheduled at 2020-03-31 09:07:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:07:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:11.440707+00:00 (in 9.999065 seconds) 2020-03-31 09:07:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:11 UTC)" (scheduled at 2020-03-31 09:07:01.440707+00:00) 2020-03-31 09:07:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:07:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:07:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:07:01,459 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:07:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:07:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:07:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:11 UTC)" executed successfully 2020-03-31 09:07:03,014 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:03,014 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:03,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:06.924497+00:00 (in 3.909509 seconds) 2020-03-31 09:07:03,015 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:03 UTC)" (scheduled at 2020-03-31 09:07:03.010318+00:00) storagereplication stdout | 2020-03-31 09:07:03,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:06.924497+00:00 (in 3.909509 seconds) 2020-03-31 09:07:03,015 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:03 UTC)" (scheduled at 2020-03-31 09:07:03.010318+00:00) 2020-03-31 09:07:03,016 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:07:03,016 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), u'imagestoragereplication/%']) 2020-03-31 09:07:03,030 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), True, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:07:03,030 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), True, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), 0, u'imagestoragereplication/%']) 2020-03-31 09:07:03,035 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), True, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:07:03,035 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), True, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 7, 3, 15827), u'imagestoragereplication/%']) 2020-03-31 09:07:03,038 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:07:03,038 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:03,040 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:03,040 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:03 UTC)" executed successfully 2020-03-31 09:07:03,041 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:07:03,041 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:03,042 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:07:03,042 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:03,042 [191] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics storagereplication stdout | 2020-03-31 09:07:03,042 [191] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:07:03,044 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:07:03,044 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:03,045 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:07:03,045 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:03,045 [191] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics storagereplication stdout | 2020-03-31 09:07:03,045 [191] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:03,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:07:03,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:03,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:24.410927+00:00 (in 21.076976 seconds) 2020-03-31 09:07:03,334 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:33 UTC)" (scheduled at 2020-03-31 09:07:03.330112+00:00) 2020-03-31 09:07:03,334 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:07:03,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:24.410927+00:00 (in 21.076976 seconds) 2020-03-31 09:07:03,334 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:33 UTC)" (scheduled at 2020-03-31 09:07:03.330112+00:00) 2020-03-31 09:07:03,334 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:03,338 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 3, 334922), True, datetime.datetime(2020, 3, 31, 9, 7, 3, 334922), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:07:03,338 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 3, 334922), True, datetime.datetime(2020, 3, 31, 9, 7, 3, 334922), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:07:03,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:07:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:07:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:33 UTC)" executed successfully 2020-03-31 09:07:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:14.323352+00:00 (in 9.998842 seconds) 2020-03-31 09:07:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:14 UTC)" (scheduled at 2020-03-31 09:07:04.323352+00:00) 2020-03-31 09:07:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:07:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:14.323352+00:00 (in 9.998842 seconds) 2020-03-31 09:07:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:14 UTC)" (scheduled at 2020-03-31 09:07:04.323352+00:00) 2020-03-31 09:07:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 4, 325530), True, datetime.datetime(2020, 3, 31, 9, 7, 4, 325530), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:07:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 4, 325530), True, datetime.datetime(2020, 3, 31, 9, 7, 4, 325530), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:07:04,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:07:04,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:14 UTC)" executed successfully 2020-03-31 09:07:04,446 [518] [DEBUG] [app] Starting request: urn:request:65dbcc4d-9eb6-4678-beac-e0321d05e042 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:04,446 [518] [DEBUG] [app] Starting request: urn:request:65dbcc4d-9eb6-4678-beac-e0321d05e042 (/health/instance) 2020-03-31 09:07:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:04,454 [550] [DEBUG] [app] Starting request: urn:request:6e559779-5007-43d5-9bfe-10eef69d60c3 (/v1/_internal_ping) 2020-03-31 09:07:04,455 [550] [DEBUG] [app] Ending request: urn:request:6e559779-5007-43d5-9bfe-10eef69d60c3 (/v1/_internal_ping) 2020-03-31 09:07:04,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:07:04,454 [550] [DEBUG] [app] Starting request: urn:request:6e559779-5007-43d5-9bfe-10eef69d60c3 (/v1/_internal_ping) 2020-03-31 09:07:04,455 [550] [DEBUG] [app] Ending request: urn:request:6e559779-5007-43d5-9bfe-10eef69d60c3 (/v1/_internal_ping) 2020-03-31 09:07:04,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:04,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:04,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:04,461 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:04,461 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:04,462 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:04,462 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:04,463 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:04,463 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:04,486 [518] [DEBUG] [app] Starting request: urn:request:db2af2e8-0b10-4b80-99c4-0741f1441471 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:04,486 [518] [DEBUG] [app] Starting request: urn:request:db2af2e8-0b10-4b80-99c4-0741f1441471 (/_internal_ping) 2020-03-31 09:07:04,487 [518] [DEBUG] [app] Ending request: urn:request:db2af2e8-0b10-4b80-99c4-0741f1441471 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:04,487 [518] [DEBUG] [app] Ending request: urn:request:db2af2e8-0b10-4b80-99c4-0741f1441471 (/_internal_ping) 2020-03-31 09:07:04,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:07:04,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:07:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:07:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:07:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:07:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:04,501 [536] [DEBUG] [app] Starting request: urn:request:6fdc755c-f386-4659-bd44-e5aa11e1ff5f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:04,501 [536] [DEBUG] [app] Starting request: urn:request:6fdc755c-f386-4659-bd44-e5aa11e1ff5f (/c1/_internal_ping) 2020-03-31 09:07:04,501 [536] [DEBUG] [app] Ending request: urn:request:6fdc755c-f386-4659-bd44-e5aa11e1ff5f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:04,501 [536] [DEBUG] [app] Ending request: urn:request:6fdc755c-f386-4659-bd44-e5aa11e1ff5f (/c1/_internal_ping) 2020-03-31 09:07:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:04,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:04,504 [518] [DEBUG] [app] Ending request: urn:request:65dbcc4d-9eb6-4678-beac-e0321d05e042 (/health/instance) 2020-03-31 09:07:04,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:07:04,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:07:04,504 [518] [DEBUG] [app] Ending request: urn:request:65dbcc4d-9eb6-4678-beac-e0321d05e042 (/health/instance) 2020-03-31 09:07:04,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:07:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:16.924497+00:00 (in 9.998948 seconds) storagereplication stdout | 2020-03-31 09:07:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:16.924497+00:00 (in 9.998948 seconds) 2020-03-31 09:07:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:16 UTC)" (scheduled at 2020-03-31 09:07:06.924497+00:00) 2020-03-31 09:07:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:07:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:16 UTC)" (scheduled at 2020-03-31 09:07:06.924497+00:00) 2020-03-31 09:07:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 6, 926555), True, datetime.datetime(2020, 3, 31, 9, 7, 6, 926555), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:07:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 6, 926555), True, datetime.datetime(2020, 3, 31, 9, 7, 6, 926555), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:07:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:07:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:16 UTC)" executed successfully 2020-03-31 09:07:08,273 [518] [DEBUG] [app] Starting request: urn:request:1354f135-d7e3-400a-9941-cb6035404eca (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:08,273 [518] [DEBUG] [app] Starting request: urn:request:1354f135-d7e3-400a-9941-cb6035404eca (/health/instance) 2020-03-31 09:07:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:08,277 [555] [DEBUG] [app] Starting request: urn:request:5dfebc4c-2b60-40b7-85ea-717b04562255 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:08,277 [555] [DEBUG] [app] Starting request: urn:request:5dfebc4c-2b60-40b7-85ea-717b04562255 (/v1/_internal_ping) 2020-03-31 09:07:08,278 [555] [DEBUG] [app] Ending request: urn:request:5dfebc4c-2b60-40b7-85ea-717b04562255 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:08,278 [555] [DEBUG] [app] Ending request: urn:request:5dfebc4c-2b60-40b7-85ea-717b04562255 (/v1/_internal_ping) 2020-03-31 09:07:08,279 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:07:08,279 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:08,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:08,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:08,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:08,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:08,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:08,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:08,283 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:08,283 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:08,284 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:08,284 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:08,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:08,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:08,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:08,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:08,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:08,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:08,310 [518] [DEBUG] [app] Starting request: urn:request:e756ed3f-5692-412f-94dd-85766ce9a972 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:08,310 [518] [DEBUG] [app] Starting request: urn:request:e756ed3f-5692-412f-94dd-85766ce9a972 (/_internal_ping) 2020-03-31 09:07:08,310 [518] [DEBUG] [app] Ending request: urn:request:e756ed3f-5692-412f-94dd-85766ce9a972 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:08,310 [518] [DEBUG] [app] Ending request: urn:request:e756ed3f-5692-412f-94dd-85766ce9a972 (/_internal_ping) 2020-03-31 09:07:08,311 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:07:08,311 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:08,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:07:08,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:08,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:08,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:07:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:07:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:07:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:07:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:08,323 [536] [DEBUG] [app] Starting request: urn:request:7cdcdd32-a0a5-4c77-bca9-734a8300146f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:08,323 [536] [DEBUG] [app] Starting request: urn:request:7cdcdd32-a0a5-4c77-bca9-734a8300146f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:08,324 [536] [DEBUG] [app] Ending request: urn:request:7cdcdd32-a0a5-4c77-bca9-734a8300146f (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:08,325 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:07:08,324 [536] [DEBUG] [app] Ending request: urn:request:7cdcdd32-a0a5-4c77-bca9-734a8300146f (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:08,326 [518] [DEBUG] [app] Ending request: urn:request:1354f135-d7e3-400a-9941-cb6035404eca (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:07:08,325 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:08,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:08,326 [518] [DEBUG] [app] Ending request: urn:request:1354f135-d7e3-400a-9941-cb6035404eca (/health/instance) 2020-03-31 09:07:08,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:07:08,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:08,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:07:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:07:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:21.440707+00:00 (in 9.999148 seconds) 2020-03-31 09:07:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:21 UTC)" (scheduled at 2020-03-31 09:07:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:07:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:21.440707+00:00 (in 9.999148 seconds) 2020-03-31 09:07:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:21 UTC)" (scheduled at 2020-03-31 09:07:11.440707+00:00) 2020-03-31 09:07:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:07:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:07:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:07:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:07:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:07:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:21 UTC)" executed successfully 2020-03-31 09:07:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:24.323352+00:00 (in 9.999083 seconds) 2020-03-31 09:07:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:24 UTC)" (scheduled at 2020-03-31 09:07:14.323352+00:00) 2020-03-31 09:07:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:07:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:24.323352+00:00 (in 9.999083 seconds) 2020-03-31 09:07:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:24 UTC)" (scheduled at 2020-03-31 09:07:14.323352+00:00) 2020-03-31 09:07:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 14, 325349), True, datetime.datetime(2020, 3, 31, 9, 7, 14, 325349), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:07:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 14, 325349), True, datetime.datetime(2020, 3, 31, 9, 7, 14, 325349), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:07:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:07:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:24 UTC)" executed successfully 2020-03-31 09:07:14,447 [518] [DEBUG] [app] Starting request: urn:request:7cd8defd-11ee-4146-902b-6fdb96eb80f1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:14,447 [518] [DEBUG] [app] Starting request: urn:request:7cd8defd-11ee-4146-902b-6fdb96eb80f1 (/health/instance) 2020-03-31 09:07:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:14,457 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:14,457 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:14,458 [550] [DEBUG] [app] Starting request: urn:request:ea8f45a9-1ccc-4037-9114-7d501c92314d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:14,458 [550] [DEBUG] [app] Starting request: urn:request:ea8f45a9-1ccc-4037-9114-7d501c92314d (/v1/_internal_ping) 2020-03-31 09:07:14,459 [550] [DEBUG] [app] Ending request: urn:request:ea8f45a9-1ccc-4037-9114-7d501c92314d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:14,459 [550] [DEBUG] [app] Ending request: urn:request:ea8f45a9-1ccc-4037-9114-7d501c92314d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:14,460 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:14,460 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:07:14,460 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:14,460 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:14,461 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:14,461 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:14,462 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:14,462 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:14,464 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:14,464 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:14,465 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:14,465 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:14,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:14,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:14,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:14,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:14,495 [518] [DEBUG] [app] Starting request: urn:request:c3265018-f646-429c-b2c6-78158c497a7d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:14,495 [518] [DEBUG] [app] Starting request: urn:request:c3265018-f646-429c-b2c6-78158c497a7d (/_internal_ping) 2020-03-31 09:07:14,498 [518] [DEBUG] [app] Ending request: urn:request:c3265018-f646-429c-b2c6-78158c497a7d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 159 0.005) gunicorn-web stdout | 2020-03-31 09:07:14,498 [518] [DEBUG] [app] Ending request: urn:request:c3265018-f646-429c-b2c6-78158c497a7d (/_internal_ping) 2020-03-31 09:07:14,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 159 0.005) gunicorn-web stdout | 2020-03-31 09:07:14,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:07:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:07:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:07:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:07:14,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:14,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:14,511 [536] [DEBUG] [app] Starting request: urn:request:241e956b-45d8-48be-b226-6e5901582732 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:14,511 [536] [DEBUG] [app] Starting request: urn:request:241e956b-45d8-48be-b226-6e5901582732 (/c1/_internal_ping) 2020-03-31 09:07:14,512 [536] [DEBUG] [app] Ending request: urn:request:241e956b-45d8-48be-b226-6e5901582732 (/c1/_internal_ping) 2020-03-31 09:07:14,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:14,513 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:14,514 [518] [DEBUG] [app] Ending request: urn:request:7cd8defd-11ee-4146-902b-6fdb96eb80f1 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:07:14,512 [536] [DEBUG] [app] Ending request: urn:request:241e956b-45d8-48be-b226-6e5901582732 (/c1/_internal_ping) 2020-03-31 09:07:14,515 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:14,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:14,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:07:14,513 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:14,514 [518] [DEBUG] [app] Ending request: urn:request:7cd8defd-11ee-4146-902b-6fdb96eb80f1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:14,515 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:14,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:07:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:07:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:45.064514+00:00 (in 29.999239 seconds) 2020-03-31 09:07:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:45 UTC)" (scheduled at 2020-03-31 09:07:15.064514+00:00) gcworker stdout | 2020-03-31 09:07:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:45.064514+00:00 (in 29.999239 seconds) 2020-03-31 09:07:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:45 UTC)" (scheduled at 2020-03-31 09:07:15.064514+00:00) 2020-03-31 09:07:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436035, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:07:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436035, 1209600, 500, 1, 0]) 2020-03-31 09:07:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:07:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:07:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:07:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:45 UTC)" executed successfully 2020-03-31 09:07:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:26.924497+00:00 (in 9.999169 seconds) 2020-03-31 09:07:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:26 UTC)" (scheduled at 2020-03-31 09:07:16.924497+00:00) 2020-03-31 09:07:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:07:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:26.924497+00:00 (in 9.999169 seconds) 2020-03-31 09:07:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:26 UTC)" (scheduled at 2020-03-31 09:07:16.924497+00:00) 2020-03-31 09:07:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 16, 926047), True, datetime.datetime(2020, 3, 31, 9, 7, 16, 926047), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:07:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 16, 926047), True, datetime.datetime(2020, 3, 31, 9, 7, 16, 926047), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:07:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:07:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:26 UTC)" executed successfully 2020-03-31 09:07:18,273 [518] [DEBUG] [app] Starting request: urn:request:1901d3f3-0494-4a51-8dbe-a5945cd95868 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:18,273 [518] [DEBUG] [app] Starting request: urn:request:1901d3f3-0494-4a51-8dbe-a5945cd95868 (/health/instance) 2020-03-31 09:07:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:18,278 [548] [DEBUG] [app] Starting request: urn:request:e13e580f-a196-40c2-9495-17ef690c0b41 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:18,278 [548] [DEBUG] [app] Starting request: urn:request:e13e580f-a196-40c2-9495-17ef690c0b41 (/v1/_internal_ping) 2020-03-31 09:07:18,278 [548] [DEBUG] [app] Ending request: urn:request:e13e580f-a196-40c2-9495-17ef690c0b41 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:18,278 [548] [DEBUG] [app] Ending request: urn:request:e13e580f-a196-40c2-9495-17ef690c0b41 (/v1/_internal_ping) 2020-03-31 09:07:18,279 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:07:18,279 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:18,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:18,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:18,283 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:18,283 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:18,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:18,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:18,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:18,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:18,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:18,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:18,310 [518] [DEBUG] [app] Starting request: urn:request:523160a7-0af9-4ab6-bb35-2f319bab1580 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:18,310 [518] [DEBUG] [app] Starting request: urn:request:523160a7-0af9-4ab6-bb35-2f319bab1580 (/_internal_ping) 2020-03-31 09:07:18,311 [518] [DEBUG] [app] Ending request: urn:request:523160a7-0af9-4ab6-bb35-2f319bab1580 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:18,311 [518] [DEBUG] [app] Ending request: urn:request:523160a7-0af9-4ab6-bb35-2f319bab1580 (/_internal_ping) 2020-03-31 09:07:18,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:07:18,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:07:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:07:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:07:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:07:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:07:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:18,326 [543] [DEBUG] [app] Starting request: urn:request:f774a81a-c14c-470a-a6aa-c12cebf3cf74 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:18,326 [543] [DEBUG] [app] Starting request: urn:request:f774a81a-c14c-470a-a6aa-c12cebf3cf74 (/c1/_internal_ping) 2020-03-31 09:07:18,327 [543] [DEBUG] [app] Ending request: urn:request:f774a81a-c14c-470a-a6aa-c12cebf3cf74 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:18,327 [543] [DEBUG] [app] Ending request: urn:request:f774a81a-c14c-470a-a6aa-c12cebf3cf74 (/c1/_internal_ping) 2020-03-31 09:07:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:18,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:07:18,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:18,329 [518] [DEBUG] [app] Ending request: urn:request:1901d3f3-0494-4a51-8dbe-a5945cd95868 (/health/instance) 2020-03-31 09:07:18,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:18,329 [518] [DEBUG] [app] Ending request: urn:request:1901d3f3-0494-4a51-8dbe-a5945cd95868 (/health/instance) 2020-03-31 09:07:18,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:18,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:07:18,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:07:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:07:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:31.440707+00:00 (in 9.999202 seconds) 2020-03-31 09:07:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:31 UTC)" (scheduled at 2020-03-31 09:07:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:07:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:31.440707+00:00 (in 9.999202 seconds) 2020-03-31 09:07:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:31 UTC)" (scheduled at 2020-03-31 09:07:21.440707+00:00) 2020-03-31 09:07:21,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:07:21,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:07:21,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:07:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:07:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:07:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:31 UTC)" executed successfully 2020-03-31 09:07:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:07:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:52.113221+00:00 (in 29.999246 seconds) 2020-03-31 09:07:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:52 UTC)" (scheduled at 2020-03-31 09:07:22.113221+00:00) securityworker stdout | 2020-03-31 09:07:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:52.113221+00:00 (in 29.999246 seconds) 2020-03-31 09:07:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:52 UTC)" (scheduled at 2020-03-31 09:07:22.113221+00:00) 2020-03-31 09:07:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:07:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:07:22,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:07:22,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:07:22,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:07:22,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:07:22,136 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:22,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:07:22,136 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:22,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:07:52 UTC)" executed successfully 2020-03-31 09:07:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:31.498397+00:00 (in 7.174029 seconds) 2020-03-31 09:07:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:34 UTC)" (scheduled at 2020-03-31 09:07:24.323352+00:00) 2020-03-31 09:07:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:07:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:31.498397+00:00 (in 7.174029 seconds) 2020-03-31 09:07:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:34 UTC)" (scheduled at 2020-03-31 09:07:24.323352+00:00) 2020-03-31 09:07:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 24, 325127), True, datetime.datetime(2020, 3, 31, 9, 7, 24, 325127), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:07:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 24, 325127), True, datetime.datetime(2020, 3, 31, 9, 7, 24, 325127), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:07:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:07:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:34 UTC)" executed successfully 2020-03-31 09:07:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:07:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:33.330112+00:00 (in 8.918371 seconds) 2020-03-31 09:07:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:24 UTC)" (scheduled at 2020-03-31 09:07:24.410927+00:00) 2020-03-31 09:07:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:07:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:33.330112+00:00 (in 8.918371 seconds) 2020-03-31 09:07:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:24 UTC)" (scheduled at 2020-03-31 09:07:24.410927+00:00) 2020-03-31 09:07:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:24 UTC)" executed successfully 2020-03-31 09:07:24,446 [518] [DEBUG] [app] Starting request: urn:request:5fe955c6-8313-4be9-add4-2eacd0aa3cbc (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:24,446 [518] [DEBUG] [app] Starting request: urn:request:5fe955c6-8313-4be9-add4-2eacd0aa3cbc (/health/instance) 2020-03-31 09:07:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:24,456 [538] [DEBUG] [app] Starting request: urn:request:af0cba31-47ad-4274-982b-be015038b63e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:24,456 [538] [DEBUG] [app] Starting request: urn:request:af0cba31-47ad-4274-982b-be015038b63e (/v1/_internal_ping) 2020-03-31 09:07:24,458 [538] [DEBUG] [app] Ending request: urn:request:af0cba31-47ad-4274-982b-be015038b63e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:24,458 [538] [DEBUG] [app] Ending request: urn:request:af0cba31-47ad-4274-982b-be015038b63e (/v1/_internal_ping) 2020-03-31 09:07:24,458 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:24,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:07:24,458 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:24,461 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:07:24,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:24,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:24,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:24,461 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:24,463 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:24,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:24,463 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:24,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:24,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:24,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:24,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:24,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:24,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:24,490 [518] [DEBUG] [app] Starting request: urn:request:cd1555c7-3cb4-40d7-b4bb-d956bdf9d3af (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:24,490 [518] [DEBUG] [app] Starting request: urn:request:cd1555c7-3cb4-40d7-b4bb-d956bdf9d3af (/_internal_ping) 2020-03-31 09:07:24,491 [518] [DEBUG] [app] Ending request: urn:request:cd1555c7-3cb4-40d7-b4bb-d956bdf9d3af (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:24,491 [518] [DEBUG] [app] Ending request: urn:request:cd1555c7-3cb4-40d7-b4bb-d956bdf9d3af (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:07:24,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:24,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:07:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:07:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:07:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:07:24,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:24,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:24,508 [543] [DEBUG] [app] Starting request: urn:request:54c6ad4f-acfb-4b58-b70f-d1e3b08d6f7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:24,508 [543] [DEBUG] [app] Starting request: urn:request:54c6ad4f-acfb-4b58-b70f-d1e3b08d6f7c (/c1/_internal_ping) 2020-03-31 09:07:24,509 [543] [DEBUG] [app] Ending request: urn:request:54c6ad4f-acfb-4b58-b70f-d1e3b08d6f7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:24,509 [543] [DEBUG] [app] Ending request: urn:request:54c6ad4f-acfb-4b58-b70f-d1e3b08d6f7c (/c1/_internal_ping) 2020-03-31 09:07:24,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:24,510 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:24,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:24,512 [518] [DEBUG] [app] Ending request: urn:request:5fe955c6-8313-4be9-add4-2eacd0aa3cbc (/health/instance) 2020-03-31 09:07:24,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:07:24,510 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:07:24,512 [518] [DEBUG] [app] Ending request: urn:request:5fe955c6-8313-4be9-add4-2eacd0aa3cbc (/health/instance) 2020-03-31 09:07:24,512 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:24,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:07:24,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:07:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:36.924497+00:00 (in 9.999207 seconds) 2020-03-31 09:07:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:36 UTC)" (scheduled at 2020-03-31 09:07:26.924497+00:00) 2020-03-31 09:07:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:07:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:36.924497+00:00 (in 9.999207 seconds) 2020-03-31 09:07:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:36 UTC)" (scheduled at 2020-03-31 09:07:26.924497+00:00) 2020-03-31 09:07:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 26, 926111), True, datetime.datetime(2020, 3, 31, 9, 7, 26, 926111), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:07:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 26, 926111), True, datetime.datetime(2020, 3, 31, 9, 7, 26, 926111), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:07:26,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:07:26,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:26,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:26,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:36 UTC)" executed successfully 2020-03-31 09:07:28,278 [526] [DEBUG] [app] Starting request: urn:request:d88cc0e3-0b00-4c08-94aa-199b6a65dc93 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:28,278 [526] [DEBUG] [app] Starting request: urn:request:d88cc0e3-0b00-4c08-94aa-199b6a65dc93 (/health/instance) 2020-03-31 09:07:28,281 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:28,281 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:28,286 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:28,286 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:28,287 [555] [DEBUG] [app] Starting request: urn:request:2b72bdc9-e1af-4e84-84d3-038764bdbf8a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:28,287 [555] [DEBUG] [app] Starting request: urn:request:2b72bdc9-e1af-4e84-84d3-038764bdbf8a (/v1/_internal_ping) 2020-03-31 09:07:28,289 [555] [DEBUG] [app] Ending request: urn:request:2b72bdc9-e1af-4e84-84d3-038764bdbf8a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:28,289 [555] [DEBUG] [app] Ending request: urn:request:2b72bdc9-e1af-4e84-84d3-038764bdbf8a (/v1/_internal_ping) 2020-03-31 09:07:28,290 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:07:28,290 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:28,290 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:28,291 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:28,290 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:28,292 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:07:28,291 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:07:28,292 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:28,293 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:28,293 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:28,294 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:28,294 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:28,301 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:28,301 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:28,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:28,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:28,322 [526] [DEBUG] [app] Starting request: urn:request:e8aa8948-970a-4a1f-a652-d184c9c270e6 (/_internal_ping) 2020-03-31 09:07:28,323 [526] [DEBUG] [app] Ending request: urn:request:e8aa8948-970a-4a1f-a652-d184c9c270e6 (/_internal_ping) 2020-03-31 09:07:28,324 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:07:28,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:28,322 [526] [DEBUG] [app] Starting request: urn:request:e8aa8948-970a-4a1f-a652-d184c9c270e6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:28,323 [526] [DEBUG] [app] Ending request: urn:request:e8aa8948-970a-4a1f-a652-d184c9c270e6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:28,324 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:07:28,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:28,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:07:28,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:28,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:28,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:07:28,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:07:28,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:07:28,337 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:28,337 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:28,339 [536] [DEBUG] [app] Starting request: urn:request:01087f77-b466-4ed1-b7a7-eebc18525dc9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:28,339 [536] [DEBUG] [app] Starting request: urn:request:01087f77-b466-4ed1-b7a7-eebc18525dc9 (/c1/_internal_ping) 2020-03-31 09:07:28,340 [536] [DEBUG] [app] Ending request: urn:request:01087f77-b466-4ed1-b7a7-eebc18525dc9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:28,340 [536] [DEBUG] [app] Ending request: urn:request:01087f77-b466-4ed1-b7a7-eebc18525dc9 (/c1/_internal_ping) 2020-03-31 09:07:28,341 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:07:28,341 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:28,343 [526] [DEBUG] [app] Ending request: urn:request:d88cc0e3-0b00-4c08-94aa-199b6a65dc93 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:28,343 [526] [DEBUG] [app] Ending request: urn:request:d88cc0e3-0b00-4c08-94aa-199b6a65dc93 (/health/instance) 2020-03-31 09:07:28,343 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:28,343 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:07:28,343 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:28,343 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:07:28,344 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-verbs stdout | 2020-03-31 09:07:28,344 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:07:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:00.337699+00:00 (in 29.999283 seconds) 2020-03-31 09:07:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:00 UTC)" (scheduled at 2020-03-31 09:07:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:07:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:00.337699+00:00 (in 29.999283 seconds) 2020-03-31 09:07:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:00 UTC)" (scheduled at 2020-03-31 09:07:30.337699+00:00) 2020-03-31 09:07:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 7, 30, 339302), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:07:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 7, 30, 339302), False, 50, 1, 0]) 2020-03-31 09:07:30,353 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:07:30,353 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:07:30,353 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:30,353 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:07:30,353 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:30,353 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:00 UTC)" executed successfully 2020-03-31 09:07:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:07:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:41.440707+00:00 (in 9.999194 seconds) 2020-03-31 09:07:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:41 UTC)" (scheduled at 2020-03-31 09:07:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:07:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:41.440707+00:00 (in 9.999194 seconds) 2020-03-31 09:07:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:41 UTC)" (scheduled at 2020-03-31 09:07:31.440707+00:00) 2020-03-31 09:07:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:07:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:07:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:07:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:07:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:07:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:41 UTC)" executed successfully 2020-03-31 09:07:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:34.323352+00:00 (in 2.824104 seconds) 2020-03-31 09:07:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:31 UTC)" (scheduled at 2020-03-31 09:07:31.498397+00:00) notificationworker stdout | 2020-03-31 09:07:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:34.323352+00:00 (in 2.824104 seconds) 2020-03-31 09:07:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:31 UTC)" (scheduled at 2020-03-31 09:07:31.498397+00:00) 2020-03-31 09:07:31,500 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), u'notification/%']) notificationworker stdout | 2020-03-31 09:07:31,500 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), u'notification/%']) 2020-03-31 09:07:31,513 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), True, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:07:31,513 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), True, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), 0, u'notification/%']) 2020-03-31 09:07:31,517 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), True, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), u'notification/%']) notificationworker stdout | 2020-03-31 09:07:31,517 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), True, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 7, 31, 499920), u'notification/%']) 2020-03-31 09:07:31,521 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:07:31,521 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:31,523 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:31,523 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:12:31 UTC)" executed successfully 2020-03-31 09:07:31,525 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:07:31,525 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:31,526 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:07:31,526 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:31,526 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 09:07:31,526 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:07:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:07:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:03 UTC)" (scheduled at 2020-03-31 09:07:33.330112+00:00) 2020-03-31 09:07:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:03.330112+00:00 (in 29.999326 seconds) 2020-03-31 09:07:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:07:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:03 UTC)" (scheduled at 2020-03-31 09:07:33.330112+00:00) 2020-03-31 09:07:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:03.330112+00:00 (in 29.999326 seconds) 2020-03-31 09:07:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 33, 331833), True, datetime.datetime(2020, 3, 31, 9, 7, 33, 331833), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:07:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 33, 331833), True, datetime.datetime(2020, 3, 31, 9, 7, 33, 331833), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:07:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:07:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:07:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:03 UTC)" executed successfully 2020-03-31 09:07:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:07:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:43.539999+00:00 (in 10.172350 seconds) 2020-03-31 09:07:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:33 UTC)" (scheduled at 2020-03-31 09:07:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:07:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:43.539999+00:00 (in 10.172350 seconds) 2020-03-31 09:07:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:33 UTC)" (scheduled at 2020-03-31 09:07:33.366791+00:00) 2020-03-31 09:07:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:07:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 33, 368380), True, datetime.datetime(2020, 3, 31, 9, 7, 33, 368380), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:07:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 33, 368380), True, datetime.datetime(2020, 3, 31, 9, 7, 33, 368380), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:07:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:07:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:07:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:33 UTC)" executed successfully 2020-03-31 09:07:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:44.323352+00:00 (in 9.999110 seconds) 2020-03-31 09:07:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:44 UTC)" (scheduled at 2020-03-31 09:07:34.323352+00:00) 2020-03-31 09:07:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:07:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:44.323352+00:00 (in 9.999110 seconds) 2020-03-31 09:07:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:44 UTC)" (scheduled at 2020-03-31 09:07:34.323352+00:00) 2020-03-31 09:07:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 34, 325594), True, datetime.datetime(2020, 3, 31, 9, 7, 34, 325594), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:07:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 34, 325594), True, datetime.datetime(2020, 3, 31, 9, 7, 34, 325594), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:07:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:07:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:44 UTC)" executed successfully 2020-03-31 09:07:34,446 [518] [DEBUG] [app] Starting request: urn:request:cebe0361-0597-4e3b-a0ce-22561ce7a44d (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:34,446 [518] [DEBUG] [app] Starting request: urn:request:cebe0361-0597-4e3b-a0ce-22561ce7a44d (/health/instance) 2020-03-31 09:07:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:34,454 [556] [DEBUG] [app] Starting request: urn:request:9ad9678f-e1d7-4042-b345-74384a73fd2f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:34,454 [556] [DEBUG] [app] Starting request: urn:request:9ad9678f-e1d7-4042-b345-74384a73fd2f (/v1/_internal_ping) 2020-03-31 09:07:34,456 [556] [DEBUG] [app] Ending request: urn:request:9ad9678f-e1d7-4042-b345-74384a73fd2f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:34,456 [556] [DEBUG] [app] Ending request: urn:request:9ad9678f-e1d7-4042-b345-74384a73fd2f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:34,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:07:34,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:34,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:34,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:34,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:34,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:34,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:34,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:34,485 [518] [DEBUG] [app] Starting request: urn:request:5a79cb69-b798-41a6-a5b4-736e8617169e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:34,485 [518] [DEBUG] [app] Starting request: urn:request:5a79cb69-b798-41a6-a5b4-736e8617169e (/_internal_ping) 2020-03-31 09:07:34,486 [518] [DEBUG] [app] Ending request: urn:request:5a79cb69-b798-41a6-a5b4-736e8617169e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:34,486 [518] [DEBUG] [app] Ending request: urn:request:5a79cb69-b798-41a6-a5b4-736e8617169e (/_internal_ping) 2020-03-31 09:07:34,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:07:34,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:07:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:07:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:07:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:07:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:07:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:34,498 [536] [DEBUG] [app] Starting request: urn:request:89934e12-530c-46b6-af2e-24d898c0f837 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:34,498 [536] [DEBUG] [app] Starting request: urn:request:89934e12-530c-46b6-af2e-24d898c0f837 (/c1/_internal_ping) 2020-03-31 09:07:34,499 [536] [DEBUG] [app] Ending request: urn:request:89934e12-530c-46b6-af2e-24d898c0f837 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:34,499 [536] [DEBUG] [app] Ending request: urn:request:89934e12-530c-46b6-af2e-24d898c0f837 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:34,500 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:07:34,500 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:34,501 [518] [DEBUG] [app] Ending request: urn:request:cebe0361-0597-4e3b-a0ce-22561ce7a44d (/health/instance) 2020-03-31 09:07:34,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:34,501 [518] [DEBUG] [app] Ending request: urn:request:cebe0361-0597-4e3b-a0ce-22561ce7a44d (/health/instance) 2020-03-31 09:07:34,501 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:34,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:07:34,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:07:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:38.010934+00:00 (in 1.085636 seconds) 2020-03-31 09:07:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:46 UTC)" (scheduled at 2020-03-31 09:07:36.924497+00:00) 2020-03-31 09:07:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 36, 926237), True, datetime.datetime(2020, 3, 31, 9, 7, 36, 926237), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:07:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:38.010934+00:00 (in 1.085636 seconds) 2020-03-31 09:07:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:46 UTC)" (scheduled at 2020-03-31 09:07:36.924497+00:00) 2020-03-31 09:07:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:07:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 36, 926237), True, datetime.datetime(2020, 3, 31, 9, 7, 36, 926237), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:07:36,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:07:36,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:46 UTC)" executed successfully 2020-03-31 09:07:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:46.924497+00:00 (in 8.912802 seconds) 2020-03-31 09:07:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:38 UTC)" (scheduled at 2020-03-31 09:07:38.010934+00:00) 2020-03-31 09:07:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:46.924497+00:00 (in 8.912802 seconds) 2020-03-31 09:07:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:38 UTC)" (scheduled at 2020-03-31 09:07:38.010934+00:00) 2020-03-31 09:07:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:38 UTC)" executed successfully 2020-03-31 09:07:38,273 [518] [DEBUG] [app] Starting request: urn:request:0068b226-f7bc-4ef2-9256-f54805f3ad41 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:38,273 [518] [DEBUG] [app] Starting request: urn:request:0068b226-f7bc-4ef2-9256-f54805f3ad41 (/health/instance) 2020-03-31 09:07:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:38,278 [547] [DEBUG] [app] Starting request: urn:request:4757c755-7708-4e73-bb2e-8d9ee2f065f4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:38,278 [547] [DEBUG] [app] Starting request: urn:request:4757c755-7708-4e73-bb2e-8d9ee2f065f4 (/v1/_internal_ping) 2020-03-31 09:07:38,280 [547] [DEBUG] [app] Ending request: urn:request:4757c755-7708-4e73-bb2e-8d9ee2f065f4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:38,280 [547] [DEBUG] [app] Ending request: urn:request:4757c755-7708-4e73-bb2e-8d9ee2f065f4 (/v1/_internal_ping) 2020-03-31 09:07:38,281 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:07:38,281 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:38,283 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:07:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:07:38,283 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:38,285 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:38,285 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:38,286 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:38,286 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:38,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:38,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:38,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:38,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:38,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:38,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:38,309 [521] [DEBUG] [app] Starting request: urn:request:0c7cde1e-f408-4520-ab3b-d523c70c0f1d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:38,309 [521] [DEBUG] [app] Starting request: urn:request:0c7cde1e-f408-4520-ab3b-d523c70c0f1d (/_internal_ping) 2020-03-31 09:07:38,310 [521] [DEBUG] [app] Ending request: urn:request:0c7cde1e-f408-4520-ab3b-d523c70c0f1d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:38,310 [521] [DEBUG] [app] Ending request: urn:request:0c7cde1e-f408-4520-ab3b-d523c70c0f1d (/_internal_ping) 2020-03-31 09:07:38,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:38,311 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:07:38,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:38,311 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:07:38,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:38,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:07:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:07:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:07:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:07:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:07:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:38,323 [543] [DEBUG] [app] Starting request: urn:request:b1331ea1-21f9-4ee1-b834-142aeaf2256a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:38,323 [543] [DEBUG] [app] Starting request: urn:request:b1331ea1-21f9-4ee1-b834-142aeaf2256a (/c1/_internal_ping) 2020-03-31 09:07:38,324 [543] [DEBUG] [app] Ending request: urn:request:b1331ea1-21f9-4ee1-b834-142aeaf2256a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:38,324 [543] [DEBUG] [app] Ending request: urn:request:b1331ea1-21f9-4ee1-b834-142aeaf2256a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:38,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:38,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:07:38,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:38,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:38,326 [518] [DEBUG] [app] Ending request: urn:request:0068b226-f7bc-4ef2-9256-f54805f3ad41 (/health/instance) 2020-03-31 09:07:38,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:38,326 [518] [DEBUG] [app] Ending request: urn:request:0068b226-f7bc-4ef2-9256-f54805f3ad41 (/health/instance) 2020-03-31 09:07:38,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:38,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:07:38,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:07:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:07:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:51.440707+00:00 (in 9.999016 seconds) 2020-03-31 09:07:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:51 UTC)" (scheduled at 2020-03-31 09:07:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:07:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:51.440707+00:00 (in 9.999016 seconds) 2020-03-31 09:07:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:51 UTC)" (scheduled at 2020-03-31 09:07:41.440707+00:00) 2020-03-31 09:07:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:07:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:07:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:07:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:07:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:07:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:51 UTC)" executed successfully 2020-03-31 09:07:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:07:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:33.366791+00:00 (in 49.825928 seconds) 2020-03-31 09:07:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:43 UTC)" (scheduled at 2020-03-31 09:07:43.539999+00:00) 2020-03-31 09:07:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:07:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:33.366791+00:00 (in 49.825928 seconds) 2020-03-31 09:07:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:43 UTC)" (scheduled at 2020-03-31 09:07:43.539999+00:00) 2020-03-31 09:07:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:07:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:43 UTC)" executed successfully 2020-03-31 09:07:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:49.498966+00:00 (in 5.174163 seconds) 2020-03-31 09:07:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:54 UTC)" (scheduled at 2020-03-31 09:07:44.323352+00:00) 2020-03-31 09:07:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:07:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:49.498966+00:00 (in 5.174163 seconds) 2020-03-31 09:07:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:54 UTC)" (scheduled at 2020-03-31 09:07:44.323352+00:00) 2020-03-31 09:07:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 44, 325835), True, datetime.datetime(2020, 3, 31, 9, 7, 44, 325835), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:07:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 44, 325835), True, datetime.datetime(2020, 3, 31, 9, 7, 44, 325835), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:07:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:07:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:54 UTC)" executed successfully 2020-03-31 09:07:44,446 [518] [DEBUG] [app] Starting request: urn:request:68e55312-2150-4d5b-a718-79a64462e3e4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:44,446 [518] [DEBUG] [app] Starting request: urn:request:68e55312-2150-4d5b-a718-79a64462e3e4 (/health/instance) 2020-03-31 09:07:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:44,454 [546] [DEBUG] [app] Starting request: urn:request:a5cab9a4-d7b2-4b95-acb5-631ae8390414 (/v1/_internal_ping) 2020-03-31 09:07:44,455 [546] [DEBUG] [app] Ending request: urn:request:a5cab9a4-d7b2-4b95-acb5-631ae8390414 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:44,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:44,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:07:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-registry stdout | 2020-03-31 09:07:44,454 [546] [DEBUG] [app] Starting request: urn:request:a5cab9a4-d7b2-4b95-acb5-631ae8390414 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:44,455 [546] [DEBUG] [app] Ending request: urn:request:a5cab9a4-d7b2-4b95-acb5-631ae8390414 (/v1/_internal_ping) 2020-03-31 09:07:44,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:44,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:44,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:07:44,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:44,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:44,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:44,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:44,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:44,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:44,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:44,495 [527] [DEBUG] [app] Starting request: urn:request:eb363f4c-64f8-46a2-83c7-f4b6d8e597ae (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:44,495 [527] [DEBUG] [app] Starting request: urn:request:eb363f4c-64f8-46a2-83c7-f4b6d8e597ae (/_internal_ping) 2020-03-31 09:07:44,496 [527] [DEBUG] [app] Ending request: urn:request:eb363f4c-64f8-46a2-83c7-f4b6d8e597ae (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:44,496 [527] [DEBUG] [app] Ending request: urn:request:eb363f4c-64f8-46a2-83c7-f4b6d8e597ae (/_internal_ping) 2020-03-31 09:07:44,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:07:44,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:07:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:07:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:44,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:44,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:07:44,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:07:44,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:07:44,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:44,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:44,512 [541] [DEBUG] [app] Starting request: urn:request:5f801d21-6bb4-4604-ae42-0e7d8fa2fbbc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:44,512 [541] [DEBUG] [app] Starting request: urn:request:5f801d21-6bb4-4604-ae42-0e7d8fa2fbbc (/c1/_internal_ping) 2020-03-31 09:07:44,513 [541] [DEBUG] [app] Ending request: urn:request:5f801d21-6bb4-4604-ae42-0e7d8fa2fbbc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:44,513 [541] [DEBUG] [app] Ending request: urn:request:5f801d21-6bb4-4604-ae42-0e7d8fa2fbbc (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:44,513 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:44,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:07:44,513 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:44,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:44,515 [518] [DEBUG] [app] Ending request: urn:request:68e55312-2150-4d5b-a718-79a64462e3e4 (/health/instance) 2020-03-31 09:07:44,516 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:44,515 [518] [DEBUG] [app] Ending request: urn:request:68e55312-2150-4d5b-a718-79a64462e3e4 (/health/instance) 2020-03-31 09:07:44,516 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:44,516 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:07:44,516 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:07:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:07:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:45,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:15.064514+00:00 (in 29.997138 seconds) 2020-03-31 09:07:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:15 UTC)" (scheduled at 2020-03-31 09:07:45.064514+00:00) gcworker stdout | 2020-03-31 09:07:45,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:15.064514+00:00 (in 29.997138 seconds) 2020-03-31 09:07:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:15 UTC)" (scheduled at 2020-03-31 09:07:45.064514+00:00) 2020-03-31 09:07:45,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436065, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:07:45,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436065, 1209600, 500, 1, 0]) 2020-03-31 09:07:45,092 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:07:45,092 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:07:45,092 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:45,092 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:07:45,092 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:45,092 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:15 UTC)" executed successfully 2020-03-31 09:07:46,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:46,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:46,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:56.924497+00:00 (in 9.996991 seconds) 2020-03-31 09:07:46,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:56 UTC)" (scheduled at 2020-03-31 09:07:46.924497+00:00) 2020-03-31 09:07:46,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:07:46,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:56.924497+00:00 (in 9.996991 seconds) 2020-03-31 09:07:46,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:56 UTC)" (scheduled at 2020-03-31 09:07:46.924497+00:00) 2020-03-31 09:07:46,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:46,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 46, 928452), True, datetime.datetime(2020, 3, 31, 9, 7, 46, 928452), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:07:46,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 46, 928452), True, datetime.datetime(2020, 3, 31, 9, 7, 46, 928452), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:07:46,950 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:07:46,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:46,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:46,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:46,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:46,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:07:56 UTC)" executed successfully 2020-03-31 09:07:48,273 [527] [DEBUG] [app] Starting request: urn:request:0d06962c-7cdf-47a9-abcd-e9bd92d2cb0c (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:48,273 [527] [DEBUG] [app] Starting request: urn:request:0d06962c-7cdf-47a9-abcd-e9bd92d2cb0c (/health/instance) 2020-03-31 09:07:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:48,280 [546] [DEBUG] [app] Starting request: urn:request:be1fb594-26bd-40f1-a55a-9bfb22445b8e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:48,280 [546] [DEBUG] [app] Starting request: urn:request:be1fb594-26bd-40f1-a55a-9bfb22445b8e (/v1/_internal_ping) 2020-03-31 09:07:48,281 [546] [DEBUG] [app] Ending request: urn:request:be1fb594-26bd-40f1-a55a-9bfb22445b8e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:48,281 [546] [DEBUG] [app] Ending request: urn:request:be1fb594-26bd-40f1-a55a-9bfb22445b8e (/v1/_internal_ping) 2020-03-31 09:07:48,281 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:07:48,281 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:48,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:48,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:48,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:48,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:48,284 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:48,284 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:48,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:48,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:48,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:48,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:48,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:48,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:48,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:48,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:48,310 [527] [DEBUG] [app] Starting request: urn:request:6c381353-3f19-4d07-83a9-ccc5fa070a8c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:48,310 [527] [DEBUG] [app] Starting request: urn:request:6c381353-3f19-4d07-83a9-ccc5fa070a8c (/_internal_ping) 2020-03-31 09:07:48,311 [527] [DEBUG] [app] Ending request: urn:request:6c381353-3f19-4d07-83a9-ccc5fa070a8c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:48,311 [527] [DEBUG] [app] Ending request: urn:request:6c381353-3f19-4d07-83a9-ccc5fa070a8c (/_internal_ping) 2020-03-31 09:07:48,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:07:48,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:48,311 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:07:48,311 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:07:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:07:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:07:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:48,324 [536] [DEBUG] [app] Starting request: urn:request:21bf83ba-0ff4-49a6-9891-5336ce9cd274 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:48,324 [536] [DEBUG] [app] Starting request: urn:request:21bf83ba-0ff4-49a6-9891-5336ce9cd274 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:48,325 [536] [DEBUG] [app] Ending request: urn:request:21bf83ba-0ff4-49a6-9891-5336ce9cd274 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:07:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:48,325 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:07:48,325 [536] [DEBUG] [app] Ending request: urn:request:21bf83ba-0ff4-49a6-9891-5336ce9cd274 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:48,328 [527] [DEBUG] [app] Ending request: urn:request:0d06962c-7cdf-47a9-abcd-e9bd92d2cb0c (/health/instance) 2020-03-31 09:07:48,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:07:48,325 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:07:48,328 [527] [DEBUG] [app] Ending request: urn:request:0d06962c-7cdf-47a9-abcd-e9bd92d2cb0c (/health/instance) 2020-03-31 09:07:48,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:07:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:54.323352+00:00 (in 4.823015 seconds) 2020-03-31 09:07:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:49 UTC)" (scheduled at 2020-03-31 09:07:49.498966+00:00) 2020-03-31 09:07:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:54.323352+00:00 (in 4.823015 seconds) 2020-03-31 09:07:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:49 UTC)" (scheduled at 2020-03-31 09:07:49.498966+00:00) 2020-03-31 09:07:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:49 UTC)" executed successfully 2020-03-31 09:07:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:07:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:59.560326+00:00 (in 8.447905 seconds) 2020-03-31 09:07:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:51 UTC)" (scheduled at 2020-03-31 09:07:51.110610+00:00) 2020-03-31 09:07:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:07:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:07:59.560326+00:00 (in 8.447905 seconds) 2020-03-31 09:07:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:51 UTC)" (scheduled at 2020-03-31 09:07:51.110610+00:00) 2020-03-31 09:07:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 51, 113206), True, datetime.datetime(2020, 3, 31, 9, 7, 51, 113206), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:07:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 51, 113206), True, datetime.datetime(2020, 3, 31, 9, 7, 51, 113206), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:07:51,302 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:07:51,302 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:51,302 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:51,302 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:07:51,302 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:51,302 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:51 UTC)" executed successfully 2020-03-31 09:07:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:07:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:01 UTC)" (scheduled at 2020-03-31 09:07:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:07:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:01 UTC)" (scheduled at 2020-03-31 09:07:51.440707+00:00) 2020-03-31 09:07:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:01.440707+00:00 (in 9.999077 seconds) repositoryactioncounter stdout | 2020-03-31 09:07:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:01.440707+00:00 (in 9.999077 seconds) 2020-03-31 09:07:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:07:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:07:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:07:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:07:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:07:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:01 UTC)" executed successfully 2020-03-31 09:07:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:07:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:22.113221+00:00 (in 29.999199 seconds) 2020-03-31 09:07:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:22 UTC)" (scheduled at 2020-03-31 09:07:52.113221+00:00) securityworker stdout | 2020-03-31 09:07:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:22.113221+00:00 (in 29.999199 seconds) 2020-03-31 09:07:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:22 UTC)" (scheduled at 2020-03-31 09:07:52.113221+00:00) 2020-03-31 09:07:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:07:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:07:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:07:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:07:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:07:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:07:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:07:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:22 UTC)" executed successfully 2020-03-31 09:07:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:07:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:04.323352+00:00 (in 9.998893 seconds) 2020-03-31 09:07:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:04 UTC)" (scheduled at 2020-03-31 09:07:54.323352+00:00) 2020-03-31 09:07:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:07:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:04.323352+00:00 (in 9.998893 seconds) 2020-03-31 09:07:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:04 UTC)" (scheduled at 2020-03-31 09:07:54.323352+00:00) 2020-03-31 09:07:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 54, 325351), True, datetime.datetime(2020, 3, 31, 9, 7, 54, 325351), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:07:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 54, 325351), True, datetime.datetime(2020, 3, 31, 9, 7, 54, 325351), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:07:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:07:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:07:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:04 UTC)" executed successfully 2020-03-31 09:07:54,446 [525] [DEBUG] [app] Starting request: urn:request:09875139-a889-42c2-a0e1-59a2d3cdcadf (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:54,446 [525] [DEBUG] [app] Starting request: urn:request:09875139-a889-42c2-a0e1-59a2d3cdcadf (/health/instance) 2020-03-31 09:07:54,449 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:54,449 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:54,453 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:54,453 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:54,455 [540] [DEBUG] [app] Starting request: urn:request:c3d353f6-67ec-4030-8f42-1d54c00d4cab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:54,455 [540] [DEBUG] [app] Starting request: urn:request:c3d353f6-67ec-4030-8f42-1d54c00d4cab (/v1/_internal_ping) 2020-03-31 09:07:54,456 [540] [DEBUG] [app] Ending request: urn:request:c3d353f6-67ec-4030-8f42-1d54c00d4cab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:54,456 [540] [DEBUG] [app] Ending request: urn:request:c3d353f6-67ec-4030-8f42-1d54c00d4cab (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:54,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:54,456 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:07:54,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:54,457 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:54,456 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:54,458 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:07:54,457 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:07:54,458 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:54,460 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:54,460 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:54,460 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:54,460 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:54,467 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:54,467 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:54,483 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:54,483 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:54,489 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:54,489 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:54,490 [518] [DEBUG] [app] Starting request: urn:request:b3c36ad5-0b89-47d9-b539-909cc26b0240 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:54,490 [518] [DEBUG] [app] Starting request: urn:request:b3c36ad5-0b89-47d9-b539-909cc26b0240 (/_internal_ping) 2020-03-31 09:07:54,491 [518] [DEBUG] [app] Ending request: urn:request:b3c36ad5-0b89-47d9-b539-909cc26b0240 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:54,491 [518] [DEBUG] [app] Ending request: urn:request:b3c36ad5-0b89-47d9-b539-909cc26b0240 (/_internal_ping) 2020-03-31 09:07:54,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:54,492 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:07:54,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:54,492 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:07:54,495 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:54,495 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:54,496 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:07:54,496 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:07:54,499 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:54,499 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:54,500 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:07:54,500 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:07:54,503 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:54,503 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:54,504 [536] [DEBUG] [app] Starting request: urn:request:b0f4f277-0150-40b2-ae82-e87369bf035f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:54,504 [536] [DEBUG] [app] Starting request: urn:request:b0f4f277-0150-40b2-ae82-e87369bf035f (/c1/_internal_ping) 2020-03-31 09:07:54,505 [536] [DEBUG] [app] Ending request: urn:request:b0f4f277-0150-40b2-ae82-e87369bf035f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:54,505 [536] [DEBUG] [app] Ending request: urn:request:b0f4f277-0150-40b2-ae82-e87369bf035f (/c1/_internal_ping) 2020-03-31 09:07:54,506 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:54,506 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:54,506 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:07:54,506 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:54,508 [525] [DEBUG] [app] Ending request: urn:request:09875139-a889-42c2-a0e1-59a2d3cdcadf (/health/instance) 2020-03-31 09:07:54,508 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:54,508 [525] [DEBUG] [app] Ending request: urn:request:09875139-a889-42c2-a0e1-59a2d3cdcadf (/health/instance) 2020-03-31 09:07:54,508 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:54,508 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:07:54,508 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:07:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:07:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:06.924497+00:00 (in 9.998995 seconds) 2020-03-31 09:07:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:06 UTC)" (scheduled at 2020-03-31 09:07:56.924497+00:00) 2020-03-31 09:07:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:07:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:06.924497+00:00 (in 9.998995 seconds) 2020-03-31 09:07:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:06 UTC)" (scheduled at 2020-03-31 09:07:56.924497+00:00) 2020-03-31 09:07:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:07:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 56, 926239), True, datetime.datetime(2020, 3, 31, 9, 7, 56, 926239), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:07:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 7, 56, 926239), True, datetime.datetime(2020, 3, 31, 9, 7, 56, 926239), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:07:56,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:07:56,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:07:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:06 UTC)" executed successfully 2020-03-31 09:07:58,273 [527] [DEBUG] [app] Starting request: urn:request:f9b88e70-e3d1-4bde-b9aa-13e8a289d9d7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:07:58,273 [527] [DEBUG] [app] Starting request: urn:request:f9b88e70-e3d1-4bde-b9aa-13e8a289d9d7 (/health/instance) 2020-03-31 09:07:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:58,281 [538] [DEBUG] [app] Starting request: urn:request:495d5cda-b871-40b1-9a09-d37e6aa0a31a (/v1/_internal_ping) 2020-03-31 09:07:58,281 [538] [DEBUG] [app] Ending request: urn:request:495d5cda-b871-40b1-9a09-d37e6aa0a31a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:07:58,281 [538] [DEBUG] [app] Starting request: urn:request:495d5cda-b871-40b1-9a09-d37e6aa0a31a (/v1/_internal_ping) 2020-03-31 09:07:58,281 [538] [DEBUG] [app] Ending request: urn:request:495d5cda-b871-40b1-9a09-d37e6aa0a31a (/v1/_internal_ping) 2020-03-31 09:07:58,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:07:58,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:07:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:07:58,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:07:58,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:07:58,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:07:58,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:07:58,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:07:58,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:07:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:07:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:07:58,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:07:58,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:07:58,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:58,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:58,315 [520] [DEBUG] [app] Starting request: urn:request:cc8a3f83-73ee-450d-82cb-2f1f04696171 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:58,315 [520] [DEBUG] [app] Starting request: urn:request:cc8a3f83-73ee-450d-82cb-2f1f04696171 (/_internal_ping) 2020-03-31 09:07:58,315 [520] [DEBUG] [app] Ending request: urn:request:cc8a3f83-73ee-450d-82cb-2f1f04696171 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:07:58,315 [520] [DEBUG] [app] Ending request: urn:request:cc8a3f83-73ee-450d-82cb-2f1f04696171 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:07:58,316 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:07:58,316 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:07:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:58,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:07:58,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:07:58,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:58,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:07:58,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:07:58,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:07:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:07:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:07:58,329 [536] [DEBUG] [app] Starting request: urn:request:ddec145a-eb30-4ff1-9dd8-3363c1396019 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:58,329 [536] [DEBUG] [app] Starting request: urn:request:ddec145a-eb30-4ff1-9dd8-3363c1396019 (/c1/_internal_ping) 2020-03-31 09:07:58,329 [536] [DEBUG] [app] Ending request: urn:request:ddec145a-eb30-4ff1-9dd8-3363c1396019 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:07:58,329 [536] [DEBUG] [app] Ending request: urn:request:ddec145a-eb30-4ff1-9dd8-3363c1396019 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:58,330 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:07:58,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:07:58,330 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:07:58,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:07:58,331 [527] [DEBUG] [app] Ending request: urn:request:f9b88e70-e3d1-4bde-b9aa-13e8a289d9d7 (/health/instance) 2020-03-31 09:07:58,332 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:07:58,331 [527] [DEBUG] [app] Ending request: urn:request:f9b88e70-e3d1-4bde-b9aa-13e8a289d9d7 (/health/instance) 2020-03-31 09:07:58,332 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:07:58,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:07:58,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:07:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:07:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:07:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:07:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:07:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:51.110610+00:00 (in 51.549513 seconds) 2020-03-31 09:07:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:59 UTC)" (scheduled at 2020-03-31 09:07:59.560326+00:00) 2020-03-31 09:07:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:07:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:51.110610+00:00 (in 51.549513 seconds) 2020-03-31 09:07:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:59 UTC)" (scheduled at 2020-03-31 09:07:59.560326+00:00) 2020-03-31 09:07:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:07:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:08:59 UTC)" executed successfully 2020-03-31 09:08:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:08:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:30.337699+00:00 (in 29.999200 seconds) 2020-03-31 09:08:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:30 UTC)" (scheduled at 2020-03-31 09:08:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:08:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:30.337699+00:00 (in 29.999200 seconds) 2020-03-31 09:08:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:30 UTC)" (scheduled at 2020-03-31 09:08:00.337699+00:00) 2020-03-31 09:08:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 8, 0, 339347), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:08:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 8, 0, 339347), False, 50, 1, 0]) 2020-03-31 09:08:00,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:08:00,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:08:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:08:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:30 UTC)" executed successfully 2020-03-31 09:08:01,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:08:01,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:11.440707+00:00 (in 9.997500 seconds) 2020-03-31 09:08:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:11 UTC)" (scheduled at 2020-03-31 09:08:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:08:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:11.440707+00:00 (in 9.997500 seconds) 2020-03-31 09:08:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:11 UTC)" (scheduled at 2020-03-31 09:08:01.440707+00:00) 2020-03-31 09:08:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:08:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:08:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:08:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:08:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:08:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:11 UTC)" executed successfully 2020-03-31 09:08:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:24.410927+00:00 (in 21.079273 seconds) 2020-03-31 09:08:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:33 UTC)" (scheduled at 2020-03-31 09:08:03.330112+00:00) 2020-03-31 09:08:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:08:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:08:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:24.410927+00:00 (in 21.079273 seconds) 2020-03-31 09:08:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:33 UTC)" (scheduled at 2020-03-31 09:08:03.330112+00:00) 2020-03-31 09:08:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 3, 332560), True, datetime.datetime(2020, 3, 31, 9, 8, 3, 332560), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:08:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 3, 332560), True, datetime.datetime(2020, 3, 31, 9, 8, 3, 332560), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:08:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:03,352 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:08:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:08:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:33 UTC)" executed successfully 2020-03-31 09:08:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:08:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:14.323352+00:00 (in 9.999253 seconds) 2020-03-31 09:08:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:14 UTC)" (scheduled at 2020-03-31 09:08:04.323352+00:00) 2020-03-31 09:08:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:08:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:14.323352+00:00 (in 9.999253 seconds) 2020-03-31 09:08:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:14 UTC)" (scheduled at 2020-03-31 09:08:04.323352+00:00) 2020-03-31 09:08:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 4, 325208), True, datetime.datetime(2020, 3, 31, 9, 8, 4, 325208), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:08:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 4, 325208), True, datetime.datetime(2020, 3, 31, 9, 8, 4, 325208), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:08:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:08:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:08:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:14 UTC)" executed successfully 2020-03-31 09:08:04,446 [518] [DEBUG] [app] Starting request: urn:request:8c46fc1d-ddb0-4038-8f2e-4ca9659a557b (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:04,446 [518] [DEBUG] [app] Starting request: urn:request:8c46fc1d-ddb0-4038-8f2e-4ca9659a557b (/health/instance) 2020-03-31 09:08:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:04,454 [551] [DEBUG] [app] Starting request: urn:request:4b056241-ab39-4060-a255-8c4ace59a5db (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:04,454 [551] [DEBUG] [app] Starting request: urn:request:4b056241-ab39-4060-a255-8c4ace59a5db (/v1/_internal_ping) 2020-03-31 09:08:04,456 [551] [DEBUG] [app] Ending request: urn:request:4b056241-ab39-4060-a255-8c4ace59a5db (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:04,456 [551] [DEBUG] [app] Ending request: urn:request:4b056241-ab39-4060-a255-8c4ace59a5db (/v1/_internal_ping) 2020-03-31 09:08:04,457 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:04,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:08:04,457 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:04,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:04,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:04,460 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:08:04,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:08:04,460 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:04,461 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:04,461 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:04,462 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:04,462 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:04,486 [527] [DEBUG] [app] Starting request: urn:request:760512ac-e326-4259-a654-6a0825933e81 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:04,486 [527] [DEBUG] [app] Starting request: urn:request:760512ac-e326-4259-a654-6a0825933e81 (/_internal_ping) 2020-03-31 09:08:04,487 [527] [DEBUG] [app] Ending request: urn:request:760512ac-e326-4259-a654-6a0825933e81 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:04,487 [527] [DEBUG] [app] Ending request: urn:request:760512ac-e326-4259-a654-6a0825933e81 (/_internal_ping) 2020-03-31 09:08:04,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:04,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:08:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:08:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:04,500 [541] [DEBUG] [app] Starting request: urn:request:7610804e-7897-43fd-ad4f-3425400047e9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:04,500 [541] [DEBUG] [app] Starting request: urn:request:7610804e-7897-43fd-ad4f-3425400047e9 (/c1/_internal_ping) 2020-03-31 09:08:04,500 [541] [DEBUG] [app] Ending request: urn:request:7610804e-7897-43fd-ad4f-3425400047e9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:04,500 [541] [DEBUG] [app] Ending request: urn:request:7610804e-7897-43fd-ad4f-3425400047e9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:04,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:08:04,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:04,502 [518] [DEBUG] [app] Ending request: urn:request:8c46fc1d-ddb0-4038-8f2e-4ca9659a557b (/health/instance) 2020-03-31 09:08:04,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:04,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:08:04,502 [518] [DEBUG] [app] Ending request: urn:request:8c46fc1d-ddb0-4038-8f2e-4ca9659a557b (/health/instance) 2020-03-31 09:08:04,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:04,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:08:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:08:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:16.924497+00:00 (in 9.999288 seconds) storagereplication stdout | 2020-03-31 09:08:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:16.924497+00:00 (in 9.999288 seconds) 2020-03-31 09:08:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:16 UTC)" (scheduled at 2020-03-31 09:08:06.924497+00:00) storagereplication stdout | 2020-03-31 09:08:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:16 UTC)" (scheduled at 2020-03-31 09:08:06.924497+00:00) 2020-03-31 09:08:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:08:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 6, 927507), True, datetime.datetime(2020, 3, 31, 9, 8, 6, 927507), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:08:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 6, 927507), True, datetime.datetime(2020, 3, 31, 9, 8, 6, 927507), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:08:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:08:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:08:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:16 UTC)" executed successfully 2020-03-31 09:08:08,272 [518] [DEBUG] [app] Starting request: urn:request:aa5d1acd-e7f5-440a-ac36-a5e841774b14 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:08,272 [518] [DEBUG] [app] Starting request: urn:request:aa5d1acd-e7f5-440a-ac36-a5e841774b14 (/health/instance) 2020-03-31 09:08:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:08,278 [538] [DEBUG] [app] Starting request: urn:request:5652a75b-7644-40bb-b3e3-5d81ab03eebf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:08,278 [538] [DEBUG] [app] Starting request: urn:request:5652a75b-7644-40bb-b3e3-5d81ab03eebf (/v1/_internal_ping) 2020-03-31 09:08:08,279 [538] [DEBUG] [app] Ending request: urn:request:5652a75b-7644-40bb-b3e3-5d81ab03eebf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:08,279 [538] [DEBUG] [app] Ending request: urn:request:5652a75b-7644-40bb-b3e3-5d81ab03eebf (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:08,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:08,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:08:08,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:08,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:08:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:08:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:08,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:08,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:08,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:08,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:08,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:08,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:08,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:08,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:08,311 [518] [DEBUG] [app] Starting request: urn:request:f39135fa-e291-45de-9a0e-0fd75707c2bb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:08,311 [518] [DEBUG] [app] Starting request: urn:request:f39135fa-e291-45de-9a0e-0fd75707c2bb (/_internal_ping) 2020-03-31 09:08:08,312 [518] [DEBUG] [app] Ending request: urn:request:f39135fa-e291-45de-9a0e-0fd75707c2bb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:08,312 [518] [DEBUG] [app] Ending request: urn:request:f39135fa-e291-45de-9a0e-0fd75707c2bb (/_internal_ping) 2020-03-31 09:08:08,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:08,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:08,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:08,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:08:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:08:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:08,325 [541] [DEBUG] [app] Starting request: urn:request:fd12dd54-206d-49a9-9da5-5c6d99426af5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:08,325 [541] [DEBUG] [app] Starting request: urn:request:fd12dd54-206d-49a9-9da5-5c6d99426af5 (/c1/_internal_ping) 2020-03-31 09:08:08,326 [541] [DEBUG] [app] Ending request: urn:request:fd12dd54-206d-49a9-9da5-5c6d99426af5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:08,326 [541] [DEBUG] [app] Ending request: urn:request:fd12dd54-206d-49a9-9da5-5c6d99426af5 (/c1/_internal_ping) 2020-03-31 09:08:08,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:08,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:08,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:08,328 [518] [DEBUG] [app] Ending request: urn:request:aa5d1acd-e7f5-440a-ac36-a5e841774b14 (/health/instance) 2020-03-31 09:08:08,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:08:08,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:08,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:08:08,328 [518] [DEBUG] [app] Ending request: urn:request:aa5d1acd-e7f5-440a-ac36-a5e841774b14 (/health/instance) 2020-03-31 09:08:08,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:08,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:08:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:08:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:21.440707+00:00 (in 9.999187 seconds) 2020-03-31 09:08:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:21 UTC)" (scheduled at 2020-03-31 09:08:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:08:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:21.440707+00:00 (in 9.999187 seconds) 2020-03-31 09:08:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:21 UTC)" (scheduled at 2020-03-31 09:08:11.440707+00:00) 2020-03-31 09:08:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:08:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:08:11,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:08:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:08:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:08:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:21 UTC)" executed successfully 2020-03-31 09:08:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:08:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:14 UTC)" (scheduled at 2020-03-31 09:08:14.323352+00:00) 2020-03-31 09:08:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:08:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:14 UTC)" (scheduled at 2020-03-31 09:08:14.323352+00:00) 2020-03-31 09:08:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:24.323352+00:00 (in 9.999048 seconds) 2020-03-31 09:08:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 14, 325454), True, datetime.datetime(2020, 3, 31, 9, 8, 14, 325454), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:08:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:24.323352+00:00 (in 9.999048 seconds) 2020-03-31 09:08:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 14, 325454), True, datetime.datetime(2020, 3, 31, 9, 8, 14, 325454), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:08:14,351 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:08:14,351 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:14,351 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:14,351 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:08:14,351 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:14,351 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:24 UTC)" executed successfully 2020-03-31 09:08:14,445 [527] [DEBUG] [app] Starting request: urn:request:71916d3a-f14f-47a6-81ac-1635976d9567 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:14,445 [527] [DEBUG] [app] Starting request: urn:request:71916d3a-f14f-47a6-81ac-1635976d9567 (/health/instance) 2020-03-31 09:08:14,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:14,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:14,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:14,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:14,453 [554] [DEBUG] [app] Starting request: urn:request:9a4f163e-d5a9-4f51-b725-01e09edebc50 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:14,453 [554] [DEBUG] [app] Starting request: urn:request:9a4f163e-d5a9-4f51-b725-01e09edebc50 (/v1/_internal_ping) 2020-03-31 09:08:14,454 [554] [DEBUG] [app] Ending request: urn:request:9a4f163e-d5a9-4f51-b725-01e09edebc50 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:14,454 [554] [DEBUG] [app] Ending request: urn:request:9a4f163e-d5a9-4f51-b725-01e09edebc50 (/v1/_internal_ping) 2020-03-31 09:08:14,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:08:14,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:08:14,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:08:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:08:14,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:14,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:14,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:14,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:14,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:14,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:14,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:14,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:14,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:14,487 [527] [DEBUG] [app] Starting request: urn:request:44a38193-7f95-4302-b6ac-dd60b6d9d129 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:14,487 [527] [DEBUG] [app] Starting request: urn:request:44a38193-7f95-4302-b6ac-dd60b6d9d129 (/_internal_ping) 2020-03-31 09:08:14,488 [527] [DEBUG] [app] Ending request: urn:request:44a38193-7f95-4302-b6ac-dd60b6d9d129 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:14,488 [527] [DEBUG] [app] Ending request: urn:request:44a38193-7f95-4302-b6ac-dd60b6d9d129 (/_internal_ping) 2020-03-31 09:08:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:08:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:08:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:14,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:08:14,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:14,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:14,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:14,502 [541] [DEBUG] [app] Starting request: urn:request:4ca01ee9-3b28-4f47-a3ff-2e7661b707cc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:14,502 [541] [DEBUG] [app] Starting request: urn:request:4ca01ee9-3b28-4f47-a3ff-2e7661b707cc (/c1/_internal_ping) 2020-03-31 09:08:14,503 [541] [DEBUG] [app] Ending request: urn:request:4ca01ee9-3b28-4f47-a3ff-2e7661b707cc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:14,503 [541] [DEBUG] [app] Ending request: urn:request:4ca01ee9-3b28-4f47-a3ff-2e7661b707cc (/c1/_internal_ping) 2020-03-31 09:08:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:14,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:14,506 [527] [DEBUG] [app] Ending request: urn:request:71916d3a-f14f-47a6-81ac-1635976d9567 (/health/instance) 2020-03-31 09:08:14,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:08:14,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:14,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:08:14,506 [527] [DEBUG] [app] Ending request: urn:request:71916d3a-f14f-47a6-81ac-1635976d9567 (/health/instance) 2020-03-31 09:08:14,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:14,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:08:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:08:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:45.064514+00:00 (in 29.998723 seconds) 2020-03-31 09:08:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:45 UTC)" (scheduled at 2020-03-31 09:08:15.064514+00:00) gcworker stdout | 2020-03-31 09:08:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:45.064514+00:00 (in 29.998723 seconds) 2020-03-31 09:08:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:45 UTC)" (scheduled at 2020-03-31 09:08:15.064514+00:00) 2020-03-31 09:08:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436095, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:08:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436095, 1209600, 500, 1, 0]) 2020-03-31 09:08:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:08:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:08:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:08:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:45 UTC)" executed successfully 2020-03-31 09:08:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:08:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:26.924497+00:00 (in 9.999254 seconds) 2020-03-31 09:08:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:26 UTC)" (scheduled at 2020-03-31 09:08:16.924497+00:00) 2020-03-31 09:08:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:08:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:26.924497+00:00 (in 9.999254 seconds) 2020-03-31 09:08:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:26 UTC)" (scheduled at 2020-03-31 09:08:16.924497+00:00) 2020-03-31 09:08:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 16, 925988), True, datetime.datetime(2020, 3, 31, 9, 8, 16, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:08:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 16, 925988), True, datetime.datetime(2020, 3, 31, 9, 8, 16, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:08:16,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:08:16,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:08:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:26 UTC)" executed successfully 2020-03-31 09:08:18,273 [518] [DEBUG] [app] Starting request: urn:request:9187c547-6716-49a6-8b71-c723b483bcd5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:18,273 [518] [DEBUG] [app] Starting request: urn:request:9187c547-6716-49a6-8b71-c723b483bcd5 (/health/instance) 2020-03-31 09:08:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:18,280 [538] [DEBUG] [app] Starting request: urn:request:483c8097-efcf-400f-a512-8ade315b39b4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:18,280 [538] [DEBUG] [app] Starting request: urn:request:483c8097-efcf-400f-a512-8ade315b39b4 (/v1/_internal_ping) 2020-03-31 09:08:18,281 [538] [DEBUG] [app] Ending request: urn:request:483c8097-efcf-400f-a512-8ade315b39b4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:18,281 [538] [DEBUG] [app] Ending request: urn:request:483c8097-efcf-400f-a512-8ade315b39b4 (/v1/_internal_ping) 2020-03-31 09:08:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:08:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:18,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:08:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:08:18,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:18,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:18,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:18,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:18,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:18,315 [526] [DEBUG] [app] Starting request: urn:request:365d71b8-e87d-4c73-9e6e-f0f16f323491 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:18,315 [526] [DEBUG] [app] Starting request: urn:request:365d71b8-e87d-4c73-9e6e-f0f16f323491 (/_internal_ping) 2020-03-31 09:08:18,316 [526] [DEBUG] [app] Ending request: urn:request:365d71b8-e87d-4c73-9e6e-f0f16f323491 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:18,316 [526] [DEBUG] [app] Ending request: urn:request:365d71b8-e87d-4c73-9e6e-f0f16f323491 (/_internal_ping) 2020-03-31 09:08:18,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:18,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:08:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:08:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:08:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:18,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:18,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:18,330 [543] [DEBUG] [app] Starting request: urn:request:775aee68-bae2-496e-a0bc-38f21421a5dd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:18,330 [543] [DEBUG] [app] Starting request: urn:request:775aee68-bae2-496e-a0bc-38f21421a5dd (/c1/_internal_ping) 2020-03-31 09:08:18,331 [543] [DEBUG] [app] Ending request: urn:request:775aee68-bae2-496e-a0bc-38f21421a5dd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:18,331 [543] [DEBUG] [app] Ending request: urn:request:775aee68-bae2-496e-a0bc-38f21421a5dd (/c1/_internal_ping) 2020-03-31 09:08:18,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:18,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:18,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:08:18,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:18,334 [518] [DEBUG] [app] Ending request: urn:request:9187c547-6716-49a6-8b71-c723b483bcd5 (/health/instance) 2020-03-31 09:08:18,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:18,334 [518] [DEBUG] [app] Ending request: urn:request:9187c547-6716-49a6-8b71-c723b483bcd5 (/health/instance) 2020-03-31 09:08:18,334 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:18,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:08:18,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:08:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:08:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:31.440707+00:00 (in 9.999027 seconds) 2020-03-31 09:08:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:31 UTC)" (scheduled at 2020-03-31 09:08:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:08:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:31.440707+00:00 (in 9.999027 seconds) 2020-03-31 09:08:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:31 UTC)" (scheduled at 2020-03-31 09:08:21.440707+00:00) 2020-03-31 09:08:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:08:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:08:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:08:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:08:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:08:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:31 UTC)" executed successfully 2020-03-31 09:08:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:08:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:52.113221+00:00 (in 29.999057 seconds) 2020-03-31 09:08:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:52 UTC)" (scheduled at 2020-03-31 09:08:22.113221+00:00) securityworker stdout | 2020-03-31 09:08:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:52.113221+00:00 (in 29.999057 seconds) 2020-03-31 09:08:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:52 UTC)" (scheduled at 2020-03-31 09:08:22.113221+00:00) 2020-03-31 09:08:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:08:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:08:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:08:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:08:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:08:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:08:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:08:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:08:52 UTC)" executed successfully 2020-03-31 09:08:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:08:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:34.323352+00:00 (in 9.995291 seconds) 2020-03-31 09:08:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:34 UTC)" (scheduled at 2020-03-31 09:08:24.323352+00:00) 2020-03-31 09:08:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:08:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:34.323352+00:00 (in 9.995291 seconds) 2020-03-31 09:08:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:34 UTC)" (scheduled at 2020-03-31 09:08:24.323352+00:00) 2020-03-31 09:08:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 24, 328829), True, datetime.datetime(2020, 3, 31, 9, 8, 24, 328829), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:08:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 24, 328829), True, datetime.datetime(2020, 3, 31, 9, 8, 24, 328829), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:08:24,349 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:08:24,349 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:24,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:24,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:08:24,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:24,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:34 UTC)" executed successfully 2020-03-31 09:08:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:08:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:33.330112+00:00 (in 8.918320 seconds) 2020-03-31 09:08:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:24 UTC)" (scheduled at 2020-03-31 09:08:24.410927+00:00) 2020-03-31 09:08:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:08:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:33.330112+00:00 (in 8.918320 seconds) 2020-03-31 09:08:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:24 UTC)" (scheduled at 2020-03-31 09:08:24.410927+00:00) 2020-03-31 09:08:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:08:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:24 UTC)" executed successfully 2020-03-31 09:08:24,445 [518] [DEBUG] [app] Starting request: urn:request:9dae2d50-aaa8-45da-802e-a418ddb4be90 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:24,445 [518] [DEBUG] [app] Starting request: urn:request:9dae2d50-aaa8-45da-802e-a418ddb4be90 (/health/instance) 2020-03-31 09:08:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:24,453 [542] [DEBUG] [app] Starting request: urn:request:06dda3ce-0074-456c-bc3c-1cf0512d5cb5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:24,453 [542] [DEBUG] [app] Starting request: urn:request:06dda3ce-0074-456c-bc3c-1cf0512d5cb5 (/v1/_internal_ping) 2020-03-31 09:08:24,455 [542] [DEBUG] [app] Ending request: urn:request:06dda3ce-0074-456c-bc3c-1cf0512d5cb5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:24,455 [542] [DEBUG] [app] Ending request: urn:request:06dda3ce-0074-456c-bc3c-1cf0512d5cb5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:24,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:08:24,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:24,458 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:08:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:08:24,458 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:24,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:24,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:24,460 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:24,460 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:24,488 [525] [DEBUG] [app] Starting request: urn:request:1c22d1bc-637b-4d3d-848c-a4475caef03e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:24,488 [525] [DEBUG] [app] Starting request: urn:request:1c22d1bc-637b-4d3d-848c-a4475caef03e (/_internal_ping) 2020-03-31 09:08:24,489 [525] [DEBUG] [app] Ending request: urn:request:1c22d1bc-637b-4d3d-848c-a4475caef03e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:24,489 [525] [DEBUG] [app] Ending request: urn:request:1c22d1bc-637b-4d3d-848c-a4475caef03e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:08:24,489 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:24,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:24,489 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:24,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:08:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:08:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:08:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:24,501 [543] [DEBUG] [app] Starting request: urn:request:3525b44d-76a8-46c4-a920-2480f3042d9a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:24,501 [543] [DEBUG] [app] Starting request: urn:request:3525b44d-76a8-46c4-a920-2480f3042d9a (/c1/_internal_ping) 2020-03-31 09:08:24,502 [543] [DEBUG] [app] Ending request: urn:request:3525b44d-76a8-46c4-a920-2480f3042d9a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:24,502 [543] [DEBUG] [app] Ending request: urn:request:3525b44d-76a8-46c4-a920-2480f3042d9a (/c1/_internal_ping) 2020-03-31 09:08:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:24,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:24,505 [518] [DEBUG] [app] Ending request: urn:request:9dae2d50-aaa8-45da-802e-a418ddb4be90 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:08:24,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:24,505 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:08:24,505 [518] [DEBUG] [app] Ending request: urn:request:9dae2d50-aaa8-45da-802e-a418ddb4be90 (/health/instance) 2020-03-31 09:08:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:08:24,505 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:08:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:08:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:36.924497+00:00 (in 9.999237 seconds) 2020-03-31 09:08:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:36 UTC)" (scheduled at 2020-03-31 09:08:26.924497+00:00) 2020-03-31 09:08:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:08:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:36.924497+00:00 (in 9.999237 seconds) 2020-03-31 09:08:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:36 UTC)" (scheduled at 2020-03-31 09:08:26.924497+00:00) 2020-03-31 09:08:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 26, 926131), True, datetime.datetime(2020, 3, 31, 9, 8, 26, 926131), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:08:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 26, 926131), True, datetime.datetime(2020, 3, 31, 9, 8, 26, 926131), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:08:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:08:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:08:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:36 UTC)" executed successfully 2020-03-31 09:08:28,273 [518] [DEBUG] [app] Starting request: urn:request:060f6620-afd4-467a-9de1-d51235b03677 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:28,273 [518] [DEBUG] [app] Starting request: urn:request:060f6620-afd4-467a-9de1-d51235b03677 (/health/instance) 2020-03-31 09:08:28,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:28,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:28,279 [538] [DEBUG] [app] Starting request: urn:request:b6104693-566d-4664-8554-643f3d6489d7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:28,279 [538] [DEBUG] [app] Starting request: urn:request:b6104693-566d-4664-8554-643f3d6489d7 (/v1/_internal_ping) 2020-03-31 09:08:28,280 [538] [DEBUG] [app] Ending request: urn:request:b6104693-566d-4664-8554-643f3d6489d7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:28,280 [538] [DEBUG] [app] Ending request: urn:request:b6104693-566d-4664-8554-643f3d6489d7 (/v1/_internal_ping) 2020-03-31 09:08:28,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:28,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:08:28,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:28,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:28,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:08:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:08:28,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:08:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:28,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:28,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:28,315 [518] [DEBUG] [app] Starting request: urn:request:8c570a65-25be-4ccd-8961-70cf926a9b71 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:28,315 [518] [DEBUG] [app] Starting request: urn:request:8c570a65-25be-4ccd-8961-70cf926a9b71 (/_internal_ping) 2020-03-31 09:08:28,316 [518] [DEBUG] [app] Ending request: urn:request:8c570a65-25be-4ccd-8961-70cf926a9b71 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:28,316 [518] [DEBUG] [app] Ending request: urn:request:8c570a65-25be-4ccd-8961-70cf926a9b71 (/_internal_ping) 2020-03-31 09:08:28,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:08:28,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:08:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:08:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:08:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:28,331 [543] [DEBUG] [app] Starting request: urn:request:0f07e03e-6648-40b1-8195-6eafa76f6aae (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:28,331 [543] [DEBUG] [app] Starting request: urn:request:0f07e03e-6648-40b1-8195-6eafa76f6aae (/c1/_internal_ping) 2020-03-31 09:08:28,333 [543] [DEBUG] [app] Ending request: urn:request:0f07e03e-6648-40b1-8195-6eafa76f6aae (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:28,333 [543] [DEBUG] [app] Ending request: urn:request:0f07e03e-6648-40b1-8195-6eafa76f6aae (/c1/_internal_ping) 2020-03-31 09:08:28,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:28,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:28,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:28,335 [518] [DEBUG] [app] Ending request: urn:request:060f6620-afd4-467a-9de1-d51235b03677 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:08:28,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:28,335 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:08:28,335 [518] [DEBUG] [app] Ending request: urn:request:060f6620-afd4-467a-9de1-d51235b03677 (/health/instance) 2020-03-31 09:08:28,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:08:28,335 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:08:28,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:08:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:08:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:00.337699+00:00 (in 29.998994 seconds) 2020-03-31 09:08:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:00 UTC)" (scheduled at 2020-03-31 09:08:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:08:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:00.337699+00:00 (in 29.998994 seconds) 2020-03-31 09:08:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:00 UTC)" (scheduled at 2020-03-31 09:08:30.337699+00:00) 2020-03-31 09:08:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 8, 30, 339394), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:08:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 8, 30, 339394), False, 50, 1, 0]) 2020-03-31 09:08:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:08:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:08:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:08:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:00 UTC)" executed successfully 2020-03-31 09:08:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:08:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:41.440707+00:00 (in 9.999092 seconds) 2020-03-31 09:08:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:41 UTC)" (scheduled at 2020-03-31 09:08:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:08:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:41.440707+00:00 (in 9.999092 seconds) 2020-03-31 09:08:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:41 UTC)" (scheduled at 2020-03-31 09:08:31.440707+00:00) 2020-03-31 09:08:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:08:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:08:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:08:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:08:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:08:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:41 UTC)" executed successfully 2020-03-31 09:08:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:08:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:03.330112+00:00 (in 29.998856 seconds) 2020-03-31 09:08:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:03 UTC)" (scheduled at 2020-03-31 09:08:33.330112+00:00) 2020-03-31 09:08:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:08:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:03.330112+00:00 (in 29.998856 seconds) 2020-03-31 09:08:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:03 UTC)" (scheduled at 2020-03-31 09:08:33.330112+00:00) 2020-03-31 09:08:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 33, 332261), True, datetime.datetime(2020, 3, 31, 9, 8, 33, 332261), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:08:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 33, 332261), True, datetime.datetime(2020, 3, 31, 9, 8, 33, 332261), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:08:33,348 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:08:33,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:08:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:03 UTC)" executed successfully 2020-03-31 09:08:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:08:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:43.539999+00:00 (in 10.171738 seconds) 2020-03-31 09:08:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:33 UTC)" (scheduled at 2020-03-31 09:08:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:08:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:43.539999+00:00 (in 10.171738 seconds) 2020-03-31 09:08:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:33 UTC)" (scheduled at 2020-03-31 09:08:33.366791+00:00) 2020-03-31 09:08:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:08:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 33, 369236), True, datetime.datetime(2020, 3, 31, 9, 8, 33, 369236), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:08:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 33, 369236), True, datetime.datetime(2020, 3, 31, 9, 8, 33, 369236), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:08:33,387 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:08:33,387 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:08:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:33 UTC)" executed successfully 2020-03-31 09:08:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:08:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:44.323352+00:00 (in 9.999241 seconds) 2020-03-31 09:08:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:44 UTC)" (scheduled at 2020-03-31 09:08:34.323352+00:00) 2020-03-31 09:08:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:08:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:44.323352+00:00 (in 9.999241 seconds) 2020-03-31 09:08:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:44 UTC)" (scheduled at 2020-03-31 09:08:34.323352+00:00) 2020-03-31 09:08:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 34, 324881), True, datetime.datetime(2020, 3, 31, 9, 8, 34, 324881), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:08:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 34, 324881), True, datetime.datetime(2020, 3, 31, 9, 8, 34, 324881), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:08:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:08:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:08:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:44 UTC)" executed successfully 2020-03-31 09:08:34,446 [527] [DEBUG] [app] Starting request: urn:request:e9871e7f-2a80-4ab2-8462-e58fa6b56212 (/health/instance) 2020-03-31 09:08:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:34,446 [527] [DEBUG] [app] Starting request: urn:request:e9871e7f-2a80-4ab2-8462-e58fa6b56212 (/health/instance) 2020-03-31 09:08:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:34,454 [553] [DEBUG] [app] Starting request: urn:request:21966bab-0404-417c-aac7-8b6fa64c2aa1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:34,454 [553] [DEBUG] [app] Starting request: urn:request:21966bab-0404-417c-aac7-8b6fa64c2aa1 (/v1/_internal_ping) 2020-03-31 09:08:34,457 [553] [DEBUG] [app] Ending request: urn:request:21966bab-0404-417c-aac7-8b6fa64c2aa1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:34,457 [553] [DEBUG] [app] Ending request: urn:request:21966bab-0404-417c-aac7-8b6fa64c2aa1 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:08:34,458 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:08:34,458 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:34,460 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:34,460 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:34,461 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:08:34,461 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:34,461 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:34,461 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:08:34,462 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:34,462 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:34,463 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:34,463 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:34,474 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:34,474 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:34,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:34,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:34,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:34,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:34,491 [518] [DEBUG] [app] Starting request: urn:request:ea2b7886-7c66-49ee-a65e-b923deba09f1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:34,491 [518] [DEBUG] [app] Starting request: urn:request:ea2b7886-7c66-49ee-a65e-b923deba09f1 (/_internal_ping) 2020-03-31 09:08:34,492 [518] [DEBUG] [app] Ending request: urn:request:ea2b7886-7c66-49ee-a65e-b923deba09f1 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:34,492 [518] [DEBUG] [app] Ending request: urn:request:ea2b7886-7c66-49ee-a65e-b923deba09f1 (/_internal_ping) 2020-03-31 09:08:34,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:34,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:34,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:34,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:08:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:08:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:34,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:34,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:34,508 [543] [DEBUG] [app] Starting request: urn:request:b92fd0e5-1cc5-40e0-b3e0-0af8e13d7cee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:34,508 [543] [DEBUG] [app] Starting request: urn:request:b92fd0e5-1cc5-40e0-b3e0-0af8e13d7cee (/c1/_internal_ping) 2020-03-31 09:08:34,509 [543] [DEBUG] [app] Ending request: urn:request:b92fd0e5-1cc5-40e0-b3e0-0af8e13d7cee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:34,509 [543] [DEBUG] [app] Ending request: urn:request:b92fd0e5-1cc5-40e0-b3e0-0af8e13d7cee (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:34,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:34,511 [527] [DEBUG] [app] Ending request: urn:request:e9871e7f-2a80-4ab2-8462-e58fa6b56212 (/health/instance) 2020-03-31 09:08:34,511 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:34,512 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:08:34,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:34,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:34,511 [527] [DEBUG] [app] Ending request: urn:request:e9871e7f-2a80-4ab2-8462-e58fa6b56212 (/health/instance) 2020-03-31 09:08:34,511 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:34,512 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-verbs stdout | 2020-03-31 09:08:34,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:36 UTC)" (scheduled at 2020-03-31 09:08:36.924497+00:00) 2020-03-31 09:08:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 36, 926321), True, datetime.datetime(2020, 3, 31, 9, 8, 36, 926321), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:08:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:38.010934+00:00 (in 1.085507 seconds) storagereplication stdout | 2020-03-31 09:08:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:36 UTC)" (scheduled at 2020-03-31 09:08:36.924497+00:00) 2020-03-31 09:08:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 36, 926321), True, datetime.datetime(2020, 3, 31, 9, 8, 36, 926321), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:08:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:38.010934+00:00 (in 1.085507 seconds) 2020-03-31 09:08:37,126 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:08:37,126 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:37,126 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:37,126 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:08:37,126 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:37,126 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:46 UTC)" executed successfully 2020-03-31 09:08:38,014 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:08:38,014 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:38,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:46.924497+00:00 (in 8.909683 seconds) 2020-03-31 09:08:38,015 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:38 UTC)" (scheduled at 2020-03-31 09:08:38.010934+00:00) 2020-03-31 09:08:38,015 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:38,015 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:08:38,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:46.924497+00:00 (in 8.909683 seconds) 2020-03-31 09:08:38,015 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:38 UTC)" (scheduled at 2020-03-31 09:08:38.010934+00:00) 2020-03-31 09:08:38,015 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:38,015 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:38 UTC)" executed successfully 2020-03-31 09:08:38,273 [518] [DEBUG] [app] Starting request: urn:request:7968440d-6ac0-4add-bdc0-76de881cb7ca (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:38,273 [518] [DEBUG] [app] Starting request: urn:request:7968440d-6ac0-4add-bdc0-76de881cb7ca (/health/instance) 2020-03-31 09:08:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:38,280 [538] [DEBUG] [app] Starting request: urn:request:c00527f1-bb62-4e11-ad1b-a5bf1af2b915 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:38,280 [538] [DEBUG] [app] Starting request: urn:request:c00527f1-bb62-4e11-ad1b-a5bf1af2b915 (/v1/_internal_ping) 2020-03-31 09:08:38,281 [538] [DEBUG] [app] Ending request: urn:request:c00527f1-bb62-4e11-ad1b-a5bf1af2b915 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:38,281 [538] [DEBUG] [app] Ending request: urn:request:c00527f1-bb62-4e11-ad1b-a5bf1af2b915 (/v1/_internal_ping) 2020-03-31 09:08:38,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:08:38,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:08:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:08:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:08:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:38,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:38,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:38,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:38,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:38,311 [518] [DEBUG] [app] Starting request: urn:request:1944fb7e-688a-4fac-89e7-9a4251e8ec08 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:38,311 [518] [DEBUG] [app] Starting request: urn:request:1944fb7e-688a-4fac-89e7-9a4251e8ec08 (/_internal_ping) 2020-03-31 09:08:38,312 [518] [DEBUG] [app] Ending request: urn:request:1944fb7e-688a-4fac-89e7-9a4251e8ec08 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:38,312 [518] [DEBUG] [app] Ending request: urn:request:1944fb7e-688a-4fac-89e7-9a4251e8ec08 (/_internal_ping) 2020-03-31 09:08:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:08:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:08:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:08:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:38,324 [543] [DEBUG] [app] Starting request: urn:request:856b1ddc-738d-4e1b-8c1b-3034b7691776 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:38,324 [543] [DEBUG] [app] Starting request: urn:request:856b1ddc-738d-4e1b-8c1b-3034b7691776 (/c1/_internal_ping) 2020-03-31 09:08:38,325 [543] [DEBUG] [app] Ending request: urn:request:856b1ddc-738d-4e1b-8c1b-3034b7691776 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:38,325 [543] [DEBUG] [app] Ending request: urn:request:856b1ddc-738d-4e1b-8c1b-3034b7691776 (/c1/_internal_ping) 2020-03-31 09:08:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:08:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:38,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:38,327 [518] [DEBUG] [app] Ending request: urn:request:7968440d-6ac0-4add-bdc0-76de881cb7ca (/health/instance) 2020-03-31 09:08:38,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:08:38,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:08:38,327 [518] [DEBUG] [app] Ending request: urn:request:7968440d-6ac0-4add-bdc0-76de881cb7ca (/health/instance) 2020-03-31 09:08:38,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:08:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:08:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:51.440707+00:00 (in 9.998885 seconds) 2020-03-31 09:08:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:51 UTC)" (scheduled at 2020-03-31 09:08:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:08:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:51.440707+00:00 (in 9.998885 seconds) 2020-03-31 09:08:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:51 UTC)" (scheduled at 2020-03-31 09:08:41.440707+00:00) 2020-03-31 09:08:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:08:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:08:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:08:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:08:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:08:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:51 UTC)" executed successfully 2020-03-31 09:08:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:08:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:33.366791+00:00 (in 49.825639 seconds) 2020-03-31 09:08:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:43 UTC)" (scheduled at 2020-03-31 09:08:43.539999+00:00) 2020-03-31 09:08:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:08:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:33.366791+00:00 (in 49.825639 seconds) 2020-03-31 09:08:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:43 UTC)" (scheduled at 2020-03-31 09:08:43.539999+00:00) 2020-03-31 09:08:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:43 UTC)" executed successfully 2020-03-31 09:08:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:08:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:49.498966+00:00 (in 5.174803 seconds) 2020-03-31 09:08:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:54 UTC)" (scheduled at 2020-03-31 09:08:44.323352+00:00) 2020-03-31 09:08:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:08:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:49.498966+00:00 (in 5.174803 seconds) 2020-03-31 09:08:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:54 UTC)" (scheduled at 2020-03-31 09:08:44.323352+00:00) 2020-03-31 09:08:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 44, 324939), True, datetime.datetime(2020, 3, 31, 9, 8, 44, 324939), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:08:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 44, 324939), True, datetime.datetime(2020, 3, 31, 9, 8, 44, 324939), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:08:44,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:08:44,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:08:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:54 UTC)" executed successfully 2020-03-31 09:08:44,446 [526] [DEBUG] [app] Starting request: urn:request:6208bb84-3435-4467-a802-218c331a6852 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:44,446 [526] [DEBUG] [app] Starting request: urn:request:6208bb84-3435-4467-a802-218c331a6852 (/health/instance) 2020-03-31 09:08:44,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:44,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:44,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:44,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:44,453 [551] [DEBUG] [app] Starting request: urn:request:b73f0b42-f876-4fd8-a51a-4697e408b511 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:44,453 [551] [DEBUG] [app] Starting request: urn:request:b73f0b42-f876-4fd8-a51a-4697e408b511 (/v1/_internal_ping) 2020-03-31 09:08:44,454 [551] [DEBUG] [app] Ending request: urn:request:b73f0b42-f876-4fd8-a51a-4697e408b511 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:44,454 [551] [DEBUG] [app] Ending request: urn:request:b73f0b42-f876-4fd8-a51a-4697e408b511 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:44,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:08:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:44,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:44,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:44,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:08:44,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:08:44,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:44,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:44,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:44,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:44,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:44,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:44,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:44,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:44,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:44,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:44,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:44,487 [521] [DEBUG] [app] Starting request: urn:request:65f07e1f-bf76-43f8-819a-3c8acf4bac53 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:44,487 [521] [DEBUG] [app] Starting request: urn:request:65f07e1f-bf76-43f8-819a-3c8acf4bac53 (/_internal_ping) 2020-03-31 09:08:44,488 [521] [DEBUG] [app] Ending request: urn:request:65f07e1f-bf76-43f8-819a-3c8acf4bac53 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:44,488 [521] [DEBUG] [app] Ending request: urn:request:65f07e1f-bf76-43f8-819a-3c8acf4bac53 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:08:44,488 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:44,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:44,488 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:44,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:44,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:44,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:44,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:08:44,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:08:44,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:44,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:44,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:44,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:44,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:44,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:44,501 [536] [DEBUG] [app] Starting request: urn:request:8434e87a-5b18-4651-a6d5-8f1545c02056 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:44,501 [536] [DEBUG] [app] Starting request: urn:request:8434e87a-5b18-4651-a6d5-8f1545c02056 (/c1/_internal_ping) 2020-03-31 09:08:44,502 [536] [DEBUG] [app] Ending request: urn:request:8434e87a-5b18-4651-a6d5-8f1545c02056 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:44,502 [536] [DEBUG] [app] Ending request: urn:request:8434e87a-5b18-4651-a6d5-8f1545c02056 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:08:44,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:44,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:44,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:08:44,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:44,504 [526] [DEBUG] [app] Ending request: urn:request:6208bb84-3435-4467-a802-218c331a6852 (/health/instance) 2020-03-31 09:08:44,504 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:44,504 [526] [DEBUG] [app] Ending request: urn:request:6208bb84-3435-4467-a802-218c331a6852 (/health/instance) 2020-03-31 09:08:44,504 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:44,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:08:44,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:08:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:08:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:15.064514+00:00 (in 29.998820 seconds) 2020-03-31 09:08:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:15 UTC)" (scheduled at 2020-03-31 09:08:45.064514+00:00) gcworker stdout | 2020-03-31 09:08:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:15.064514+00:00 (in 29.998820 seconds) 2020-03-31 09:08:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:15 UTC)" (scheduled at 2020-03-31 09:08:45.064514+00:00) 2020-03-31 09:08:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436125, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:08:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436125, 1209600, 500, 1, 0]) 2020-03-31 09:08:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:08:45,084 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:08:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:08:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:08:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:15 UTC)" executed successfully 2020-03-31 09:08:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:08:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:56.924497+00:00 (in 9.998759 seconds) 2020-03-31 09:08:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:56 UTC)" (scheduled at 2020-03-31 09:08:46.924497+00:00) 2020-03-31 09:08:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:08:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:56.924497+00:00 (in 9.998759 seconds) 2020-03-31 09:08:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:56 UTC)" (scheduled at 2020-03-31 09:08:46.924497+00:00) 2020-03-31 09:08:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 46, 926590), True, datetime.datetime(2020, 3, 31, 9, 8, 46, 926590), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:08:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 46, 926590), True, datetime.datetime(2020, 3, 31, 9, 8, 46, 926590), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:08:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:08:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:08:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:08:56 UTC)" executed successfully 2020-03-31 09:08:48,272 [527] [DEBUG] [app] Starting request: urn:request:0527510e-0b60-4099-863b-883347cb2436 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:48,272 [527] [DEBUG] [app] Starting request: urn:request:0527510e-0b60-4099-863b-883347cb2436 (/health/instance) 2020-03-31 09:08:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:48,280 [555] [DEBUG] [app] Starting request: urn:request:31bba016-5629-45c1-ab8a-b3162d8c45b0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:48,280 [555] [DEBUG] [app] Starting request: urn:request:31bba016-5629-45c1-ab8a-b3162d8c45b0 (/v1/_internal_ping) 2020-03-31 09:08:48,281 [555] [DEBUG] [app] Ending request: urn:request:31bba016-5629-45c1-ab8a-b3162d8c45b0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:48,281 [555] [DEBUG] [app] Ending request: urn:request:31bba016-5629-45c1-ab8a-b3162d8c45b0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:48,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:08:48,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:08:48,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:08:48,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:48,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:48,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:48,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:48,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:48,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:48,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:48,311 [527] [DEBUG] [app] Starting request: urn:request:8647799d-da64-4613-a0fa-ef314921f8c0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:48,311 [527] [DEBUG] [app] Starting request: urn:request:8647799d-da64-4613-a0fa-ef314921f8c0 (/_internal_ping) 2020-03-31 09:08:48,312 [527] [DEBUG] [app] Ending request: urn:request:8647799d-da64-4613-a0fa-ef314921f8c0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:48,312 [527] [DEBUG] [app] Ending request: urn:request:8647799d-da64-4613-a0fa-ef314921f8c0 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:08:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:08:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:08:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:08:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:48,324 [536] [DEBUG] [app] Starting request: urn:request:ccb23a76-a4c9-488c-baba-06e780e15d80 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:48,324 [536] [DEBUG] [app] Starting request: urn:request:ccb23a76-a4c9-488c-baba-06e780e15d80 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:48,325 [536] [DEBUG] [app] Ending request: urn:request:ccb23a76-a4c9-488c-baba-06e780e15d80 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:48,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:08:48,325 [536] [DEBUG] [app] Ending request: urn:request:ccb23a76-a4c9-488c-baba-06e780e15d80 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:48,327 [527] [DEBUG] [app] Ending request: urn:request:0527510e-0b60-4099-863b-883347cb2436 (/health/instance) 2020-03-31 09:08:48,327 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:08:48,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:08:48,327 [527] [DEBUG] [app] Ending request: urn:request:0527510e-0b60-4099-863b-883347cb2436 (/health/instance) 2020-03-31 09:08:48,327 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:08:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:08:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:49 UTC)" (scheduled at 2020-03-31 09:08:49.498966+00:00) 2020-03-31 09:08:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:08:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:49 UTC)" (scheduled at 2020-03-31 09:08:49.498966+00:00) 2020-03-31 09:08:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:49 UTC)" executed successfully 2020-03-31 09:08:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:54.323352+00:00 (in 4.823456 seconds) notificationworker stdout | 2020-03-31 09:08:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:49 UTC)" executed successfully 2020-03-31 09:08:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:54.323352+00:00 (in 4.823456 seconds) 2020-03-31 09:08:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:08:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:59.560326+00:00 (in 8.448940 seconds) 2020-03-31 09:08:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:51 UTC)" (scheduled at 2020-03-31 09:08:51.110610+00:00) 2020-03-31 09:08:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:08:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:08:59.560326+00:00 (in 8.448940 seconds) 2020-03-31 09:08:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:51 UTC)" (scheduled at 2020-03-31 09:08:51.110610+00:00) 2020-03-31 09:08:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 51, 112146), True, datetime.datetime(2020, 3, 31, 9, 8, 51, 112146), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:08:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 51, 112146), True, datetime.datetime(2020, 3, 31, 9, 8, 51, 112146), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:08:51,127 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:08:51,127 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:51,127 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:51,128 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:08:51,127 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:51,128 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:51 UTC)" executed successfully 2020-03-31 09:08:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:08:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:01.440707+00:00 (in 9.999139 seconds) 2020-03-31 09:08:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:01 UTC)" (scheduled at 2020-03-31 09:08:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:08:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:01.440707+00:00 (in 9.999139 seconds) 2020-03-31 09:08:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:01 UTC)" (scheduled at 2020-03-31 09:08:51.440707+00:00) 2020-03-31 09:08:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:08:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:08:51,457 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:08:51,457 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:08:51,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:51,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:08:51,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:51,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:01 UTC)" executed successfully 2020-03-31 09:08:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:08:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:22.113221+00:00 (in 29.997583 seconds) 2020-03-31 09:08:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:22 UTC)" (scheduled at 2020-03-31 09:08:52.113221+00:00) securityworker stdout | 2020-03-31 09:08:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:22.113221+00:00 (in 29.997583 seconds) 2020-03-31 09:08:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:22 UTC)" (scheduled at 2020-03-31 09:08:52.113221+00:00) 2020-03-31 09:08:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:08:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:08:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:08:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:08:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:08:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:08:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:08:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:22 UTC)" executed successfully 2020-03-31 09:08:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:08:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:04 UTC)" (scheduled at 2020-03-31 09:08:54.323352+00:00) 2020-03-31 09:08:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:04.323352+00:00 (in 9.998439 seconds) 2020-03-31 09:08:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:08:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:04 UTC)" (scheduled at 2020-03-31 09:08:54.323352+00:00) 2020-03-31 09:08:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:04.323352+00:00 (in 9.998439 seconds) 2020-03-31 09:08:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:54,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 54, 326138), True, datetime.datetime(2020, 3, 31, 9, 8, 54, 326138), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:08:54,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 54, 326138), True, datetime.datetime(2020, 3, 31, 9, 8, 54, 326138), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:08:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:08:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:08:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:04 UTC)" executed successfully 2020-03-31 09:08:54,446 [527] [DEBUG] [app] Starting request: urn:request:83bf13cd-81a8-430f-a167-7bcfef1141f2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:54,446 [527] [DEBUG] [app] Starting request: urn:request:83bf13cd-81a8-430f-a167-7bcfef1141f2 (/health/instance) 2020-03-31 09:08:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:54,453 [544] [DEBUG] [app] Starting request: urn:request:ef5b2800-c431-4b84-ac38-304a52a1d282 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:54,453 [544] [DEBUG] [app] Starting request: urn:request:ef5b2800-c431-4b84-ac38-304a52a1d282 (/v1/_internal_ping) 2020-03-31 09:08:54,455 [544] [DEBUG] [app] Ending request: urn:request:ef5b2800-c431-4b84-ac38-304a52a1d282 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:54,455 [544] [DEBUG] [app] Ending request: urn:request:ef5b2800-c431-4b84-ac38-304a52a1d282 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:54,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:08:54,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:08:54,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:08:54,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:54,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:54,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:54,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:54,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:54,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:54,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:54,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:54,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:54,484 [527] [DEBUG] [app] Starting request: urn:request:980fb14c-71f5-47a6-8749-52ce553b81c6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:54,484 [527] [DEBUG] [app] Starting request: urn:request:980fb14c-71f5-47a6-8749-52ce553b81c6 (/_internal_ping) 2020-03-31 09:08:54,485 [527] [DEBUG] [app] Ending request: urn:request:980fb14c-71f5-47a6-8749-52ce553b81c6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:54,485 [527] [DEBUG] [app] Ending request: urn:request:980fb14c-71f5-47a6-8749-52ce553b81c6 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:08:54,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:54,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:08:54,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:54,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:08:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:54,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:54,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:08:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:08:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:54,498 [536] [DEBUG] [app] Starting request: urn:request:e549b1d3-b624-4838-9344-8d8ed615efb1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:54,498 [536] [DEBUG] [app] Starting request: urn:request:e549b1d3-b624-4838-9344-8d8ed615efb1 (/c1/_internal_ping) 2020-03-31 09:08:54,499 [536] [DEBUG] [app] Ending request: urn:request:e549b1d3-b624-4838-9344-8d8ed615efb1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:54,499 [536] [DEBUG] [app] Ending request: urn:request:e549b1d3-b624-4838-9344-8d8ed615efb1 (/c1/_internal_ping) 2020-03-31 09:08:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:54,500 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:08:54,500 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:54,501 [527] [DEBUG] [app] Ending request: urn:request:83bf13cd-81a8-430f-a167-7bcfef1141f2 (/health/instance) 2020-03-31 09:08:54,501 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:08:54,501 [527] [DEBUG] [app] Ending request: urn:request:83bf13cd-81a8-430f-a167-7bcfef1141f2 (/health/instance) 2020-03-31 09:08:54,501 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:54,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:08:54,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:08:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:08:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:06.924497+00:00 (in 9.999104 seconds) 2020-03-31 09:08:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:06 UTC)" (scheduled at 2020-03-31 09:08:56.924497+00:00) 2020-03-31 09:08:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:08:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 56, 926254), True, datetime.datetime(2020, 3, 31, 9, 8, 56, 926254), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:08:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:06.924497+00:00 (in 9.999104 seconds) 2020-03-31 09:08:56,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:06 UTC)" (scheduled at 2020-03-31 09:08:56.924497+00:00) 2020-03-31 09:08:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:08:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 8, 56, 926254), True, datetime.datetime(2020, 3, 31, 9, 8, 56, 926254), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:08:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:08:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:08:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:08:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:06 UTC)" executed successfully 2020-03-31 09:08:58,272 [526] [DEBUG] [app] Starting request: urn:request:9427e792-0ea8-4f5f-8e12-34f73b3c09de (/health/instance) gunicorn-web stdout | 2020-03-31 09:08:58,272 [526] [DEBUG] [app] Starting request: urn:request:9427e792-0ea8-4f5f-8e12-34f73b3c09de (/health/instance) 2020-03-31 09:08:58,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:58,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:58,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:58,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:58,280 [551] [DEBUG] [app] Starting request: urn:request:a1cea754-7699-4471-9d1d-3c650c9e3f6f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:58,280 [551] [DEBUG] [app] Starting request: urn:request:a1cea754-7699-4471-9d1d-3c650c9e3f6f (/v1/_internal_ping) 2020-03-31 09:08:58,281 [551] [DEBUG] [app] Ending request: urn:request:a1cea754-7699-4471-9d1d-3c650c9e3f6f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:08:58,281 [551] [DEBUG] [app] Ending request: urn:request:a1cea754-7699-4471-9d1d-3c650c9e3f6f (/v1/_internal_ping) 2020-03-31 09:08:58,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:58,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:08:58,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:58,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:08:58,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:08:58,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:08:58,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:08:58,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:08:58,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:08:58,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:08:58,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:08:58,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:08:58,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:08:58,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:08:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:08:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:08:58,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:58,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:58,313 [526] [DEBUG] [app] Starting request: urn:request:aee3e107-bb84-4076-8cf3-77b118b23a2d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:58,313 [526] [DEBUG] [app] Starting request: urn:request:aee3e107-bb84-4076-8cf3-77b118b23a2d (/_internal_ping) 2020-03-31 09:08:58,314 [526] [DEBUG] [app] Ending request: urn:request:aee3e107-bb84-4076-8cf3-77b118b23a2d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:08:58,314 [526] [DEBUG] [app] Ending request: urn:request:aee3e107-bb84-4076-8cf3-77b118b23a2d (/_internal_ping) 2020-03-31 09:08:58,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:58,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:08:58,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:08:58,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:08:58,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:58,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:58,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:08:58,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:08:58,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:58,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:58,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:08:58,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:08:58,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:08:58,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:08:58,326 [536] [DEBUG] [app] Starting request: urn:request:d99d6c2b-92f7-4c70-8b29-e4ffd4d02d41 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:58,326 [536] [DEBUG] [app] Starting request: urn:request:d99d6c2b-92f7-4c70-8b29-e4ffd4d02d41 (/c1/_internal_ping) 2020-03-31 09:08:58,327 [536] [DEBUG] [app] Ending request: urn:request:d99d6c2b-92f7-4c70-8b29-e4ffd4d02d41 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:08:58,327 [536] [DEBUG] [app] Ending request: urn:request:d99d6c2b-92f7-4c70-8b29-e4ffd4d02d41 (/c1/_internal_ping) 2020-03-31 09:08:58,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:58,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:08:58,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:08:58,330 [526] [DEBUG] [app] Ending request: urn:request:9427e792-0ea8-4f5f-8e12-34f73b3c09de (/health/instance) 2020-03-31 09:08:58,330 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:08:58,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:08:58,330 [526] [DEBUG] [app] Ending request: urn:request:9427e792-0ea8-4f5f-8e12-34f73b3c09de (/health/instance) 2020-03-31 09:08:58,330 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:08:58,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:08:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:08:58,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:08:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:08:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:08:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:08:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:51.110610+00:00 (in 51.549281 seconds) 2020-03-31 09:08:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:59 UTC)" (scheduled at 2020-03-31 09:08:59.560326+00:00) 2020-03-31 09:08:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:08:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:51.110610+00:00 (in 51.549281 seconds) 2020-03-31 09:08:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:59 UTC)" (scheduled at 2020-03-31 09:08:59.560326+00:00) 2020-03-31 09:08:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:08:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:59 UTC)" executed successfully 2020-03-31 09:09:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:09:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:30.337699+00:00 (in 29.999281 seconds) 2020-03-31 09:09:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:30 UTC)" (scheduled at 2020-03-31 09:09:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:09:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:30.337699+00:00 (in 29.999281 seconds) 2020-03-31 09:09:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:30 UTC)" (scheduled at 2020-03-31 09:09:00.337699+00:00) 2020-03-31 09:09:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 9, 0, 339134), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:09:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 9, 0, 339134), False, 50, 1, 0]) 2020-03-31 09:09:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:09:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:09:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:09:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:30 UTC)" executed successfully 2020-03-31 09:09:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:09:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:11.440707+00:00 (in 9.998970 seconds) 2020-03-31 09:09:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:11 UTC)" (scheduled at 2020-03-31 09:09:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:09:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:11.440707+00:00 (in 9.998970 seconds) 2020-03-31 09:09:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:11 UTC)" (scheduled at 2020-03-31 09:09:01.440707+00:00) 2020-03-31 09:09:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:09:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:09:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:09:01,459 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:09:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:09:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:09:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:11 UTC)" executed successfully 2020-03-31 09:09:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:09:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:23.410371+00:00 (in 20.079367 seconds) 2020-03-31 09:09:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:33 UTC)" (scheduled at 2020-03-31 09:09:03.330112+00:00) 2020-03-31 09:09:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:09:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:23.410371+00:00 (in 20.079367 seconds) 2020-03-31 09:09:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:33 UTC)" (scheduled at 2020-03-31 09:09:03.330112+00:00) 2020-03-31 09:09:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 3, 331888), True, datetime.datetime(2020, 3, 31, 9, 9, 3, 331888), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:09:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 3, 331888), True, datetime.datetime(2020, 3, 31, 9, 9, 3, 331888), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:09:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:09:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:09:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:33 UTC)" executed successfully 2020-03-31 09:09:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:09:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:14.323352+00:00 (in 9.998464 seconds) 2020-03-31 09:09:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:14 UTC)" (scheduled at 2020-03-31 09:09:04.323352+00:00) 2020-03-31 09:09:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:09:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:14.323352+00:00 (in 9.998464 seconds) 2020-03-31 09:09:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:14 UTC)" (scheduled at 2020-03-31 09:09:04.323352+00:00) 2020-03-31 09:09:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 4, 325944), True, datetime.datetime(2020, 3, 31, 9, 9, 4, 325944), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:09:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 4, 325944), True, datetime.datetime(2020, 3, 31, 9, 9, 4, 325944), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:09:04,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:04,346 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:09:04,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:04,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:04,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:09:04,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:14 UTC)" executed successfully 2020-03-31 09:09:04,445 [527] [DEBUG] [app] Starting request: urn:request:8aafe831-4c41-4695-a606-5cc5acbfa803 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:04,445 [527] [DEBUG] [app] Starting request: urn:request:8aafe831-4c41-4695-a606-5cc5acbfa803 (/health/instance) 2020-03-31 09:09:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:04,453 [552] [DEBUG] [app] Starting request: urn:request:8ee566b5-9f8d-48c8-a526-ccf0505e5a83 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:04,453 [552] [DEBUG] [app] Starting request: urn:request:8ee566b5-9f8d-48c8-a526-ccf0505e5a83 (/v1/_internal_ping) 2020-03-31 09:09:04,454 [552] [DEBUG] [app] Ending request: urn:request:8ee566b5-9f8d-48c8-a526-ccf0505e5a83 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:04,454 [552] [DEBUG] [app] Ending request: urn:request:8ee566b5-9f8d-48c8-a526-ccf0505e5a83 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:09:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:09:04,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:09:04,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:04,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:04,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:04,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:04,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:04,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:04,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:04,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:04,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:04,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:04,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:04,490 [527] [DEBUG] [app] Starting request: urn:request:8fe7d239-640b-47c4-9f6f-71a5109b086c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:04,490 [527] [DEBUG] [app] Starting request: urn:request:8fe7d239-640b-47c4-9f6f-71a5109b086c (/_internal_ping) 2020-03-31 09:09:04,491 [527] [DEBUG] [app] Ending request: urn:request:8fe7d239-640b-47c4-9f6f-71a5109b086c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:04,491 [527] [DEBUG] [app] Ending request: urn:request:8fe7d239-640b-47c4-9f6f-71a5109b086c (/_internal_ping) 2020-03-31 09:09:04,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:09:04,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:04,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:09:04,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:04,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:04,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:04,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:04,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:09:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:04,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:04,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:04,503 [541] [DEBUG] [app] Starting request: urn:request:1fdc8b14-dc88-4684-9f12-a1d960df41f2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:04,503 [541] [DEBUG] [app] Starting request: urn:request:1fdc8b14-dc88-4684-9f12-a1d960df41f2 (/c1/_internal_ping) 2020-03-31 09:09:04,504 [541] [DEBUG] [app] Ending request: urn:request:1fdc8b14-dc88-4684-9f12-a1d960df41f2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:04,504 [541] [DEBUG] [app] Ending request: urn:request:1fdc8b14-dc88-4684-9f12-a1d960df41f2 (/c1/_internal_ping) 2020-03-31 09:09:04,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:04,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:04,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:04,507 [527] [DEBUG] [app] Ending request: urn:request:8aafe831-4c41-4695-a606-5cc5acbfa803 (/health/instance) 2020-03-31 09:09:04,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:09:04,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:04,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:09:04,507 [527] [DEBUG] [app] Ending request: urn:request:8aafe831-4c41-4695-a606-5cc5acbfa803 (/health/instance) 2020-03-31 09:09:04,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:04,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:09:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:09:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:16.924497+00:00 (in 9.998997 seconds) 2020-03-31 09:09:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:16 UTC)" (scheduled at 2020-03-31 09:09:06.924497+00:00) 2020-03-31 09:09:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:09:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:16.924497+00:00 (in 9.998997 seconds) 2020-03-31 09:09:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:16 UTC)" (scheduled at 2020-03-31 09:09:06.924497+00:00) 2020-03-31 09:09:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 6, 926206), True, datetime.datetime(2020, 3, 31, 9, 9, 6, 926206), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:09:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 6, 926206), True, datetime.datetime(2020, 3, 31, 9, 9, 6, 926206), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:09:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:09:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:09:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:16 UTC)" executed successfully 2020-03-31 09:09:08,272 [518] [DEBUG] [app] Starting request: urn:request:234a4c15-8dd5-4a11-9b48-a3758150e243 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:08,272 [518] [DEBUG] [app] Starting request: urn:request:234a4c15-8dd5-4a11-9b48-a3758150e243 (/health/instance) 2020-03-31 09:09:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:08,280 [556] [DEBUG] [app] Starting request: urn:request:2b84fca9-5bee-4b4b-aa4c-2205bce6dbf9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:08,280 [556] [DEBUG] [app] Starting request: urn:request:2b84fca9-5bee-4b4b-aa4c-2205bce6dbf9 (/v1/_internal_ping) 2020-03-31 09:09:08,281 [556] [DEBUG] [app] Ending request: urn:request:2b84fca9-5bee-4b4b-aa4c-2205bce6dbf9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:08,281 [556] [DEBUG] [app] Ending request: urn:request:2b84fca9-5bee-4b4b-aa4c-2205bce6dbf9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:09:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:09:08,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:09:08,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:08,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:08,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:08,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:08,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:08,312 [518] [DEBUG] [app] Starting request: urn:request:df17dbbe-b06c-4c0f-acdc-15ae24d0d46f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:08,312 [518] [DEBUG] [app] Starting request: urn:request:df17dbbe-b06c-4c0f-acdc-15ae24d0d46f (/_internal_ping) 2020-03-31 09:09:08,313 [518] [DEBUG] [app] Ending request: urn:request:df17dbbe-b06c-4c0f-acdc-15ae24d0d46f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:08,313 [518] [DEBUG] [app] Ending request: urn:request:df17dbbe-b06c-4c0f-acdc-15ae24d0d46f (/_internal_ping) 2020-03-31 09:09:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:09:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:09:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:09:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:08,326 [543] [DEBUG] [app] Starting request: urn:request:22845fde-9d4d-46a5-aeaa-290d5fcc3cfc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:08,326 [543] [DEBUG] [app] Starting request: urn:request:22845fde-9d4d-46a5-aeaa-290d5fcc3cfc (/c1/_internal_ping) 2020-03-31 09:09:08,327 [543] [DEBUG] [app] Ending request: urn:request:22845fde-9d4d-46a5-aeaa-290d5fcc3cfc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:08,327 [543] [DEBUG] [app] Ending request: urn:request:22845fde-9d4d-46a5-aeaa-290d5fcc3cfc (/c1/_internal_ping) 2020-03-31 09:09:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:08,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:08,329 [518] [DEBUG] [app] Ending request: urn:request:234a4c15-8dd5-4a11-9b48-a3758150e243 (/health/instance) 2020-03-31 09:09:08,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:09:08,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:09:08,329 [518] [DEBUG] [app] Ending request: urn:request:234a4c15-8dd5-4a11-9b48-a3758150e243 (/health/instance) 2020-03-31 09:09:08,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:09:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:09:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:21 UTC)" (scheduled at 2020-03-31 09:09:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:09:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:21 UTC)" (scheduled at 2020-03-31 09:09:11.440707+00:00) 2020-03-31 09:09:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:21.440707+00:00 (in 9.998824 seconds) repositoryactioncounter stdout | 2020-03-31 09:09:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:21.440707+00:00 (in 9.998824 seconds) 2020-03-31 09:09:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:09:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:09:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:09:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:09:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:09:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:21 UTC)" executed successfully 2020-03-31 09:09:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:09:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:24.323352+00:00 (in 9.998680 seconds) 2020-03-31 09:09:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:24 UTC)" (scheduled at 2020-03-31 09:09:14.323352+00:00) 2020-03-31 09:09:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:09:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:24.323352+00:00 (in 9.998680 seconds) 2020-03-31 09:09:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:24 UTC)" (scheduled at 2020-03-31 09:09:14.323352+00:00) 2020-03-31 09:09:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 14, 325860), True, datetime.datetime(2020, 3, 31, 9, 9, 14, 325860), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:09:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 14, 325860), True, datetime.datetime(2020, 3, 31, 9, 9, 14, 325860), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:09:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:09:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:09:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:24 UTC)" executed successfully 2020-03-31 09:09:14,446 [518] [DEBUG] [app] Starting request: urn:request:7d05dd0e-52dd-410c-b57b-bcc7be0655ae (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:14,446 [518] [DEBUG] [app] Starting request: urn:request:7d05dd0e-52dd-410c-b57b-bcc7be0655ae (/health/instance) 2020-03-31 09:09:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:14,454 [556] [DEBUG] [app] Starting request: urn:request:5cb5b600-ea2a-4328-bdd2-f98910d765a2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:14,454 [556] [DEBUG] [app] Starting request: urn:request:5cb5b600-ea2a-4328-bdd2-f98910d765a2 (/v1/_internal_ping) 2020-03-31 09:09:14,456 [556] [DEBUG] [app] Ending request: urn:request:5cb5b600-ea2a-4328-bdd2-f98910d765a2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:14,456 [556] [DEBUG] [app] Ending request: urn:request:5cb5b600-ea2a-4328-bdd2-f98910d765a2 (/v1/_internal_ping) 2020-03-31 09:09:14,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:09:14,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:09:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:14,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:14,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:14,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:14,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:14,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:14,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:14,490 [525] [DEBUG] [app] Starting request: urn:request:a0f9a426-6a35-460e-81b9-48dda7abf357 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:14,490 [525] [DEBUG] [app] Starting request: urn:request:a0f9a426-6a35-460e-81b9-48dda7abf357 (/_internal_ping) 2020-03-31 09:09:14,491 [525] [DEBUG] [app] Ending request: urn:request:a0f9a426-6a35-460e-81b9-48dda7abf357 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:14,491 [525] [DEBUG] [app] Ending request: urn:request:a0f9a426-6a35-460e-81b9-48dda7abf357 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:09:14,492 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:14,492 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:09:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:09:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:14,505 [539] [DEBUG] [app] Starting request: urn:request:c5264ed5-3475-4d5e-ac4b-b2acd2dbae6e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:14,505 [539] [DEBUG] [app] Starting request: urn:request:c5264ed5-3475-4d5e-ac4b-b2acd2dbae6e (/c1/_internal_ping) 2020-03-31 09:09:14,506 [539] [DEBUG] [app] Ending request: urn:request:c5264ed5-3475-4d5e-ac4b-b2acd2dbae6e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:14,506 [539] [DEBUG] [app] Ending request: urn:request:c5264ed5-3475-4d5e-ac4b-b2acd2dbae6e (/c1/_internal_ping) 2020-03-31 09:09:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:14,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:09:14,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:14,509 [518] [DEBUG] [app] Ending request: urn:request:7d05dd0e-52dd-410c-b57b-bcc7be0655ae (/health/instance) 2020-03-31 09:09:14,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:14,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:09:14,509 [518] [DEBUG] [app] Ending request: urn:request:7d05dd0e-52dd-410c-b57b-bcc7be0655ae (/health/instance) 2020-03-31 09:09:14,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:14,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:09:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:09:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:45.064514+00:00 (in 29.998933 seconds) 2020-03-31 09:09:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:45 UTC)" (scheduled at 2020-03-31 09:09:15.064514+00:00) gcworker stdout | 2020-03-31 09:09:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:45.064514+00:00 (in 29.998933 seconds) 2020-03-31 09:09:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:45 UTC)" (scheduled at 2020-03-31 09:09:15.064514+00:00) 2020-03-31 09:09:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436155, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:09:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436155, 1209600, 500, 1, 0]) 2020-03-31 09:09:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:09:15,084 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:09:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:09:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:09:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:45 UTC)" executed successfully 2020-03-31 09:09:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:09:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:26.924497+00:00 (in 9.999290 seconds) 2020-03-31 09:09:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:26 UTC)" (scheduled at 2020-03-31 09:09:16.924497+00:00) 2020-03-31 09:09:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:09:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:26.924497+00:00 (in 9.999290 seconds) 2020-03-31 09:09:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:26 UTC)" (scheduled at 2020-03-31 09:09:16.924497+00:00) 2020-03-31 09:09:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 16, 925894), True, datetime.datetime(2020, 3, 31, 9, 9, 16, 925894), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:09:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 16, 925894), True, datetime.datetime(2020, 3, 31, 9, 9, 16, 925894), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:09:16,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:09:16,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:09:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:26 UTC)" executed successfully 2020-03-31 09:09:18,273 [518] [DEBUG] [app] Starting request: urn:request:8f4da157-3223-4ac8-ab7d-cf87b8a291d4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:18,273 [518] [DEBUG] [app] Starting request: urn:request:8f4da157-3223-4ac8-ab7d-cf87b8a291d4 (/health/instance) 2020-03-31 09:09:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:18,278 [548] [DEBUG] [app] Starting request: urn:request:4d00613a-4716-4487-800d-6952e12778dd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:18,278 [548] [DEBUG] [app] Starting request: urn:request:4d00613a-4716-4487-800d-6952e12778dd (/v1/_internal_ping) 2020-03-31 09:09:18,279 [548] [DEBUG] [app] Ending request: urn:request:4d00613a-4716-4487-800d-6952e12778dd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:18,279 [548] [DEBUG] [app] Ending request: urn:request:4d00613a-4716-4487-800d-6952e12778dd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:18,279 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:09:18,279 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:18,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:09:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:18,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:18,283 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:18,283 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:18,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:18,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:18,309 [523] [DEBUG] [app] Starting request: urn:request:aa7ca275-0e8e-43d2-903a-94927dac0a85 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:18,309 [523] [DEBUG] [app] Starting request: urn:request:aa7ca275-0e8e-43d2-903a-94927dac0a85 (/_internal_ping) 2020-03-31 09:09:18,310 [523] [DEBUG] [app] Ending request: urn:request:aa7ca275-0e8e-43d2-903a-94927dac0a85 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:18,310 [523] [DEBUG] [app] Ending request: urn:request:aa7ca275-0e8e-43d2-903a-94927dac0a85 (/_internal_ping) 2020-03-31 09:09:18,310 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:09:18,310 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:09:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:09:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:18,322 [539] [DEBUG] [app] Starting request: urn:request:b054d1c4-d97c-446f-ab5c-df8a2f9a6854 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:18,322 [539] [DEBUG] [app] Starting request: urn:request:b054d1c4-d97c-446f-ab5c-df8a2f9a6854 (/c1/_internal_ping) 2020-03-31 09:09:18,323 [539] [DEBUG] [app] Ending request: urn:request:b054d1c4-d97c-446f-ab5c-df8a2f9a6854 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:18,323 [539] [DEBUG] [app] Ending request: urn:request:b054d1c4-d97c-446f-ab5c-df8a2f9a6854 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:09:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:18,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:09:18,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:18,327 [518] [DEBUG] [app] Ending request: urn:request:8f4da157-3223-4ac8-ab7d-cf87b8a291d4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:18,327 [518] [DEBUG] [app] Ending request: urn:request:8f4da157-3223-4ac8-ab7d-cf87b8a291d4 (/health/instance) 2020-03-31 09:09:18,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:09:18,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:09:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:09:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:31.440707+00:00 (in 9.996472 seconds) 2020-03-31 09:09:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:31 UTC)" (scheduled at 2020-03-31 09:09:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:09:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:31.440707+00:00 (in 9.996472 seconds) 2020-03-31 09:09:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:31 UTC)" (scheduled at 2020-03-31 09:09:21.440707+00:00) 2020-03-31 09:09:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:09:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:09:21,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:09:21,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:09:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:09:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:31 UTC)" executed successfully 2020-03-31 09:09:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:09:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:22 UTC)" (scheduled at 2020-03-31 09:09:22.113221+00:00) securityworker stdout | 2020-03-31 09:09:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:22 UTC)" (scheduled at 2020-03-31 09:09:22.113221+00:00) 2020-03-31 09:09:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:09:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:52.113221+00:00 (in 29.999164 seconds) securityworker stdout | 2020-03-31 09:09:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:09:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:52.113221+00:00 (in 29.999164 seconds) 2020-03-31 09:09:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:09:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:09:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:09:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:09:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:09:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:09:52 UTC)" executed successfully 2020-03-31 09:09:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:09:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:23,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:24.410927+00:00 (in 0.999047 seconds) 2020-03-31 09:09:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:23 UTC)" (scheduled at 2020-03-31 09:09:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:09:23,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:24.410927+00:00 (in 0.999047 seconds) 2020-03-31 09:09:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:23 UTC)" (scheduled at 2020-03-31 09:09:23.410371+00:00) 2020-03-31 09:09:23,414 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:09:23,414 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), u'security_notification/%']) 2020-03-31 09:09:23,428 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), True, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:09:23,428 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), True, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), 0, u'security_notification/%']) 2020-03-31 09:09:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), True, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:09:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), True, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 9, 23, 412794), u'security_notification/%']) 2020-03-31 09:09:23,437 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:09:23,437 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:23,440 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:09:23,440 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:23 UTC)" executed successfully 2020-03-31 09:09:23,441 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:09:23,441 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:23,442 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:09:23,442 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:23,443 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:09:23,443 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:09:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:09:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:34.323352+00:00 (in 9.998887 seconds) 2020-03-31 09:09:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:34 UTC)" (scheduled at 2020-03-31 09:09:24.323352+00:00) 2020-03-31 09:09:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:09:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:34.323352+00:00 (in 9.998887 seconds) 2020-03-31 09:09:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:34 UTC)" (scheduled at 2020-03-31 09:09:24.323352+00:00) 2020-03-31 09:09:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 24, 325516), True, datetime.datetime(2020, 3, 31, 9, 9, 24, 325516), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:09:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 24, 325516), True, datetime.datetime(2020, 3, 31, 9, 9, 24, 325516), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:09:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:09:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:09:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:34 UTC)" executed successfully 2020-03-31 09:09:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:09:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:33.330112+00:00 (in 8.918377 seconds) 2020-03-31 09:09:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:24 UTC)" (scheduled at 2020-03-31 09:09:24.410927+00:00) 2020-03-31 09:09:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:09:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:33.330112+00:00 (in 8.918377 seconds) 2020-03-31 09:09:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:24 UTC)" (scheduled at 2020-03-31 09:09:24.410927+00:00) 2020-03-31 09:09:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:09:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:24 UTC)" executed successfully 2020-03-31 09:09:24,447 [521] [DEBUG] [app] Starting request: urn:request:19d6de0b-6c01-4578-9fac-926bee3f19d6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:24,447 [521] [DEBUG] [app] Starting request: urn:request:19d6de0b-6c01-4578-9fac-926bee3f19d6 (/health/instance) 2020-03-31 09:09:24,455 [521] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 09:09:24,455 [521] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 09:09:24,469 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:24,469 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:24,471 [538] [DEBUG] [app] Starting request: urn:request:8734df68-5872-45b3-83f4-019150e1576b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:24,471 [538] [DEBUG] [app] Starting request: urn:request:8734df68-5872-45b3-83f4-019150e1576b (/v1/_internal_ping) 2020-03-31 09:09:24,472 [538] [DEBUG] [app] Ending request: urn:request:8734df68-5872-45b3-83f4-019150e1576b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:24,472 [538] [DEBUG] [app] Ending request: urn:request:8734df68-5872-45b3-83f4-019150e1576b (/v1/_internal_ping) 2020-03-31 09:09:24,473 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:24,474 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-registry stdout | 2020-03-31 09:09:24,473 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:24,474 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:24,476 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:09:24,477 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:24,476 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:24,478 [521] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:24,477 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:24,478 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:09:24,478 [521] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:24,478 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:24,489 [521] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:24,489 [521] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:24,502 [521] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:24,502 [521] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:24,506 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:24,506 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:24,508 [521] [DEBUG] [app] Starting request: urn:request:f78e8758-6b64-4acf-baf3-552ad42315c9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:24,508 [521] [DEBUG] [app] Starting request: urn:request:f78e8758-6b64-4acf-baf3-552ad42315c9 (/_internal_ping) 2020-03-31 09:09:24,509 [521] [DEBUG] [app] Ending request: urn:request:f78e8758-6b64-4acf-baf3-552ad42315c9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:24,509 [521] [DEBUG] [app] Ending request: urn:request:f78e8758-6b64-4acf-baf3-552ad42315c9 (/_internal_ping) 2020-03-31 09:09:24,510 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:24,510 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:09:24,510 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:24,510 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:09:24,514 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:24,514 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:24,515 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:24,515 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:24,518 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:24,518 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:24,520 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:09:24,520 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:09:24,523 [521] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 9, 24, 521724), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 9, 24, 521789), u'quay', None, datetime.datetime(2020, 3, 30, 9, 9, 24, 521826), datetime.datetime(2020, 3, 24, 9, 9, 24, 521853), None]) gunicorn-web stdout | 2020-03-31 09:09:24,523 [521] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 9, 24, 521724), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 9, 24, 521789), u'quay', None, datetime.datetime(2020, 3, 30, 9, 9, 24, 521826), datetime.datetime(2020, 3, 24, 9, 9, 24, 521853), None]) 2020-03-31 09:09:24,538 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:24,538 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:24,539 [541] [DEBUG] [app] Starting request: urn:request:83db9245-c7e3-4912-9596-119179b66a6e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:24,539 [541] [DEBUG] [app] Starting request: urn:request:83db9245-c7e3-4912-9596-119179b66a6e (/c1/_internal_ping) 2020-03-31 09:09:24,541 [541] [DEBUG] [app] Ending request: urn:request:83db9245-c7e3-4912-9596-119179b66a6e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:24,541 [541] [DEBUG] [app] Ending request: urn:request:83db9245-c7e3-4912-9596-119179b66a6e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:24,541 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:24,542 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:24,544 [521] [DEBUG] [app] Ending request: urn:request:19d6de0b-6c01-4578-9fac-926bee3f19d6 (/health/instance) 2020-03-31 09:09:24,544 [521] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:09:24,541 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:24,545 [521] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.099 49 0.099) gunicorn-web stdout | 2020-03-31 09:09:24,542 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:24,544 [521] [DEBUG] [app] Ending request: urn:request:19d6de0b-6c01-4578-9fac-926bee3f19d6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:24,544 [521] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:24,545 [521] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.099 49 0.099) 2020-03-31 09:09:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:09:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:36.924497+00:00 (in 9.999153 seconds) 2020-03-31 09:09:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:36 UTC)" (scheduled at 2020-03-31 09:09:26.924497+00:00) 2020-03-31 09:09:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:09:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:36.924497+00:00 (in 9.999153 seconds) 2020-03-31 09:09:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:36 UTC)" (scheduled at 2020-03-31 09:09:26.924497+00:00) 2020-03-31 09:09:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 26, 926314), True, datetime.datetime(2020, 3, 31, 9, 9, 26, 926314), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:09:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 26, 926314), True, datetime.datetime(2020, 3, 31, 9, 9, 26, 926314), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:09:26,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:09:26,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:09:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:36 UTC)" executed successfully 2020-03-31 09:09:28,273 [518] [DEBUG] [app] Starting request: urn:request:52a5fc98-d371-42eb-9e43-465562cb202e (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:28,273 [518] [DEBUG] [app] Starting request: urn:request:52a5fc98-d371-42eb-9e43-465562cb202e (/health/instance) 2020-03-31 09:09:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:28,281 [542] [DEBUG] [app] Starting request: urn:request:ff0b5ad1-14d0-4926-adbf-c9e82a79dcdf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:28,281 [542] [DEBUG] [app] Starting request: urn:request:ff0b5ad1-14d0-4926-adbf-c9e82a79dcdf (/v1/_internal_ping) 2020-03-31 09:09:28,282 [542] [DEBUG] [app] Ending request: urn:request:ff0b5ad1-14d0-4926-adbf-c9e82a79dcdf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:28,282 [542] [DEBUG] [app] Ending request: urn:request:ff0b5ad1-14d0-4926-adbf-c9e82a79dcdf (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:28,283 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:09:28,283 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:28,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:28,285 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:09:28,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:28,285 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:28,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:28,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:28,287 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:28,287 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:28,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:28,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:28,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:28,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:28,313 [526] [DEBUG] [app] Starting request: urn:request:ffc6cac7-1880-4842-8618-e42e315cdfc5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:28,313 [526] [DEBUG] [app] Starting request: urn:request:ffc6cac7-1880-4842-8618-e42e315cdfc5 (/_internal_ping) 2020-03-31 09:09:28,314 [526] [DEBUG] [app] Ending request: urn:request:ffc6cac7-1880-4842-8618-e42e315cdfc5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:28,314 [526] [DEBUG] [app] Ending request: urn:request:ffc6cac7-1880-4842-8618-e42e315cdfc5 (/_internal_ping) 2020-03-31 09:09:28,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:28,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:09:28,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:28,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:09:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:09:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:28,329 [539] [DEBUG] [app] Starting request: urn:request:bfcd2263-ce36-4e8c-b00b-12e48cf171ad (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:28,329 [539] [DEBUG] [app] Starting request: urn:request:bfcd2263-ce36-4e8c-b00b-12e48cf171ad (/c1/_internal_ping) 2020-03-31 09:09:28,330 [539] [DEBUG] [app] Ending request: urn:request:bfcd2263-ce36-4e8c-b00b-12e48cf171ad (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:28,330 [539] [DEBUG] [app] Ending request: urn:request:bfcd2263-ce36-4e8c-b00b-12e48cf171ad (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:28,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:09:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:28,332 [518] [DEBUG] [app] Ending request: urn:request:52a5fc98-d371-42eb-9e43-465562cb202e (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:09:28,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:28,332 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:09:28,332 [518] [DEBUG] [app] Ending request: urn:request:52a5fc98-d371-42eb-9e43-465562cb202e (/health/instance) 2020-03-31 09:09:28,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:09:28,332 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:09:28,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:09:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:09:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:00.337699+00:00 (in 29.999305 seconds) 2020-03-31 09:09:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:00 UTC)" (scheduled at 2020-03-31 09:09:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:09:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:00.337699+00:00 (in 29.999305 seconds) 2020-03-31 09:09:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:00 UTC)" (scheduled at 2020-03-31 09:09:30.337699+00:00) 2020-03-31 09:09:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 9, 30, 339164), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:09:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 9, 30, 339164), False, 50, 1, 0]) 2020-03-31 09:09:30,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:09:30,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:09:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:09:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:00 UTC)" executed successfully 2020-03-31 09:09:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:09:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:41 UTC)" (scheduled at 2020-03-31 09:09:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:09:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:41 UTC)" (scheduled at 2020-03-31 09:09:31.440707+00:00) 2020-03-31 09:09:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:09:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:09:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:41.440707+00:00 (in 9.998867 seconds) repositoryactioncounter stdout | 2020-03-31 09:09:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:41.440707+00:00 (in 9.998867 seconds) 2020-03-31 09:09:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:09:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:09:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:09:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:41 UTC)" executed successfully 2020-03-31 09:09:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:09:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:03.330112+00:00 (in 29.998616 seconds) 2020-03-31 09:09:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:03 UTC)" (scheduled at 2020-03-31 09:09:33.330112+00:00) 2020-03-31 09:09:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:09:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:03.330112+00:00 (in 29.998616 seconds) 2020-03-31 09:09:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:03 UTC)" (scheduled at 2020-03-31 09:09:33.330112+00:00) 2020-03-31 09:09:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 33, 332755), True, datetime.datetime(2020, 3, 31, 9, 9, 33, 332755), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:09:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 33, 332755), True, datetime.datetime(2020, 3, 31, 9, 9, 33, 332755), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:09:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:09:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:09:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:03 UTC)" executed successfully 2020-03-31 09:09:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:09:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:33 UTC)" (scheduled at 2020-03-31 09:09:33.366791+00:00) 2020-03-31 09:09:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:09:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:33 UTC)" (scheduled at 2020-03-31 09:09:33.366791+00:00) 2020-03-31 09:09:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:33,370 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:43.539999+00:00 (in 10.172177 seconds) exportactionlogsworker stdout | 2020-03-31 09:09:33,370 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:43.539999+00:00 (in 10.172177 seconds) 2020-03-31 09:09:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 33, 368781), True, datetime.datetime(2020, 3, 31, 9, 9, 33, 368781), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:09:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 33, 368781), True, datetime.datetime(2020, 3, 31, 9, 9, 33, 368781), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:09:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:09:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:09:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:33 UTC)" executed successfully 2020-03-31 09:09:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:09:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:44.323352+00:00 (in 9.999202 seconds) 2020-03-31 09:09:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:44 UTC)" (scheduled at 2020-03-31 09:09:34.323352+00:00) 2020-03-31 09:09:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:09:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:44.323352+00:00 (in 9.999202 seconds) 2020-03-31 09:09:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:44 UTC)" (scheduled at 2020-03-31 09:09:34.323352+00:00) 2020-03-31 09:09:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 34, 325145), True, datetime.datetime(2020, 3, 31, 9, 9, 34, 325145), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:09:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 34, 325145), True, datetime.datetime(2020, 3, 31, 9, 9, 34, 325145), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:09:34,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:34,341 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:09:34,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:09:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:44 UTC)" executed successfully 2020-03-31 09:09:34,446 [518] [DEBUG] [app] Starting request: urn:request:1eda76d2-2a35-4c80-bc0b-7cf9cf881372 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:34,446 [518] [DEBUG] [app] Starting request: urn:request:1eda76d2-2a35-4c80-bc0b-7cf9cf881372 (/health/instance) 2020-03-31 09:09:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:34,453 [556] [DEBUG] [app] Starting request: urn:request:e5d4abbb-9c85-4768-ba02-b8d379a060f6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:34,453 [556] [DEBUG] [app] Starting request: urn:request:e5d4abbb-9c85-4768-ba02-b8d379a060f6 (/v1/_internal_ping) 2020-03-31 09:09:34,454 [556] [DEBUG] [app] Ending request: urn:request:e5d4abbb-9c85-4768-ba02-b8d379a060f6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:34,454 [556] [DEBUG] [app] Ending request: urn:request:e5d4abbb-9c85-4768-ba02-b8d379a060f6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:34,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:09:34,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:09:34,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:09:34,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:34,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:34,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:34,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:34,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:34,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:34,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:34,485 [526] [DEBUG] [app] Starting request: urn:request:1e5e6ded-7400-44ed-88dd-213f87eb44d8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:34,485 [526] [DEBUG] [app] Starting request: urn:request:1e5e6ded-7400-44ed-88dd-213f87eb44d8 (/_internal_ping) 2020-03-31 09:09:34,486 [526] [DEBUG] [app] Ending request: urn:request:1e5e6ded-7400-44ed-88dd-213f87eb44d8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:34,486 [526] [DEBUG] [app] Ending request: urn:request:1e5e6ded-7400-44ed-88dd-213f87eb44d8 (/_internal_ping) 2020-03-31 09:09:34,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:09:34,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:09:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:09:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:09:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:34,499 [543] [DEBUG] [app] Starting request: urn:request:a965a135-6c6c-4b7d-8f83-7178399e99f4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:34,499 [543] [DEBUG] [app] Starting request: urn:request:a965a135-6c6c-4b7d-8f83-7178399e99f4 (/c1/_internal_ping) 2020-03-31 09:09:34,500 [543] [DEBUG] [app] Ending request: urn:request:a965a135-6c6c-4b7d-8f83-7178399e99f4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:34,500 [543] [DEBUG] [app] Ending request: urn:request:a965a135-6c6c-4b7d-8f83-7178399e99f4 (/c1/_internal_ping) 2020-03-31 09:09:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:34,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:34,503 [518] [DEBUG] [app] Ending request: urn:request:1eda76d2-2a35-4c80-bc0b-7cf9cf881372 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:09:34,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:34,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:09:34,503 [518] [DEBUG] [app] Ending request: urn:request:1eda76d2-2a35-4c80-bc0b-7cf9cf881372 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:34,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:09:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:09:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:38.010934+00:00 (in 1.085352 seconds) 2020-03-31 09:09:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:46 UTC)" (scheduled at 2020-03-31 09:09:36.924497+00:00) 2020-03-31 09:09:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:09:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:38.010934+00:00 (in 1.085352 seconds) 2020-03-31 09:09:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:46 UTC)" (scheduled at 2020-03-31 09:09:36.924497+00:00) 2020-03-31 09:09:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 36, 926435), True, datetime.datetime(2020, 3, 31, 9, 9, 36, 926435), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:09:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 36, 926435), True, datetime.datetime(2020, 3, 31, 9, 9, 36, 926435), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:09:36,941 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:09:36,941 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:36,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:36,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:09:36,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:36,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:46 UTC)" executed successfully 2020-03-31 09:09:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:09:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:46.924497+00:00 (in 8.912827 seconds) 2020-03-31 09:09:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:38 UTC)" (scheduled at 2020-03-31 09:09:38.010934+00:00) 2020-03-31 09:09:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:09:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:46.924497+00:00 (in 8.912827 seconds) 2020-03-31 09:09:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:38 UTC)" (scheduled at 2020-03-31 09:09:38.010934+00:00) 2020-03-31 09:09:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:38 UTC)" executed successfully 2020-03-31 09:09:38,273 [526] [DEBUG] [app] Starting request: urn:request:e9b58cea-d8bc-4af3-87d6-623ad4f89f30 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:38,273 [526] [DEBUG] [app] Starting request: urn:request:e9b58cea-d8bc-4af3-87d6-623ad4f89f30 (/health/instance) 2020-03-31 09:09:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:38,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:38,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:38,281 [550] [DEBUG] [app] Starting request: urn:request:f0349280-45c2-462f-915c-b6d2f3629d81 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:38,281 [550] [DEBUG] [app] Starting request: urn:request:f0349280-45c2-462f-915c-b6d2f3629d81 (/v1/_internal_ping) 2020-03-31 09:09:38,282 [550] [DEBUG] [app] Ending request: urn:request:f0349280-45c2-462f-915c-b6d2f3629d81 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:38,282 [550] [DEBUG] [app] Ending request: urn:request:f0349280-45c2-462f-915c-b6d2f3629d81 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:38,283 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:38,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:09:38,283 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:38,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:38,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:38,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:09:38,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:09:38,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:38,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:38,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:38,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:38,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:38,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:38,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:38,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:38,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:38,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:38,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:38,315 [526] [DEBUG] [app] Starting request: urn:request:d59baf2d-8b4c-4725-bf66-dc6d0d625c4b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:38,315 [526] [DEBUG] [app] Starting request: urn:request:d59baf2d-8b4c-4725-bf66-dc6d0d625c4b (/_internal_ping) 2020-03-31 09:09:38,316 [526] [DEBUG] [app] Ending request: urn:request:d59baf2d-8b4c-4725-bf66-dc6d0d625c4b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:38,316 [526] [DEBUG] [app] Ending request: urn:request:d59baf2d-8b4c-4725-bf66-dc6d0d625c4b (/_internal_ping) 2020-03-31 09:09:38,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:09:38,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:09:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:38,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:38,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:38,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:38,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:38,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:38,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:09:38,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:38,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:38,329 [541] [DEBUG] [app] Starting request: urn:request:239058ce-01d6-45a2-bd36-0c7fa7c71905 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:38,329 [541] [DEBUG] [app] Starting request: urn:request:239058ce-01d6-45a2-bd36-0c7fa7c71905 (/c1/_internal_ping) 2020-03-31 09:09:38,330 [541] [DEBUG] [app] Ending request: urn:request:239058ce-01d6-45a2-bd36-0c7fa7c71905 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:38,330 [541] [DEBUG] [app] Ending request: urn:request:239058ce-01d6-45a2-bd36-0c7fa7c71905 (/c1/_internal_ping) 2020-03-31 09:09:38,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:38,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:38,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:09:38,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:38,332 [526] [DEBUG] [app] Ending request: urn:request:e9b58cea-d8bc-4af3-87d6-623ad4f89f30 (/health/instance) 2020-03-31 09:09:38,333 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:38,332 [526] [DEBUG] [app] Ending request: urn:request:e9b58cea-d8bc-4af3-87d6-623ad4f89f30 (/health/instance) 2020-03-31 09:09:38,333 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:38,333 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:09:38,333 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:09:41,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:09:41,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:41,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:51.440707+00:00 (in 9.996289 seconds) 2020-03-31 09:09:41,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:51 UTC)" (scheduled at 2020-03-31 09:09:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:09:41,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:51.440707+00:00 (in 9.996289 seconds) 2020-03-31 09:09:41,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:51 UTC)" (scheduled at 2020-03-31 09:09:41.440707+00:00) 2020-03-31 09:09:41,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:09:41,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:09:41,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:09:41,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:09:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:09:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:51 UTC)" executed successfully 2020-03-31 09:09:41,836 [179] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2020-03-31 09:09:41,836 [179] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:41,836 [179] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:09:41.835723+00:00 (in 3599.999545 seconds) 2020-03-31 09:09:41,837 [179] [INFO] [apscheduler.executors.default] Running job "_cleanup_uploads (trigger: interval[1:00:00], next run at: 2020-03-31 10:09:41 UTC)" (scheduled at 2020-03-31 09:09:41.835723+00:00) blobuploadcleanupworker stdout | 2020-03-31 09:09:41,836 [179] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:09:41.835723+00:00 (in 3599.999545 seconds) 2020-03-31 09:09:41,837 [179] [INFO] [apscheduler.executors.default] Running job "_cleanup_uploads (trigger: interval[1:00:00], next run at: 2020-03-31 10:09:41 UTC)" (scheduled at 2020-03-31 09:09:41.835723+00:00) 2020-03-31 09:09:41,838 [179] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT DISTINCT "t1"."id" FROM "blobupload" AS "t1" WHERE ("t1"."created" <= %s) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 29, 9, 9, 41, 837491), 500, 1, 0]) blobuploadcleanupworker stdout | 2020-03-31 09:09:41,838 [179] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT DISTINCT "t1"."id" FROM "blobupload" AS "t1" WHERE ("t1"."created" <= %s) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 29, 9, 9, 41, 837491), 500, 1, 0]) 2020-03-31 09:09:41,853 [179] [DEBUG] [__main__] No additional stale blob uploads found blobuploadcleanupworker stdout | 2020-03-31 09:09:41,853 [179] [DEBUG] [__main__] No additional stale blob uploads found 2020-03-31 09:09:41,853 [179] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:41,854 [179] [INFO] [apscheduler.executors.default] Job "_cleanup_uploads (trigger: interval[1:00:00], next run at: 2020-03-31 10:09:41 UTC)" executed successfully blobuploadcleanupworker stdout | 2020-03-31 09:09:41,853 [179] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:41,854 [179] [INFO] [apscheduler.executors.default] Job "_cleanup_uploads (trigger: interval[1:00:00], next run at: 2020-03-31 10:09:41 UTC)" executed successfully 2020-03-31 09:09:43,543 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:09:43,543 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:43,543 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:14.539408+00:00 (in 30.996076 seconds) 2020-03-31 09:09:43,543 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:43 UTC)" (scheduled at 2020-03-31 09:09:43.539999+00:00) 2020-03-31 09:09:43,544 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:43,544 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:09:43,543 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:14.539408+00:00 (in 30.996076 seconds) 2020-03-31 09:09:43,543 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:43 UTC)" (scheduled at 2020-03-31 09:09:43.539999+00:00) 2020-03-31 09:09:43,544 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:43,544 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:43 UTC)" executed successfully 2020-03-31 09:09:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:09:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:54 UTC)" (scheduled at 2020-03-31 09:09:44.323352+00:00) 2020-03-31 09:09:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:09:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:54 UTC)" (scheduled at 2020-03-31 09:09:44.323352+00:00) 2020-03-31 09:09:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:49.498966+00:00 (in 5.174337 seconds) notificationworker stdout | 2020-03-31 09:09:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:49.498966+00:00 (in 5.174337 seconds) 2020-03-31 09:09:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 44, 325957), True, datetime.datetime(2020, 3, 31, 9, 9, 44, 325957), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:09:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 44, 325957), True, datetime.datetime(2020, 3, 31, 9, 9, 44, 325957), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:09:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:09:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:09:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:54 UTC)" executed successfully 2020-03-31 09:09:44,446 [518] [DEBUG] [app] Starting request: urn:request:02d2c607-fffe-4853-95e2-fb8c3fc020f4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:44,446 [518] [DEBUG] [app] Starting request: urn:request:02d2c607-fffe-4853-95e2-fb8c3fc020f4 (/health/instance) 2020-03-31 09:09:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:44,454 [547] [DEBUG] [app] Starting request: urn:request:8b95f135-78ee-4729-9bb1-0f0867a772cb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:44,454 [547] [DEBUG] [app] Starting request: urn:request:8b95f135-78ee-4729-9bb1-0f0867a772cb (/v1/_internal_ping) 2020-03-31 09:09:44,455 [547] [DEBUG] [app] Ending request: urn:request:8b95f135-78ee-4729-9bb1-0f0867a772cb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:44,455 [547] [DEBUG] [app] Ending request: urn:request:8b95f135-78ee-4729-9bb1-0f0867a772cb (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:44,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:09:44,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:09:44,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:09:44,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:44,459 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:44,459 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:44,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:44,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:44,486 [518] [DEBUG] [app] Starting request: urn:request:4d21f028-6c7e-4365-93d8-358c5f674c9c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:44,486 [518] [DEBUG] [app] Starting request: urn:request:4d21f028-6c7e-4365-93d8-358c5f674c9c (/_internal_ping) 2020-03-31 09:09:44,487 [518] [DEBUG] [app] Ending request: urn:request:4d21f028-6c7e-4365-93d8-358c5f674c9c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:44,487 [518] [DEBUG] [app] Ending request: urn:request:4d21f028-6c7e-4365-93d8-358c5f674c9c (/_internal_ping) 2020-03-31 09:09:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:09:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:09:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:09:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:44,499 [543] [DEBUG] [app] Starting request: urn:request:f5b76255-2a16-43a2-b856-83cd0b31d050 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:44,499 [543] [DEBUG] [app] Starting request: urn:request:f5b76255-2a16-43a2-b856-83cd0b31d050 (/c1/_internal_ping) 2020-03-31 09:09:44,500 [543] [DEBUG] [app] Ending request: urn:request:f5b76255-2a16-43a2-b856-83cd0b31d050 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:44,500 [543] [DEBUG] [app] Ending request: urn:request:f5b76255-2a16-43a2-b856-83cd0b31d050 (/c1/_internal_ping) 2020-03-31 09:09:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:44,500 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:44,502 [518] [DEBUG] [app] Ending request: urn:request:02d2c607-fffe-4853-95e2-fb8c3fc020f4 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:09:44,500 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:44,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:44,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:09:44,502 [518] [DEBUG] [app] Ending request: urn:request:02d2c607-fffe-4853-95e2-fb8c3fc020f4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:44,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:44,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:09:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:09:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:15.064514+00:00 (in 29.999016 seconds) 2020-03-31 09:09:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:15 UTC)" (scheduled at 2020-03-31 09:09:45.064514+00:00) gcworker stdout | 2020-03-31 09:09:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:15.064514+00:00 (in 29.999016 seconds) 2020-03-31 09:09:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:15 UTC)" (scheduled at 2020-03-31 09:09:45.064514+00:00) 2020-03-31 09:09:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436185, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:09:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436185, 1209600, 500, 1, 0]) 2020-03-31 09:09:45,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:09:45,088 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:09:45,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:09:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:09:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:15 UTC)" executed successfully 2020-03-31 09:09:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:09:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:56.924497+00:00 (in 9.999206 seconds) 2020-03-31 09:09:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:56 UTC)" (scheduled at 2020-03-31 09:09:46.924497+00:00) 2020-03-31 09:09:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:09:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:56.924497+00:00 (in 9.999206 seconds) 2020-03-31 09:09:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:56 UTC)" (scheduled at 2020-03-31 09:09:46.924497+00:00) 2020-03-31 09:09:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 46, 926117), True, datetime.datetime(2020, 3, 31, 9, 9, 46, 926117), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:09:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 46, 926117), True, datetime.datetime(2020, 3, 31, 9, 9, 46, 926117), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:09:46,949 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:09:46,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:46,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:09:46,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:09:56 UTC)" executed successfully 2020-03-31 09:09:48,273 [518] [DEBUG] [app] Starting request: urn:request:5be7341f-3ba8-4986-8cbd-65d546c06d27 (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:48,273 [518] [DEBUG] [app] Starting request: urn:request:5be7341f-3ba8-4986-8cbd-65d546c06d27 (/health/instance) 2020-03-31 09:09:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:48,278 [555] [DEBUG] [app] Starting request: urn:request:fdeb596a-94a1-4f3b-b4c0-98ad5ccb7e72 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:48,278 [555] [DEBUG] [app] Starting request: urn:request:fdeb596a-94a1-4f3b-b4c0-98ad5ccb7e72 (/v1/_internal_ping) 2020-03-31 09:09:48,279 [555] [DEBUG] [app] Ending request: urn:request:fdeb596a-94a1-4f3b-b4c0-98ad5ccb7e72 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:48,279 [555] [DEBUG] [app] Ending request: urn:request:fdeb596a-94a1-4f3b-b4c0-98ad5ccb7e72 (/v1/_internal_ping) 2020-03-31 09:09:48,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:48,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:09:48,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:48,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:09:48,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:09:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:48,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:48,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:48,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:48,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:48,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:48,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:48,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:48,302 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:48,302 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:48,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:48,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:48,308 [518] [DEBUG] [app] Starting request: urn:request:44d4ba02-73f2-48d1-8bb0-a385808ccfec (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:48,308 [518] [DEBUG] [app] Starting request: urn:request:44d4ba02-73f2-48d1-8bb0-a385808ccfec (/_internal_ping) 2020-03-31 09:09:48,309 [518] [DEBUG] [app] Ending request: urn:request:44d4ba02-73f2-48d1-8bb0-a385808ccfec (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:48,309 [518] [DEBUG] [app] Ending request: urn:request:44d4ba02-73f2-48d1-8bb0-a385808ccfec (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:09:48,309 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:48,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:48,309 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:48,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:09:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:09:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:09:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:09:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:48,322 [539] [DEBUG] [app] Starting request: urn:request:efa5cc87-2747-4678-bb69-30dd4187936b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:48,322 [539] [DEBUG] [app] Starting request: urn:request:efa5cc87-2747-4678-bb69-30dd4187936b (/c1/_internal_ping) 2020-03-31 09:09:48,323 [539] [DEBUG] [app] Ending request: urn:request:efa5cc87-2747-4678-bb69-30dd4187936b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:48,323 [539] [DEBUG] [app] Ending request: urn:request:efa5cc87-2747-4678-bb69-30dd4187936b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:48,323 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:48,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:09:48,323 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:48,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:48,325 [518] [DEBUG] [app] Ending request: urn:request:5be7341f-3ba8-4986-8cbd-65d546c06d27 (/health/instance) 2020-03-31 09:09:48,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:48,325 [518] [DEBUG] [app] Ending request: urn:request:5be7341f-3ba8-4986-8cbd-65d546c06d27 (/health/instance) 2020-03-31 09:09:48,326 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:09:48,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:09:48,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:09:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:09:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:49,502 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:54.323352+00:00 (in 4.821464 seconds) 2020-03-31 09:09:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:49 UTC)" (scheduled at 2020-03-31 09:09:49.498966+00:00) 2020-03-31 09:09:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:09:49,502 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:54.323352+00:00 (in 4.821464 seconds) 2020-03-31 09:09:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:49 UTC)" (scheduled at 2020-03-31 09:09:49.498966+00:00) 2020-03-31 09:09:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:49 UTC)" executed successfully 2020-03-31 09:09:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:09:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:53.556282+00:00 (in 2.444759 seconds) 2020-03-31 09:09:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:51 UTC)" (scheduled at 2020-03-31 09:09:51.110610+00:00) 2020-03-31 09:09:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:09:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:53.556282+00:00 (in 2.444759 seconds) 2020-03-31 09:09:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:51 UTC)" (scheduled at 2020-03-31 09:09:51.110610+00:00) 2020-03-31 09:09:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 51, 112361), True, datetime.datetime(2020, 3, 31, 9, 9, 51, 112361), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:09:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 51, 112361), True, datetime.datetime(2020, 3, 31, 9, 9, 51, 112361), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:09:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:09:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:09:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:51 UTC)" executed successfully 2020-03-31 09:09:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:09:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:01.440707+00:00 (in 9.995624 seconds) 2020-03-31 09:09:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:01 UTC)" (scheduled at 2020-03-31 09:09:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:09:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:01.440707+00:00 (in 9.995624 seconds) 2020-03-31 09:09:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:01 UTC)" (scheduled at 2020-03-31 09:09:51.440707+00:00) 2020-03-31 09:09:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:09:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:09:51,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:09:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:09:51,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:09:51,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:01 UTC)" executed successfully 2020-03-31 09:09:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:09:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:22.113221+00:00 (in 29.997474 seconds) 2020-03-31 09:09:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:22 UTC)" (scheduled at 2020-03-31 09:09:52.113221+00:00) securityworker stdout | 2020-03-31 09:09:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:22.113221+00:00 (in 29.997474 seconds) 2020-03-31 09:09:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:22 UTC)" (scheduled at 2020-03-31 09:09:52.113221+00:00) 2020-03-31 09:09:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:09:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:09:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:09:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:09:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:09:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:09:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:09:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:22 UTC)" executed successfully 2020-03-31 09:09:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:09:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:53 UTC)" (scheduled at 2020-03-31 09:09:53.556282+00:00) 2020-03-31 09:09:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:59.560326+00:00 (in 6.003383 seconds) namespacegcworker stdout | 2020-03-31 09:09:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:53 UTC)" (scheduled at 2020-03-31 09:09:53.556282+00:00) 2020-03-31 09:09:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:09:59.560326+00:00 (in 6.003383 seconds) 2020-03-31 09:09:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:09:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), u'namespacegc/%']) 2020-03-31 09:09:53,574 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), True, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:09:53,574 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), True, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), 0, u'namespacegc/%']) 2020-03-31 09:09:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), True, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:09:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), True, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 9, 53, 557747), u'namespacegc/%']) 2020-03-31 09:09:53,581 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:09:53,581 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:09:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:53,584 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:53 UTC)" executed successfully 2020-03-31 09:09:53,585 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:09:53,584 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:14:53 UTC)" executed successfully 2020-03-31 09:09:53,585 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:53,586 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:09:53,586 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:09:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:09:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:04.323352+00:00 (in 9.998947 seconds) 2020-03-31 09:09:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:04 UTC)" (scheduled at 2020-03-31 09:09:54.323352+00:00) 2020-03-31 09:09:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:09:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:04.323352+00:00 (in 9.998947 seconds) 2020-03-31 09:09:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:04 UTC)" (scheduled at 2020-03-31 09:09:54.323352+00:00) 2020-03-31 09:09:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 54, 325619), True, datetime.datetime(2020, 3, 31, 9, 9, 54, 325619), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:09:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 54, 325619), True, datetime.datetime(2020, 3, 31, 9, 9, 54, 325619), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:09:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:09:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:09:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:04 UTC)" executed successfully 2020-03-31 09:09:54,445 [518] [DEBUG] [app] Starting request: urn:request:671e5a31-1271-4a48-9015-db637a4da6db (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:54,445 [518] [DEBUG] [app] Starting request: urn:request:671e5a31-1271-4a48-9015-db637a4da6db (/health/instance) 2020-03-31 09:09:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:54,453 [556] [DEBUG] [app] Starting request: urn:request:a5c9412a-7665-44f6-88fd-0727880d4886 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:54,453 [556] [DEBUG] [app] Starting request: urn:request:a5c9412a-7665-44f6-88fd-0727880d4886 (/v1/_internal_ping) 2020-03-31 09:09:54,454 [556] [DEBUG] [app] Ending request: urn:request:a5c9412a-7665-44f6-88fd-0727880d4886 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:54,454 [556] [DEBUG] [app] Ending request: urn:request:a5c9412a-7665-44f6-88fd-0727880d4886 (/v1/_internal_ping) 2020-03-31 09:09:54,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:09:54,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:54,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:09:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:54,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:54,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:54,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:54,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:54,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:54,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:54,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:54,481 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:54,481 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:54,483 [527] [DEBUG] [app] Starting request: urn:request:d15884dd-9b1d-4de5-8cef-de9fef183aa7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:54,483 [527] [DEBUG] [app] Starting request: urn:request:d15884dd-9b1d-4de5-8cef-de9fef183aa7 (/_internal_ping) 2020-03-31 09:09:54,484 [527] [DEBUG] [app] Ending request: urn:request:d15884dd-9b1d-4de5-8cef-de9fef183aa7 (/_internal_ping) 2020-03-31 09:09:54,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:54,485 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:09:54,484 [527] [DEBUG] [app] Ending request: urn:request:d15884dd-9b1d-4de5-8cef-de9fef183aa7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:54,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:54,485 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:09:54,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:54,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:09:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:09:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:09:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:54,498 [536] [DEBUG] [app] Starting request: urn:request:da8e5b3e-7def-4e0f-88e6-4c0dd912b320 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:54,498 [536] [DEBUG] [app] Starting request: urn:request:da8e5b3e-7def-4e0f-88e6-4c0dd912b320 (/c1/_internal_ping) 2020-03-31 09:09:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:54,499 [536] [DEBUG] [app] Ending request: urn:request:da8e5b3e-7def-4e0f-88e6-4c0dd912b320 (/c1/_internal_ping) 2020-03-31 09:09:54,500 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:09:54,499 [536] [DEBUG] [app] Ending request: urn:request:da8e5b3e-7def-4e0f-88e6-4c0dd912b320 (/c1/_internal_ping) 2020-03-31 09:09:54,500 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:54,501 [518] [DEBUG] [app] Ending request: urn:request:671e5a31-1271-4a48-9015-db637a4da6db (/health/instance) 2020-03-31 09:09:54,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:54,501 [518] [DEBUG] [app] Ending request: urn:request:671e5a31-1271-4a48-9015-db637a4da6db (/health/instance) 2020-03-31 09:09:54,501 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:09:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:09:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:09:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:06.924497+00:00 (in 9.999138 seconds) 2020-03-31 09:09:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:06 UTC)" (scheduled at 2020-03-31 09:09:56.924497+00:00) 2020-03-31 09:09:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:09:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:06.924497+00:00 (in 9.999138 seconds) 2020-03-31 09:09:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:06 UTC)" (scheduled at 2020-03-31 09:09:56.924497+00:00) 2020-03-31 09:09:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:09:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 56, 926149), True, datetime.datetime(2020, 3, 31, 9, 9, 56, 926149), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:09:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 9, 56, 926149), True, datetime.datetime(2020, 3, 31, 9, 9, 56, 926149), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:09:56,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:09:56,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:09:56,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:56,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:09:56,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:56,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:06 UTC)" executed successfully 2020-03-31 09:09:58,273 [518] [DEBUG] [app] Starting request: urn:request:38f2ee68-c7ba-4372-a9b0-ed55e84f963e (/health/instance) gunicorn-web stdout | 2020-03-31 09:09:58,273 [518] [DEBUG] [app] Starting request: urn:request:38f2ee68-c7ba-4372-a9b0-ed55e84f963e (/health/instance) 2020-03-31 09:09:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:58,278 [538] [DEBUG] [app] Starting request: urn:request:499cbccc-cee6-44ca-a9ba-ffeca55c0f1b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:58,278 [538] [DEBUG] [app] Starting request: urn:request:499cbccc-cee6-44ca-a9ba-ffeca55c0f1b (/v1/_internal_ping) 2020-03-31 09:09:58,279 [538] [DEBUG] [app] Ending request: urn:request:499cbccc-cee6-44ca-a9ba-ffeca55c0f1b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:09:58,279 [538] [DEBUG] [app] Ending request: urn:request:499cbccc-cee6-44ca-a9ba-ffeca55c0f1b (/v1/_internal_ping) 2020-03-31 09:09:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:09:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:09:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:09:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:09:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:09:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:09:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:09:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:09:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:09:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:09:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:09:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:09:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:09:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:09:58,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:58,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:58,309 [518] [DEBUG] [app] Starting request: urn:request:2b135786-d608-41c7-ad1d-3d09dee9aaec (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:58,309 [518] [DEBUG] [app] Starting request: urn:request:2b135786-d608-41c7-ad1d-3d09dee9aaec (/_internal_ping) 2020-03-31 09:09:58,309 [518] [DEBUG] [app] Ending request: urn:request:2b135786-d608-41c7-ad1d-3d09dee9aaec (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:09:58,309 [518] [DEBUG] [app] Ending request: urn:request:2b135786-d608-41c7-ad1d-3d09dee9aaec (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:09:58,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:58,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:09:58,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:58,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:09:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:09:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:09:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:09:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:09:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:09:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:09:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:09:58,323 [536] [DEBUG] [app] Starting request: urn:request:b5a78d37-5e85-4552-9768-60a0e2d9d3c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:58,323 [536] [DEBUG] [app] Starting request: urn:request:b5a78d37-5e85-4552-9768-60a0e2d9d3c9 (/c1/_internal_ping) 2020-03-31 09:09:58,324 [536] [DEBUG] [app] Ending request: urn:request:b5a78d37-5e85-4552-9768-60a0e2d9d3c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:09:58,324 [536] [DEBUG] [app] Ending request: urn:request:b5a78d37-5e85-4552-9768-60a0e2d9d3c9 (/c1/_internal_ping) 2020-03-31 09:09:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:09:58,325 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:09:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:09:58,325 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:09:58,326 [518] [DEBUG] [app] Ending request: urn:request:38f2ee68-c7ba-4372-a9b0-ed55e84f963e (/health/instance) 2020-03-31 09:09:58,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:09:58,326 [518] [DEBUG] [app] Ending request: urn:request:38f2ee68-c7ba-4372-a9b0-ed55e84f963e (/health/instance) 2020-03-31 09:09:58,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:09:58,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:09:58,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:09:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:09:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:09:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:09:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:09:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:59 UTC)" (scheduled at 2020-03-31 09:09:59.560326+00:00) 2020-03-31 09:09:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:09:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:09:59 UTC)" (scheduled at 2020-03-31 09:09:59.560326+00:00) 2020-03-31 09:09:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:09:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:59 UTC)" executed successfully 2020-03-31 09:09:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:51.110610+00:00 (in 51.548894 seconds) namespacegcworker stdout | 2020-03-31 09:09:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:10:59 UTC)" executed successfully 2020-03-31 09:09:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:51.110610+00:00 (in 51.548894 seconds) 2020-03-31 09:10:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:10:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:30.337699+00:00 (in 29.999140 seconds) 2020-03-31 09:10:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:30 UTC)" (scheduled at 2020-03-31 09:10:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:10:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:30.337699+00:00 (in 29.999140 seconds) 2020-03-31 09:10:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:30 UTC)" (scheduled at 2020-03-31 09:10:00.337699+00:00) 2020-03-31 09:10:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 10, 0, 339558), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:10:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 10, 0, 339558), False, 50, 1, 0]) 2020-03-31 09:10:00,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:10:00,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:10:00,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:00,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:10:00,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:00,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:30 UTC)" executed successfully 2020-03-31 09:10:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:10:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:11.440707+00:00 (in 9.999196 seconds) 2020-03-31 09:10:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:11 UTC)" (scheduled at 2020-03-31 09:10:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:10:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:11.440707+00:00 (in 9.999196 seconds) 2020-03-31 09:10:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:11 UTC)" (scheduled at 2020-03-31 09:10:01.440707+00:00) 2020-03-31 09:10:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:10:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:10:01,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:10:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:10:01,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:01,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:10:01,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:01,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:11 UTC)" executed successfully 2020-03-31 09:10:03,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:10:03,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:03,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:24.410927+00:00 (in 21.076626 seconds) 2020-03-31 09:10:03,335 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:33 UTC)" (scheduled at 2020-03-31 09:10:03.330112+00:00) 2020-03-31 09:10:03,335 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:10:03,334 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:24.410927+00:00 (in 21.076626 seconds) 2020-03-31 09:10:03,335 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:33 UTC)" (scheduled at 2020-03-31 09:10:03.330112+00:00) 2020-03-31 09:10:03,335 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:03,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 3, 335535), True, datetime.datetime(2020, 3, 31, 9, 10, 3, 335535), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:10:03,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 3, 335535), True, datetime.datetime(2020, 3, 31, 9, 10, 3, 335535), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:10:03,361 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:10:03,361 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:03,361 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:03,361 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:10:03,361 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:03,361 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:33 UTC)" executed successfully 2020-03-31 09:10:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:10:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:14.323352+00:00 (in 9.998786 seconds) 2020-03-31 09:10:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:14 UTC)" (scheduled at 2020-03-31 09:10:04.323352+00:00) 2020-03-31 09:10:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:10:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:14.323352+00:00 (in 9.998786 seconds) 2020-03-31 09:10:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:14 UTC)" (scheduled at 2020-03-31 09:10:04.323352+00:00) 2020-03-31 09:10:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 4, 325573), True, datetime.datetime(2020, 3, 31, 9, 10, 4, 325573), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:10:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 4, 325573), True, datetime.datetime(2020, 3, 31, 9, 10, 4, 325573), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:10:04,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:10:04,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:04,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:10:04,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:14 UTC)" executed successfully 2020-03-31 09:10:04,446 [525] [DEBUG] [app] Starting request: urn:request:43427ee9-153a-4483-b4ff-67deb420b927 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:04,446 [525] [DEBUG] [app] Starting request: urn:request:43427ee9-153a-4483-b4ff-67deb420b927 (/health/instance) 2020-03-31 09:10:04,448 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:04,448 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:04,452 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:04,452 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:04,453 [547] [DEBUG] [app] Starting request: urn:request:ff8124cf-9814-4d31-a020-b0936d9205c9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:04,453 [547] [DEBUG] [app] Starting request: urn:request:ff8124cf-9814-4d31-a020-b0936d9205c9 (/v1/_internal_ping) 2020-03-31 09:10:04,454 [547] [DEBUG] [app] Ending request: urn:request:ff8124cf-9814-4d31-a020-b0936d9205c9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:04,454 [547] [DEBUG] [app] Ending request: urn:request:ff8124cf-9814-4d31-a020-b0936d9205c9 (/v1/_internal_ping) 2020-03-31 09:10:04,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:10:04,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:04,456 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:04,456 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:04,457 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:04,457 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:04,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:04,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:04,460 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:04,460 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:04,461 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:04,461 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:04,466 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:04,466 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:04,481 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:04,481 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:04,485 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:04,485 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:04,486 [525] [DEBUG] [app] Starting request: urn:request:9d4a9dc8-ad2c-43e7-b574-348836173e67 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:04,486 [525] [DEBUG] [app] Starting request: urn:request:9d4a9dc8-ad2c-43e7-b574-348836173e67 (/_internal_ping) 2020-03-31 09:10:04,487 [525] [DEBUG] [app] Ending request: urn:request:9d4a9dc8-ad2c-43e7-b574-348836173e67 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:04,487 [525] [DEBUG] [app] Ending request: urn:request:9d4a9dc8-ad2c-43e7-b574-348836173e67 (/_internal_ping) 2020-03-31 09:10:04,487 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:10:04,487 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:04,488 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:10:04,488 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:04,491 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:04,492 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:10:04,491 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:04,492 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:10:04,494 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:04,495 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:04,494 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:10:04,495 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:04,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:04,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:04,500 [536] [DEBUG] [app] Starting request: urn:request:5b2ec651-e721-4fb9-9cd8-51309664b109 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:04,500 [536] [DEBUG] [app] Starting request: urn:request:5b2ec651-e721-4fb9-9cd8-51309664b109 (/c1/_internal_ping) 2020-03-31 09:10:04,501 [536] [DEBUG] [app] Ending request: urn:request:5b2ec651-e721-4fb9-9cd8-51309664b109 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:04,501 [536] [DEBUG] [app] Ending request: urn:request:5b2ec651-e721-4fb9-9cd8-51309664b109 (/c1/_internal_ping) 2020-03-31 09:10:04,501 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:04,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:04,501 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:04,503 [525] [DEBUG] [app] Ending request: urn:request:43427ee9-153a-4483-b4ff-67deb420b927 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:10:04,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:04,503 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:04,504 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:10:04,503 [525] [DEBUG] [app] Ending request: urn:request:43427ee9-153a-4483-b4ff-67deb420b927 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:04,503 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:04,504 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:10:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:10:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:16.924497+00:00 (in 9.999188 seconds) 2020-03-31 09:10:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:16 UTC)" (scheduled at 2020-03-31 09:10:06.924497+00:00) 2020-03-31 09:10:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:10:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:16.924497+00:00 (in 9.999188 seconds) 2020-03-31 09:10:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:16 UTC)" (scheduled at 2020-03-31 09:10:06.924497+00:00) 2020-03-31 09:10:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 6, 926168), True, datetime.datetime(2020, 3, 31, 9, 10, 6, 926168), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:10:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 6, 926168), True, datetime.datetime(2020, 3, 31, 9, 10, 6, 926168), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:10:06,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:10:06,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:06,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:06,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:10:06,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:06,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:16 UTC)" executed successfully 2020-03-31 09:10:08,273 [518] [DEBUG] [app] Starting request: urn:request:3192d03b-c236-48cc-848c-7ac2711a9f61 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:08,273 [518] [DEBUG] [app] Starting request: urn:request:3192d03b-c236-48cc-848c-7ac2711a9f61 (/health/instance) 2020-03-31 09:10:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:08,281 [555] [DEBUG] [app] Starting request: urn:request:e7fec9d7-0efc-4dfd-b7e2-d32b86c950c8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:08,281 [555] [DEBUG] [app] Starting request: urn:request:e7fec9d7-0efc-4dfd-b7e2-d32b86c950c8 (/v1/_internal_ping) 2020-03-31 09:10:08,282 [555] [DEBUG] [app] Ending request: urn:request:e7fec9d7-0efc-4dfd-b7e2-d32b86c950c8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:08,282 [555] [DEBUG] [app] Ending request: urn:request:e7fec9d7-0efc-4dfd-b7e2-d32b86c950c8 (/v1/_internal_ping) 2020-03-31 09:10:08,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:10:08,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:10:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:08,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:08,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:08,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:08,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:08,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:08,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:08,314 [518] [DEBUG] [app] Starting request: urn:request:162acad2-ac8d-4df7-96dc-7bbc30f7804f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:08,314 [518] [DEBUG] [app] Starting request: urn:request:162acad2-ac8d-4df7-96dc-7bbc30f7804f (/_internal_ping) 2020-03-31 09:10:08,315 [518] [DEBUG] [app] Ending request: urn:request:162acad2-ac8d-4df7-96dc-7bbc30f7804f (/_internal_ping) 2020-03-31 09:10:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:10:08,315 [518] [DEBUG] [app] Ending request: urn:request:162acad2-ac8d-4df7-96dc-7bbc30f7804f (/_internal_ping) 2020-03-31 09:10:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:10:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:10:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:10:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:08,329 [539] [DEBUG] [app] Starting request: urn:request:b3c6a9f9-5b93-4a79-b249-afeac238989f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:08,329 [539] [DEBUG] [app] Starting request: urn:request:b3c6a9f9-5b93-4a79-b249-afeac238989f (/c1/_internal_ping) 2020-03-31 09:10:08,330 [539] [DEBUG] [app] Ending request: urn:request:b3c6a9f9-5b93-4a79-b249-afeac238989f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:08,330 [539] [DEBUG] [app] Ending request: urn:request:b3c6a9f9-5b93-4a79-b249-afeac238989f (/c1/_internal_ping) 2020-03-31 09:10:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:08,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:08,332 [518] [DEBUG] [app] Ending request: urn:request:3192d03b-c236-48cc-848c-7ac2711a9f61 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:10:08,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:08,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:08,332 [518] [DEBUG] [app] Ending request: urn:request:3192d03b-c236-48cc-848c-7ac2711a9f61 (/health/instance) 2020-03-31 09:10:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:10:08,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:10:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:10:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:21.440707+00:00 (in 9.999290 seconds) 2020-03-31 09:10:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:21 UTC)" (scheduled at 2020-03-31 09:10:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:10:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:21.440707+00:00 (in 9.999290 seconds) 2020-03-31 09:10:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:21 UTC)" (scheduled at 2020-03-31 09:10:11.440707+00:00) 2020-03-31 09:10:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:10:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:10:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:10:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:10:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:10:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:21 UTC)" executed successfully 2020-03-31 09:10:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:10:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:24.323352+00:00 (in 9.998925 seconds) 2020-03-31 09:10:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:24 UTC)" (scheduled at 2020-03-31 09:10:14.323352+00:00) 2020-03-31 09:10:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:10:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:24.323352+00:00 (in 9.998925 seconds) 2020-03-31 09:10:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:24 UTC)" (scheduled at 2020-03-31 09:10:14.323352+00:00) 2020-03-31 09:10:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 14, 325091), True, datetime.datetime(2020, 3, 31, 9, 10, 14, 325091), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:10:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 14, 325091), True, datetime.datetime(2020, 3, 31, 9, 10, 14, 325091), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:10:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:10:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:10:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:24 UTC)" executed successfully 2020-03-31 09:10:14,445 [518] [DEBUG] [app] Starting request: urn:request:9733f7e9-f03c-494f-9501-4c7e8c78855e (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:14,445 [518] [DEBUG] [app] Starting request: urn:request:9733f7e9-f03c-494f-9501-4c7e8c78855e (/health/instance) 2020-03-31 09:10:14,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:14,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:14,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:14,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:14,452 [548] [DEBUG] [app] Starting request: urn:request:d42293ff-5095-4589-bf83-2e3d481255d7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:14,452 [548] [DEBUG] [app] Starting request: urn:request:d42293ff-5095-4589-bf83-2e3d481255d7 (/v1/_internal_ping) 2020-03-31 09:10:14,453 [548] [DEBUG] [app] Ending request: urn:request:d42293ff-5095-4589-bf83-2e3d481255d7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:14,453 [548] [DEBUG] [app] Ending request: urn:request:d42293ff-5095-4589-bf83-2e3d481255d7 (/v1/_internal_ping) 2020-03-31 09:10:14,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:10:14,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:14,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:14,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:10:14,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:14,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:14,456 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:14,456 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:14,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:14,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:14,458 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:14,458 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:14,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:14,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:14,486 [518] [DEBUG] [app] Starting request: urn:request:26b25752-b069-4525-b00f-ccfae7f869a5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:14,486 [518] [DEBUG] [app] Starting request: urn:request:26b25752-b069-4525-b00f-ccfae7f869a5 (/_internal_ping) 2020-03-31 09:10:14,487 [518] [DEBUG] [app] Ending request: urn:request:26b25752-b069-4525-b00f-ccfae7f869a5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:14,487 [518] [DEBUG] [app] Ending request: urn:request:26b25752-b069-4525-b00f-ccfae7f869a5 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:10:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:14,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:14,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:14,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:14,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:14,500 [536] [DEBUG] [app] Starting request: urn:request:e2b80906-3b34-408e-abb4-bec30c3ee562 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:14,500 [536] [DEBUG] [app] Starting request: urn:request:e2b80906-3b34-408e-abb4-bec30c3ee562 (/c1/_internal_ping) 2020-03-31 09:10:14,501 [536] [DEBUG] [app] Ending request: urn:request:e2b80906-3b34-408e-abb4-bec30c3ee562 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:14,501 [536] [DEBUG] [app] Ending request: urn:request:e2b80906-3b34-408e-abb4-bec30c3ee562 (/c1/_internal_ping) 2020-03-31 09:10:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:14,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:14,504 [518] [DEBUG] [app] Ending request: urn:request:9733f7e9-f03c-494f-9501-4c7e8c78855e (/health/instance) 2020-03-31 09:10:14,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:10:14,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:14,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:10:14,504 [518] [DEBUG] [app] Ending request: urn:request:9733f7e9-f03c-494f-9501-4c7e8c78855e (/health/instance) 2020-03-31 09:10:14,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:14,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:10:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:10:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:33.366791+00:00 (in 18.826101 seconds) 2020-03-31 09:10:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:15:14 UTC)" (scheduled at 2020-03-31 09:10:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:10:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:33.366791+00:00 (in 18.826101 seconds) 2020-03-31 09:10:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:15:14 UTC)" (scheduled at 2020-03-31 09:10:14.539408+00:00) 2020-03-31 09:10:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:10:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), u'exportactionlogs/%']) 2020-03-31 09:10:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), True, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:10:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), True, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), 0, u'exportactionlogs/%']) 2020-03-31 09:10:14,561 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), True, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:10:14,561 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), True, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 10, 14, 541628), u'exportactionlogs/%']) 2020-03-31 09:10:14,565 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:10:14,565 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:14,567 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:15:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:10:14,567 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:15:14 UTC)" executed successfully 2020-03-31 09:10:14,567 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:10:14,567 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:10:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:14,569 [166] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:10:14,569 [166] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:14,571 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:10:14,571 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:14,572 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:10:14,572 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:14,573 [166] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:10:14,573 [166] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:10:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:10:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:45.064514+00:00 (in 29.999130 seconds) 2020-03-31 09:10:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:45 UTC)" (scheduled at 2020-03-31 09:10:15.064514+00:00) gcworker stdout | 2020-03-31 09:10:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:45.064514+00:00 (in 29.999130 seconds) 2020-03-31 09:10:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:45 UTC)" (scheduled at 2020-03-31 09:10:15.064514+00:00) 2020-03-31 09:10:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436215, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:10:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436215, 1209600, 500, 1, 0]) 2020-03-31 09:10:15,081 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:10:15,081 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:10:15,081 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:10:15,081 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:45 UTC)" executed successfully 2020-03-31 09:10:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:10:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:26.924497+00:00 (in 9.999224 seconds) 2020-03-31 09:10:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:26 UTC)" (scheduled at 2020-03-31 09:10:16.924497+00:00) 2020-03-31 09:10:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:10:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:26.924497+00:00 (in 9.999224 seconds) 2020-03-31 09:10:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:26 UTC)" (scheduled at 2020-03-31 09:10:16.924497+00:00) 2020-03-31 09:10:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 16, 926000), True, datetime.datetime(2020, 3, 31, 9, 10, 16, 926000), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:10:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 16, 926000), True, datetime.datetime(2020, 3, 31, 9, 10, 16, 926000), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:10:16,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:10:16,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:10:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:26 UTC)" executed successfully 2020-03-31 09:10:18,273 [518] [DEBUG] [app] Starting request: urn:request:80316595-55fc-46d0-8286-62b8d63f9e60 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:18,273 [518] [DEBUG] [app] Starting request: urn:request:80316595-55fc-46d0-8286-62b8d63f9e60 (/health/instance) 2020-03-31 09:10:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:18,278 [554] [DEBUG] [app] Starting request: urn:request:53965a5c-8ac9-4c22-8b5a-1d22e1a1394f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:18,278 [554] [DEBUG] [app] Starting request: urn:request:53965a5c-8ac9-4c22-8b5a-1d22e1a1394f (/v1/_internal_ping) 2020-03-31 09:10:18,279 [554] [DEBUG] [app] Ending request: urn:request:53965a5c-8ac9-4c22-8b5a-1d22e1a1394f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:18,279 [554] [DEBUG] [app] Ending request: urn:request:53965a5c-8ac9-4c22-8b5a-1d22e1a1394f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:18,280 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:10:18,280 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:18,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:18,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:18,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:18,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:18,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:18,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:18,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:18,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:18,308 [518] [DEBUG] [app] Starting request: urn:request:886f423c-d1ed-4db4-8cf2-761d9b4ebf66 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:18,308 [518] [DEBUG] [app] Starting request: urn:request:886f423c-d1ed-4db4-8cf2-761d9b4ebf66 (/_internal_ping) 2020-03-31 09:10:18,309 [518] [DEBUG] [app] Ending request: urn:request:886f423c-d1ed-4db4-8cf2-761d9b4ebf66 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:18,309 [518] [DEBUG] [app] Ending request: urn:request:886f423c-d1ed-4db4-8cf2-761d9b4ebf66 (/_internal_ping) 2020-03-31 09:10:18,309 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:10:18,309 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:18,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:10:18,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:10:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:18,321 [536] [DEBUG] [app] Starting request: urn:request:7c7f58d6-e490-418d-b80b-b6de276c1b09 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:18,321 [536] [DEBUG] [app] Starting request: urn:request:7c7f58d6-e490-418d-b80b-b6de276c1b09 (/c1/_internal_ping) 2020-03-31 09:10:18,322 [536] [DEBUG] [app] Ending request: urn:request:7c7f58d6-e490-418d-b80b-b6de276c1b09 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:18,322 [536] [DEBUG] [app] Ending request: urn:request:7c7f58d6-e490-418d-b80b-b6de276c1b09 (/c1/_internal_ping) 2020-03-31 09:10:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:18,323 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:18,325 [518] [DEBUG] [app] Ending request: urn:request:80316595-55fc-46d0-8286-62b8d63f9e60 (/health/instance) 2020-03-31 09:10:18,325 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:10:18,323 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:18,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.053 49 0.053) gunicorn-web stdout | 2020-03-31 09:10:18,325 [518] [DEBUG] [app] Ending request: urn:request:80316595-55fc-46d0-8286-62b8d63f9e60 (/health/instance) 2020-03-31 09:10:18,325 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:18,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.053 49 0.053) 2020-03-31 09:10:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:10:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:31.440707+00:00 (in 9.999192 seconds) repositoryactioncounter stdout | 2020-03-31 09:10:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:31.440707+00:00 (in 9.999192 seconds) 2020-03-31 09:10:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:31 UTC)" (scheduled at 2020-03-31 09:10:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:10:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:31 UTC)" (scheduled at 2020-03-31 09:10:21.440707+00:00) 2020-03-31 09:10:21,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:10:21,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:10:21,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:10:21,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:10:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:21,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:10:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:21,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:31 UTC)" executed successfully 2020-03-31 09:10:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:10:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:52.113221+00:00 (in 29.998942 seconds) 2020-03-31 09:10:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:52 UTC)" (scheduled at 2020-03-31 09:10:22.113221+00:00) securityworker stdout | 2020-03-31 09:10:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:52.113221+00:00 (in 29.998942 seconds) 2020-03-31 09:10:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:52 UTC)" (scheduled at 2020-03-31 09:10:22.113221+00:00) 2020-03-31 09:10:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:10:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:10:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:10:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:10:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:10:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:10:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:10:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:10:52 UTC)" executed successfully 2020-03-31 09:10:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:10:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:34 UTC)" (scheduled at 2020-03-31 09:10:24.323352+00:00) 2020-03-31 09:10:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:10:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:34 UTC)" (scheduled at 2020-03-31 09:10:24.323352+00:00) 2020-03-31 09:10:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:34.323352+00:00 (in 9.999271 seconds) notificationworker stdout | 2020-03-31 09:10:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:34.323352+00:00 (in 9.999271 seconds) 2020-03-31 09:10:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 24, 324857), True, datetime.datetime(2020, 3, 31, 9, 10, 24, 324857), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:10:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 24, 324857), True, datetime.datetime(2020, 3, 31, 9, 10, 24, 324857), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:10:24,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:24,340 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:10:24,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:24,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:24,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:10:24,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:34 UTC)" executed successfully 2020-03-31 09:10:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:10:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:33.330112+00:00 (in 8.918083 seconds) 2020-03-31 09:10:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:24 UTC)" (scheduled at 2020-03-31 09:10:24.410927+00:00) 2020-03-31 09:10:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:10:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:33.330112+00:00 (in 8.918083 seconds) 2020-03-31 09:10:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:24 UTC)" (scheduled at 2020-03-31 09:10:24.410927+00:00) 2020-03-31 09:10:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:24 UTC)" executed successfully 2020-03-31 09:10:24,445 [518] [DEBUG] [app] Starting request: urn:request:dda39798-4b4a-444a-9d2e-26668dfca7c8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:24,445 [518] [DEBUG] [app] Starting request: urn:request:dda39798-4b4a-444a-9d2e-26668dfca7c8 (/health/instance) 2020-03-31 09:10:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:24,453 [545] [DEBUG] [app] Starting request: urn:request:5d8273e5-cc49-4095-8824-54d321bc3a65 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:24,453 [545] [DEBUG] [app] Starting request: urn:request:5d8273e5-cc49-4095-8824-54d321bc3a65 (/v1/_internal_ping) 2020-03-31 09:10:24,454 [545] [DEBUG] [app] Ending request: urn:request:5d8273e5-cc49-4095-8824-54d321bc3a65 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:24,454 [545] [DEBUG] [app] Ending request: urn:request:5d8273e5-cc49-4095-8824-54d321bc3a65 (/v1/_internal_ping) 2020-03-31 09:10:24,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:10:24,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:24,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:10:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:10:24,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:24,458 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:24,458 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:24,459 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:24,459 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:24,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:24,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:24,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:24,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:24,484 [523] [DEBUG] [app] Starting request: urn:request:8279427c-6a61-4c50-adc6-52d9f10b15eb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:24,484 [523] [DEBUG] [app] Starting request: urn:request:8279427c-6a61-4c50-adc6-52d9f10b15eb (/_internal_ping) 2020-03-31 09:10:24,485 [523] [DEBUG] [app] Ending request: urn:request:8279427c-6a61-4c50-adc6-52d9f10b15eb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:24,485 [523] [DEBUG] [app] Ending request: urn:request:8279427c-6a61-4c50-adc6-52d9f10b15eb (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:10:24,485 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:24,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:24,485 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:24,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:10:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:24,500 [536] [DEBUG] [app] Starting request: urn:request:4d3afab8-a052-4965-acf4-c2c557e4f173 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:24,500 [536] [DEBUG] [app] Starting request: urn:request:4d3afab8-a052-4965-acf4-c2c557e4f173 (/c1/_internal_ping) 2020-03-31 09:10:24,501 [536] [DEBUG] [app] Ending request: urn:request:4d3afab8-a052-4965-acf4-c2c557e4f173 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:24,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:10:24,501 [536] [DEBUG] [app] Ending request: urn:request:4d3afab8-a052-4965-acf4-c2c557e4f173 (/c1/_internal_ping) 2020-03-31 09:10:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:10:24,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:24,504 [518] [DEBUG] [app] Ending request: urn:request:dda39798-4b4a-444a-9d2e-26668dfca7c8 (/health/instance) 2020-03-31 09:10:24,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:24,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:10:24,504 [518] [DEBUG] [app] Ending request: urn:request:dda39798-4b4a-444a-9d2e-26668dfca7c8 (/health/instance) 2020-03-31 09:10:24,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:24,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:10:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:10:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:36.924497+00:00 (in 9.999206 seconds) 2020-03-31 09:10:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:36 UTC)" (scheduled at 2020-03-31 09:10:26.924497+00:00) 2020-03-31 09:10:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:10:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:36.924497+00:00 (in 9.999206 seconds) 2020-03-31 09:10:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:36 UTC)" (scheduled at 2020-03-31 09:10:26.924497+00:00) 2020-03-31 09:10:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 26, 926069), True, datetime.datetime(2020, 3, 31, 9, 10, 26, 926069), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:10:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 26, 926069), True, datetime.datetime(2020, 3, 31, 9, 10, 26, 926069), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:10:26,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:10:26,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:26,945 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:10:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:36 UTC)" executed successfully 2020-03-31 09:10:28,272 [527] [DEBUG] [app] Starting request: urn:request:56de767c-bb7c-4265-b8b2-34a51ac379d3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:28,272 [527] [DEBUG] [app] Starting request: urn:request:56de767c-bb7c-4265-b8b2-34a51ac379d3 (/health/instance) 2020-03-31 09:10:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:28,280 [549] [DEBUG] [app] Starting request: urn:request:d0f26a43-414e-4a91-8d9e-01adf364fbaa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:28,280 [549] [DEBUG] [app] Starting request: urn:request:d0f26a43-414e-4a91-8d9e-01adf364fbaa (/v1/_internal_ping) 2020-03-31 09:10:28,281 [549] [DEBUG] [app] Ending request: urn:request:d0f26a43-414e-4a91-8d9e-01adf364fbaa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:28,281 [549] [DEBUG] [app] Ending request: urn:request:d0f26a43-414e-4a91-8d9e-01adf364fbaa (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:28,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:28,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:10:28,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:28,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:28,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:28,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:28,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:28,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:28,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:28,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:28,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:28,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:28,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:28,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:28,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:28,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:28,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:28,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:28,316 [526] [DEBUG] [app] Starting request: urn:request:52213b27-f6e5-448a-bf10-a9bca9d7a9d1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:28,316 [526] [DEBUG] [app] Starting request: urn:request:52213b27-f6e5-448a-bf10-a9bca9d7a9d1 (/_internal_ping) 2020-03-31 09:10:28,317 [526] [DEBUG] [app] Ending request: urn:request:52213b27-f6e5-448a-bf10-a9bca9d7a9d1 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:10:28,317 [526] [DEBUG] [app] Ending request: urn:request:52213b27-f6e5-448a-bf10-a9bca9d7a9d1 (/_internal_ping) 2020-03-31 09:10:28,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:28,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:10:28,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:28,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:28,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:28,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:28,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:28,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:10:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:10:28,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:28,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:28,333 [539] [DEBUG] [app] Starting request: urn:request:7fc10bed-7380-4ec5-b9af-e098f9e41fa3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:28,333 [539] [DEBUG] [app] Starting request: urn:request:7fc10bed-7380-4ec5-b9af-e098f9e41fa3 (/c1/_internal_ping) 2020-03-31 09:10:28,334 [539] [DEBUG] [app] Ending request: urn:request:7fc10bed-7380-4ec5-b9af-e098f9e41fa3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:28,334 [539] [DEBUG] [app] Ending request: urn:request:7fc10bed-7380-4ec5-b9af-e098f9e41fa3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:28,334 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:28,334 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:10:28,334 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:28,334 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:28,336 [527] [DEBUG] [app] Ending request: urn:request:56de767c-bb7c-4265-b8b2-34a51ac379d3 (/health/instance) 2020-03-31 09:10:28,337 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:28,336 [527] [DEBUG] [app] Ending request: urn:request:56de767c-bb7c-4265-b8b2-34a51ac379d3 (/health/instance) 2020-03-31 09:10:28,337 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:28,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:10:28,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:10:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:10:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:00.337699+00:00 (in 29.998839 seconds) 2020-03-31 09:10:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:00 UTC)" (scheduled at 2020-03-31 09:10:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:10:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:00.337699+00:00 (in 29.998839 seconds) 2020-03-31 09:10:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:00 UTC)" (scheduled at 2020-03-31 09:10:30.337699+00:00) 2020-03-31 09:10:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 10, 30, 339882), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:10:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 10, 30, 339882), False, 50, 1, 0]) 2020-03-31 09:10:30,359 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:10:30,359 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:10:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:10:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:00 UTC)" executed successfully 2020-03-31 09:10:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:10:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:41.440707+00:00 (in 9.999260 seconds) 2020-03-31 09:10:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:41 UTC)" (scheduled at 2020-03-31 09:10:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:10:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:41.440707+00:00 (in 9.999260 seconds) 2020-03-31 09:10:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:41 UTC)" (scheduled at 2020-03-31 09:10:31.440707+00:00) 2020-03-31 09:10:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:10:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:10:31,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:10:31,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:10:31,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:10:31,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:41 UTC)" executed successfully 2020-03-31 09:10:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:10:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:03.330112+00:00 (in 29.999208 seconds) 2020-03-31 09:10:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:03 UTC)" (scheduled at 2020-03-31 09:10:33.330112+00:00) 2020-03-31 09:10:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:10:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:03.330112+00:00 (in 29.999208 seconds) 2020-03-31 09:10:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:03 UTC)" (scheduled at 2020-03-31 09:10:33.330112+00:00) 2020-03-31 09:10:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 33, 331815), True, datetime.datetime(2020, 3, 31, 9, 10, 33, 331815), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:10:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 33, 331815), True, datetime.datetime(2020, 3, 31, 9, 10, 33, 331815), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:10:33,354 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:10:33,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:33,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:33,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:10:33,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:33,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:03 UTC)" executed successfully 2020-03-31 09:10:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:10:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:33 UTC)" (scheduled at 2020-03-31 09:10:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:10:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:33 UTC)" (scheduled at 2020-03-31 09:10:33.366791+00:00) 2020-03-31 09:10:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:10:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:33,371 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:43.539999+00:00 (in 10.172277 seconds) exportactionlogsworker stdout | 2020-03-31 09:10:33,371 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:43.539999+00:00 (in 10.172277 seconds) 2020-03-31 09:10:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 33, 368443), True, datetime.datetime(2020, 3, 31, 9, 10, 33, 368443), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:10:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 33, 368443), True, datetime.datetime(2020, 3, 31, 9, 10, 33, 368443), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:10:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:10:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:10:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:33 UTC)" executed successfully 2020-03-31 09:10:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:10:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:44.323352+00:00 (in 9.997029 seconds) 2020-03-31 09:10:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:44 UTC)" (scheduled at 2020-03-31 09:10:34.323352+00:00) 2020-03-31 09:10:34,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:10:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:44.323352+00:00 (in 9.997029 seconds) 2020-03-31 09:10:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:44 UTC)" (scheduled at 2020-03-31 09:10:34.323352+00:00) 2020-03-31 09:10:34,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 34, 327176), True, datetime.datetime(2020, 3, 31, 9, 10, 34, 327176), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:10:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 34, 327176), True, datetime.datetime(2020, 3, 31, 9, 10, 34, 327176), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:10:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:10:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:10:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:44 UTC)" executed successfully 2020-03-31 09:10:34,446 [526] [DEBUG] [app] Starting request: urn:request:72400049-689b-4ae7-99f8-14538b9f5f33 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:34,446 [526] [DEBUG] [app] Starting request: urn:request:72400049-689b-4ae7-99f8-14538b9f5f33 (/health/instance) 2020-03-31 09:10:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:34,454 [555] [DEBUG] [app] Starting request: urn:request:37b86bcb-edd0-408e-a97c-5ad130a1e70b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:34,454 [555] [DEBUG] [app] Starting request: urn:request:37b86bcb-edd0-408e-a97c-5ad130a1e70b (/v1/_internal_ping) 2020-03-31 09:10:34,455 [555] [DEBUG] [app] Ending request: urn:request:37b86bcb-edd0-408e-a97c-5ad130a1e70b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:34,455 [555] [DEBUG] [app] Ending request: urn:request:37b86bcb-edd0-408e-a97c-5ad130a1e70b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:10:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:34,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:34,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:34,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:34,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:34,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:34,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:34,480 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:34,480 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:34,493 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:34,493 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:34,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:34,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:34,499 [521] [DEBUG] [app] Starting request: urn:request:5b084bd4-2460-4a2d-973d-1abc74125341 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:34,499 [521] [DEBUG] [app] Starting request: urn:request:5b084bd4-2460-4a2d-973d-1abc74125341 (/_internal_ping) 2020-03-31 09:10:34,500 [521] [DEBUG] [app] Ending request: urn:request:5b084bd4-2460-4a2d-973d-1abc74125341 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:34,500 [521] [DEBUG] [app] Ending request: urn:request:5b084bd4-2460-4a2d-973d-1abc74125341 (/_internal_ping) 2020-03-31 09:10:34,501 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:10:34,501 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:10:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:34,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:34,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:34,505 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:34,505 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:34,508 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:34,508 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:10:34,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:10:34,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:34,513 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:34,513 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:34,515 [539] [DEBUG] [app] Starting request: urn:request:86656779-8079-4b50-86db-a8b1e1ac8e17 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:34,515 [539] [DEBUG] [app] Starting request: urn:request:86656779-8079-4b50-86db-a8b1e1ac8e17 (/c1/_internal_ping) 2020-03-31 09:10:34,516 [539] [DEBUG] [app] Ending request: urn:request:86656779-8079-4b50-86db-a8b1e1ac8e17 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:34,516 [539] [DEBUG] [app] Ending request: urn:request:86656779-8079-4b50-86db-a8b1e1ac8e17 (/c1/_internal_ping) 2020-03-31 09:10:34,517 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:34,517 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:34,517 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:10:34,517 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:34,518 [526] [DEBUG] [app] Ending request: urn:request:72400049-689b-4ae7-99f8-14538b9f5f33 (/health/instance) 2020-03-31 09:10:34,518 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:34,518 [526] [DEBUG] [app] Ending request: urn:request:72400049-689b-4ae7-99f8-14538b9f5f33 (/health/instance) 2020-03-31 09:10:34,518 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:34,519 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) gunicorn-web stdout | 2020-03-31 09:10:34,519 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) 2020-03-31 09:10:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:10:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:38.010934+00:00 (in 1.085653 seconds) 2020-03-31 09:10:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:46 UTC)" (scheduled at 2020-03-31 09:10:36.924497+00:00) 2020-03-31 09:10:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:10:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:38.010934+00:00 (in 1.085653 seconds) 2020-03-31 09:10:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:46 UTC)" (scheduled at 2020-03-31 09:10:36.924497+00:00) 2020-03-31 09:10:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 36, 925970), True, datetime.datetime(2020, 3, 31, 9, 10, 36, 925970), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:10:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 36, 925970), True, datetime.datetime(2020, 3, 31, 9, 10, 36, 925970), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:10:36,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:10:36,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:10:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:46 UTC)" executed successfully 2020-03-31 09:10:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:10:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:38 UTC)" (scheduled at 2020-03-31 09:10:38.010934+00:00) 2020-03-31 09:10:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:46.924497+00:00 (in 8.912861 seconds) 2020-03-31 09:10:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:10:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:38 UTC)" (scheduled at 2020-03-31 09:10:38.010934+00:00) 2020-03-31 09:10:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:46.924497+00:00 (in 8.912861 seconds) 2020-03-31 09:10:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:10:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:38 UTC)" executed successfully 2020-03-31 09:10:38,273 [527] [DEBUG] [app] Starting request: urn:request:8acade4d-31aa-4c2d-83e4-0c6609efa636 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:38,273 [527] [DEBUG] [app] Starting request: urn:request:8acade4d-31aa-4c2d-83e4-0c6609efa636 (/health/instance) 2020-03-31 09:10:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:38,280 [545] [DEBUG] [app] Starting request: urn:request:72801de2-6a10-4ccf-a3d3-9051a76b59a8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:38,280 [545] [DEBUG] [app] Starting request: urn:request:72801de2-6a10-4ccf-a3d3-9051a76b59a8 (/v1/_internal_ping) 2020-03-31 09:10:38,281 [545] [DEBUG] [app] Ending request: urn:request:72801de2-6a10-4ccf-a3d3-9051a76b59a8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:38,281 [545] [DEBUG] [app] Ending request: urn:request:72801de2-6a10-4ccf-a3d3-9051a76b59a8 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:38,282 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:10:38,282 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:38,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:38,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:38,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:38,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:38,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:38,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:38,311 [527] [DEBUG] [app] Starting request: urn:request:ecc3a740-b025-4342-8b1e-144a6b7c4324 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:38,311 [527] [DEBUG] [app] Starting request: urn:request:ecc3a740-b025-4342-8b1e-144a6b7c4324 (/_internal_ping) 2020-03-31 09:10:38,312 [527] [DEBUG] [app] Ending request: urn:request:ecc3a740-b025-4342-8b1e-144a6b7c4324 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:38,312 [527] [DEBUG] [app] Ending request: urn:request:ecc3a740-b025-4342-8b1e-144a6b7c4324 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:10:38,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:38,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:38,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:38,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:38,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:38,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:10:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:10:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:38,325 [536] [DEBUG] [app] Starting request: urn:request:b7557f32-cd11-4421-bb5e-0bb5d8724f6a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:38,325 [536] [DEBUG] [app] Starting request: urn:request:b7557f32-cd11-4421-bb5e-0bb5d8724f6a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:38,326 [536] [DEBUG] [app] Ending request: urn:request:b7557f32-cd11-4421-bb5e-0bb5d8724f6a (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:38,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:10:38,326 [536] [DEBUG] [app] Ending request: urn:request:b7557f32-cd11-4421-bb5e-0bb5d8724f6a (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:38,328 [527] [DEBUG] [app] Ending request: urn:request:8acade4d-31aa-4c2d-83e4-0c6609efa636 (/health/instance) 2020-03-31 09:10:38,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:10:38,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:38,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:10:38,328 [527] [DEBUG] [app] Ending request: urn:request:8acade4d-31aa-4c2d-83e4-0c6609efa636 (/health/instance) 2020-03-31 09:10:38,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:38,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:10:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:10:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:51.440707+00:00 (in 9.999067 seconds) 2020-03-31 09:10:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:51 UTC)" (scheduled at 2020-03-31 09:10:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:10:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:51.440707+00:00 (in 9.999067 seconds) 2020-03-31 09:10:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:51 UTC)" (scheduled at 2020-03-31 09:10:41.440707+00:00) 2020-03-31 09:10:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:10:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:10:41,469 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:10:41,469 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:10:41,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:41,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:10:41,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:41,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:51 UTC)" executed successfully 2020-03-31 09:10:43,543 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:10:43,543 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:43,544 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:33.366791+00:00 (in 49.822655 seconds) 2020-03-31 09:10:43,544 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:43 UTC)" (scheduled at 2020-03-31 09:10:43.539999+00:00) 2020-03-31 09:10:43,544 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:43,545 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:10:43,544 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:33.366791+00:00 (in 49.822655 seconds) 2020-03-31 09:10:43,544 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:43 UTC)" (scheduled at 2020-03-31 09:10:43.539999+00:00) 2020-03-31 09:10:43,544 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:43,545 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:43 UTC)" executed successfully 2020-03-31 09:10:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:10:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:49.498966+00:00 (in 5.174580 seconds) 2020-03-31 09:10:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:54 UTC)" (scheduled at 2020-03-31 09:10:44.323352+00:00) 2020-03-31 09:10:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:10:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:49.498966+00:00 (in 5.174580 seconds) 2020-03-31 09:10:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:54 UTC)" (scheduled at 2020-03-31 09:10:44.323352+00:00) 2020-03-31 09:10:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 44, 325349), True, datetime.datetime(2020, 3, 31, 9, 10, 44, 325349), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:10:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 44, 325349), True, datetime.datetime(2020, 3, 31, 9, 10, 44, 325349), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:10:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:10:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:10:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:54 UTC)" executed successfully 2020-03-31 09:10:44,446 [518] [DEBUG] [app] Starting request: urn:request:f33246c5-9148-4e45-9d07-7d802d76f912 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:44,446 [518] [DEBUG] [app] Starting request: urn:request:f33246c5-9148-4e45-9d07-7d802d76f912 (/health/instance) 2020-03-31 09:10:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:44,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:44,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:44,456 [546] [DEBUG] [app] Starting request: urn:request:ec77fd81-6415-4e81-9135-e9b2c3e24c5f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:44,456 [546] [DEBUG] [app] Starting request: urn:request:ec77fd81-6415-4e81-9135-e9b2c3e24c5f (/v1/_internal_ping) 2020-03-31 09:10:44,457 [546] [DEBUG] [app] Ending request: urn:request:ec77fd81-6415-4e81-9135-e9b2c3e24c5f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:44,457 [546] [DEBUG] [app] Ending request: urn:request:ec77fd81-6415-4e81-9135-e9b2c3e24c5f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:44,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:10:44,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:44,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:44,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:10:44,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:10:44,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:44,461 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:44,461 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:44,462 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:44,462 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:44,487 [518] [DEBUG] [app] Starting request: urn:request:bfc5aee0-1c55-417a-9578-65b8c671ca03 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:44,487 [518] [DEBUG] [app] Starting request: urn:request:bfc5aee0-1c55-417a-9578-65b8c671ca03 (/_internal_ping) 2020-03-31 09:10:44,488 [518] [DEBUG] [app] Ending request: urn:request:bfc5aee0-1c55-417a-9578-65b8c671ca03 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:44,488 [518] [DEBUG] [app] Ending request: urn:request:bfc5aee0-1c55-417a-9578-65b8c671ca03 (/_internal_ping) 2020-03-31 09:10:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:10:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:10:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:10:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:10:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:10:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:44,503 [532] [DEBUG] [app] Starting request: urn:request:017a91c1-2ecb-48e0-9b2c-293fef3310e1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:44,503 [532] [DEBUG] [app] Starting request: urn:request:017a91c1-2ecb-48e0-9b2c-293fef3310e1 (/c1/_internal_ping) 2020-03-31 09:10:44,504 [532] [DEBUG] [app] Ending request: urn:request:017a91c1-2ecb-48e0-9b2c-293fef3310e1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:44,504 [532] [DEBUG] [app] Ending request: urn:request:017a91c1-2ecb-48e0-9b2c-293fef3310e1 (/c1/_internal_ping) 2020-03-31 09:10:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:44,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:10:44,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:44,506 [518] [DEBUG] [app] Ending request: urn:request:f33246c5-9148-4e45-9d07-7d802d76f912 (/health/instance) 2020-03-31 09:10:44,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:44,506 [518] [DEBUG] [app] Ending request: urn:request:f33246c5-9148-4e45-9d07-7d802d76f912 (/health/instance) 2020-03-31 09:10:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:10:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:10:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:10:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:15.064514+00:00 (in 29.998944 seconds) 2020-03-31 09:10:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:15 UTC)" (scheduled at 2020-03-31 09:10:45.064514+00:00) gcworker stdout | 2020-03-31 09:10:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:15.064514+00:00 (in 29.998944 seconds) 2020-03-31 09:10:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:15 UTC)" (scheduled at 2020-03-31 09:10:45.064514+00:00) 2020-03-31 09:10:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436245, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:10:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436245, 1209600, 500, 1, 0]) 2020-03-31 09:10:45,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:10:45,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:10:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:10:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:15 UTC)" executed successfully 2020-03-31 09:10:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:10:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:56.924497+00:00 (in 9.998236 seconds) 2020-03-31 09:10:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:56 UTC)" (scheduled at 2020-03-31 09:10:46.924497+00:00) 2020-03-31 09:10:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:10:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:56.924497+00:00 (in 9.998236 seconds) 2020-03-31 09:10:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:56 UTC)" (scheduled at 2020-03-31 09:10:46.924497+00:00) 2020-03-31 09:10:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 46, 926888), True, datetime.datetime(2020, 3, 31, 9, 10, 46, 926888), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:10:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 46, 926888), True, datetime.datetime(2020, 3, 31, 9, 10, 46, 926888), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:10:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:10:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:10:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:10:56 UTC)" executed successfully 2020-03-31 09:10:48,278 [527] [DEBUG] [app] Starting request: urn:request:d5a5f60b-5886-49ac-89f2-88d052bd1418 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:48,278 [527] [DEBUG] [app] Starting request: urn:request:d5a5f60b-5886-49ac-89f2-88d052bd1418 (/health/instance) 2020-03-31 09:10:48,280 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:48,280 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:48,284 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:48,284 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:48,286 [553] [DEBUG] [app] Starting request: urn:request:9e567b51-9396-467c-a45c-175d6290cf0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:48,286 [553] [DEBUG] [app] Starting request: urn:request:9e567b51-9396-467c-a45c-175d6290cf0e (/v1/_internal_ping) 2020-03-31 09:10:48,287 [553] [DEBUG] [app] Ending request: urn:request:9e567b51-9396-467c-a45c-175d6290cf0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:48,287 [553] [DEBUG] [app] Ending request: urn:request:9e567b51-9396-467c-a45c-175d6290cf0e (/v1/_internal_ping) 2020-03-31 09:10:48,288 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:10:48,288 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:48,290 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:48,289 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:48,290 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:10:48,290 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:48,291 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:10:48,289 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:48,290 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:48,292 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:48,291 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:48,292 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:48,308 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:48,308 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:48,321 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:48,321 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:48,326 [527] [DEBUG] [app] Starting request: urn:request:4e3002b6-7b25-46af-89e0-79d3c20fd762 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:48,326 [527] [DEBUG] [app] Starting request: urn:request:4e3002b6-7b25-46af-89e0-79d3c20fd762 (/_internal_ping) 2020-03-31 09:10:48,327 [527] [DEBUG] [app] Ending request: urn:request:4e3002b6-7b25-46af-89e0-79d3c20fd762 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:48,327 [527] [DEBUG] [app] Ending request: urn:request:4e3002b6-7b25-46af-89e0-79d3c20fd762 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:10:48,328 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:48,328 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:10:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:48,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:48,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:48,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:10:48,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:10:48,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:48,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:10:48,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:48,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:48,341 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:48,341 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:48,342 [532] [DEBUG] [app] Starting request: urn:request:4d3e4e4a-c97a-4a23-a0d9-4293f4ca86cc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:48,342 [532] [DEBUG] [app] Starting request: urn:request:4d3e4e4a-c97a-4a23-a0d9-4293f4ca86cc (/c1/_internal_ping) 2020-03-31 09:10:48,343 [532] [DEBUG] [app] Ending request: urn:request:4d3e4e4a-c97a-4a23-a0d9-4293f4ca86cc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:48,343 [532] [DEBUG] [app] Ending request: urn:request:4d3e4e4a-c97a-4a23-a0d9-4293f4ca86cc (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:48,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:48,344 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:48,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:10:48,344 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:48,345 [527] [DEBUG] [app] Ending request: urn:request:d5a5f60b-5886-49ac-89f2-88d052bd1418 (/health/instance) 2020-03-31 09:10:48,345 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:48,345 [527] [DEBUG] [app] Ending request: urn:request:d5a5f60b-5886-49ac-89f2-88d052bd1418 (/health/instance) 2020-03-31 09:10:48,345 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:48,346 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:10:48,346 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:10:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:10:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:54.323352+00:00 (in 4.823573 seconds) 2020-03-31 09:10:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:49 UTC)" (scheduled at 2020-03-31 09:10:49.498966+00:00) 2020-03-31 09:10:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:10:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:54.323352+00:00 (in 4.823573 seconds) 2020-03-31 09:10:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:49 UTC)" (scheduled at 2020-03-31 09:10:49.498966+00:00) 2020-03-31 09:10:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:49 UTC)" executed successfully 2020-03-31 09:10:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:10:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:59.560326+00:00 (in 8.447659 seconds) namespacegcworker stdout | 2020-03-31 09:10:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:10:59.560326+00:00 (in 8.447659 seconds) 2020-03-31 09:10:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:51 UTC)" (scheduled at 2020-03-31 09:10:51.110610+00:00) 2020-03-31 09:10:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:10:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:51 UTC)" (scheduled at 2020-03-31 09:10:51.110610+00:00) 2020-03-31 09:10:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 51, 113884), True, datetime.datetime(2020, 3, 31, 9, 10, 51, 113884), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:10:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 51, 113884), True, datetime.datetime(2020, 3, 31, 9, 10, 51, 113884), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:10:51,131 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:10:51,131 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:10:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:51 UTC)" executed successfully 2020-03-31 09:10:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:10:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:01.440707+00:00 (in 9.999066 seconds) 2020-03-31 09:10:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:01 UTC)" (scheduled at 2020-03-31 09:10:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:10:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:01.440707+00:00 (in 9.999066 seconds) 2020-03-31 09:10:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:01 UTC)" (scheduled at 2020-03-31 09:10:51.440707+00:00) 2020-03-31 09:10:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:10:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:10:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:10:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:10:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:10:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:01 UTC)" executed successfully 2020-03-31 09:10:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:10:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:22.113221+00:00 (in 29.999185 seconds) 2020-03-31 09:10:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:22 UTC)" (scheduled at 2020-03-31 09:10:52.113221+00:00) securityworker stdout | 2020-03-31 09:10:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:22.113221+00:00 (in 29.999185 seconds) 2020-03-31 09:10:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:22 UTC)" (scheduled at 2020-03-31 09:10:52.113221+00:00) 2020-03-31 09:10:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:10:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:10:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:10:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:10:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:10:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:10:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:10:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:22 UTC)" executed successfully 2020-03-31 09:10:54,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:10:54,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:54,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:04.323352+00:00 (in 9.996870 seconds) 2020-03-31 09:10:54,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:04 UTC)" (scheduled at 2020-03-31 09:10:54.323352+00:00) 2020-03-31 09:10:54,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:10:54,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:04.323352+00:00 (in 9.996870 seconds) 2020-03-31 09:10:54,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:04 UTC)" (scheduled at 2020-03-31 09:10:54.323352+00:00) 2020-03-31 09:10:54,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:54,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 54, 327153), True, datetime.datetime(2020, 3, 31, 9, 10, 54, 327153), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:10:54,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 54, 327153), True, datetime.datetime(2020, 3, 31, 9, 10, 54, 327153), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:10:54,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:54,345 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:10:54,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:10:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:04 UTC)" executed successfully 2020-03-31 09:10:54,446 [518] [DEBUG] [app] Starting request: urn:request:0f23fe45-8148-44a6-93e8-ab4605e32e12 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:54,446 [518] [DEBUG] [app] Starting request: urn:request:0f23fe45-8148-44a6-93e8-ab4605e32e12 (/health/instance) 2020-03-31 09:10:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:54,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:54,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:54,455 [546] [DEBUG] [app] Starting request: urn:request:37724605-cd00-4486-9bc2-8841638eb1c0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:54,455 [546] [DEBUG] [app] Starting request: urn:request:37724605-cd00-4486-9bc2-8841638eb1c0 (/v1/_internal_ping) 2020-03-31 09:10:54,456 [546] [DEBUG] [app] Ending request: urn:request:37724605-cd00-4486-9bc2-8841638eb1c0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:54,456 [546] [DEBUG] [app] Ending request: urn:request:37724605-cd00-4486-9bc2-8841638eb1c0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:10:54,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:10:54,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:10:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:54,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:10:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:10:54,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:54,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:54,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:54,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:54,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:54,487 [518] [DEBUG] [app] Starting request: urn:request:ddd335db-b739-492e-9776-cdb04d045d2a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:54,487 [518] [DEBUG] [app] Starting request: urn:request:ddd335db-b739-492e-9776-cdb04d045d2a (/_internal_ping) 2020-03-31 09:10:54,488 [518] [DEBUG] [app] Ending request: urn:request:ddd335db-b739-492e-9776-cdb04d045d2a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:54,488 [518] [DEBUG] [app] Ending request: urn:request:ddd335db-b739-492e-9776-cdb04d045d2a (/_internal_ping) 2020-03-31 09:10:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:10:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:10:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:54,503 [532] [DEBUG] [app] Starting request: urn:request:1bae7a33-87a2-421d-9c2a-f68b36218e39 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:54,503 [532] [DEBUG] [app] Starting request: urn:request:1bae7a33-87a2-421d-9c2a-f68b36218e39 (/c1/_internal_ping) 2020-03-31 09:10:54,504 [532] [DEBUG] [app] Ending request: urn:request:1bae7a33-87a2-421d-9c2a-f68b36218e39 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:54,504 [532] [DEBUG] [app] Ending request: urn:request:1bae7a33-87a2-421d-9c2a-f68b36218e39 (/c1/_internal_ping) 2020-03-31 09:10:54,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:54,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:54,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:10:54,506 [518] [DEBUG] [app] Ending request: urn:request:0f23fe45-8148-44a6-93e8-ab4605e32e12 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:10:54,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:54,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:54,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:10:54,506 [518] [DEBUG] [app] Ending request: urn:request:0f23fe45-8148-44a6-93e8-ab4605e32e12 (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:54,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:54,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:10:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:10:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:06.924497+00:00 (in 9.999315 seconds) 2020-03-31 09:10:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:06 UTC)" (scheduled at 2020-03-31 09:10:56.924497+00:00) 2020-03-31 09:10:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:10:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:06.924497+00:00 (in 9.999315 seconds) 2020-03-31 09:10:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:06 UTC)" (scheduled at 2020-03-31 09:10:56.924497+00:00) 2020-03-31 09:10:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:10:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 56, 926034), True, datetime.datetime(2020, 3, 31, 9, 10, 56, 926034), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:10:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 10, 56, 926034), True, datetime.datetime(2020, 3, 31, 9, 10, 56, 926034), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:10:56,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:10:56,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:10:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:56,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:10:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:56,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:06 UTC)" executed successfully 2020-03-31 09:10:58,273 [527] [DEBUG] [app] Starting request: urn:request:96b8ab39-4d51-470c-90f4-42b9a09c28ff (/health/instance) gunicorn-web stdout | 2020-03-31 09:10:58,273 [527] [DEBUG] [app] Starting request: urn:request:96b8ab39-4d51-470c-90f4-42b9a09c28ff (/health/instance) 2020-03-31 09:10:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:58,280 [549] [DEBUG] [app] Starting request: urn:request:e63e4432-d963-412a-9adc-e520e0766d0a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:58,280 [549] [DEBUG] [app] Starting request: urn:request:e63e4432-d963-412a-9adc-e520e0766d0a (/v1/_internal_ping) 2020-03-31 09:10:58,282 [549] [DEBUG] [app] Ending request: urn:request:e63e4432-d963-412a-9adc-e520e0766d0a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:10:58,282 [549] [DEBUG] [app] Ending request: urn:request:e63e4432-d963-412a-9adc-e520e0766d0a (/v1/_internal_ping) 2020-03-31 09:10:58,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:10:58,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:10:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:10:58,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:10:58,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:10:58,287 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:10:58,287 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:10:58,288 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:10:58,288 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:10:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:10:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:10:58,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:10:58,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:10:58,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:58,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:58,311 [527] [DEBUG] [app] Starting request: urn:request:e542ec9c-2dfa-416b-a6bc-d4c776a7ae4a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:58,311 [527] [DEBUG] [app] Starting request: urn:request:e542ec9c-2dfa-416b-a6bc-d4c776a7ae4a (/_internal_ping) 2020-03-31 09:10:58,312 [527] [DEBUG] [app] Ending request: urn:request:e542ec9c-2dfa-416b-a6bc-d4c776a7ae4a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:10:58,312 [527] [DEBUG] [app] Ending request: urn:request:e542ec9c-2dfa-416b-a6bc-d4c776a7ae4a (/_internal_ping) 2020-03-31 09:10:58,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:10:58,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:10:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:10:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:10:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:10:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:10:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:10:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:10:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:10:58,329 [539] [DEBUG] [app] Starting request: urn:request:6001b7a6-ccfb-4947-84f5-d5c6a6d06b1f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:58,329 [539] [DEBUG] [app] Starting request: urn:request:6001b7a6-ccfb-4947-84f5-d5c6a6d06b1f (/c1/_internal_ping) 2020-03-31 09:10:58,330 [539] [DEBUG] [app] Ending request: urn:request:6001b7a6-ccfb-4947-84f5-d5c6a6d06b1f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:10:58,330 [539] [DEBUG] [app] Ending request: urn:request:6001b7a6-ccfb-4947-84f5-d5c6a6d06b1f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:10:58,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:10:58,330 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:10:58,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:10:58,330 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:10:58,332 [527] [DEBUG] [app] Ending request: urn:request:96b8ab39-4d51-470c-90f4-42b9a09c28ff (/health/instance) 2020-03-31 09:10:58,332 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:10:58,332 [527] [DEBUG] [app] Ending request: urn:request:96b8ab39-4d51-470c-90f4-42b9a09c28ff (/health/instance) 2020-03-31 09:10:58,332 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:10:58,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:10:58,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:10:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:10:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:10:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:10:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:10:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:51.110610+00:00 (in 51.549506 seconds) namespacegcworker stdout | 2020-03-31 09:10:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:51.110610+00:00 (in 51.549506 seconds) 2020-03-31 09:10:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:59 UTC)" (scheduled at 2020-03-31 09:10:59.560326+00:00) 2020-03-31 09:10:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:10:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:59 UTC)" (scheduled at 2020-03-31 09:10:59.560326+00:00) 2020-03-31 09:10:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:10:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:11:59 UTC)" executed successfully 2020-03-31 09:11:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:11:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:30 UTC)" (scheduled at 2020-03-31 09:11:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:11:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:30 UTC)" (scheduled at 2020-03-31 09:11:00.337699+00:00) 2020-03-31 09:11:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 11, 0, 339492), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:11:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 11, 0, 339492), False, 50, 1, 0]) 2020-03-31 09:11:00,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:30.337699+00:00 (in 29.998869 seconds) buildlogsarchiver stdout | 2020-03-31 09:11:00,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:30.337699+00:00 (in 29.998869 seconds) 2020-03-31 09:11:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:11:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:11:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:11:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:30 UTC)" executed successfully 2020-03-31 09:11:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:11:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:11.440707+00:00 (in 9.999151 seconds) 2020-03-31 09:11:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:11 UTC)" (scheduled at 2020-03-31 09:11:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:11:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:11.440707+00:00 (in 9.999151 seconds) 2020-03-31 09:11:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:11 UTC)" (scheduled at 2020-03-31 09:11:01.440707+00:00) 2020-03-31 09:11:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:11:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:11:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:11:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:11:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:11:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:11 UTC)" executed successfully 2020-03-31 09:11:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:11:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:24.410927+00:00 (in 21.079615 seconds) 2020-03-31 09:11:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:33 UTC)" (scheduled at 2020-03-31 09:11:03.330112+00:00) 2020-03-31 09:11:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:11:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:24.410927+00:00 (in 21.079615 seconds) 2020-03-31 09:11:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:33 UTC)" (scheduled at 2020-03-31 09:11:03.330112+00:00) 2020-03-31 09:11:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 3, 332279), True, datetime.datetime(2020, 3, 31, 9, 11, 3, 332279), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:11:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 3, 332279), True, datetime.datetime(2020, 3, 31, 9, 11, 3, 332279), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:11:03,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:03,348 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:11:03,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:03,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:11:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:33 UTC)" executed successfully 2020-03-31 09:11:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:11:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:14.323352+00:00 (in 9.999203 seconds) 2020-03-31 09:11:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:14 UTC)" (scheduled at 2020-03-31 09:11:04.323352+00:00) 2020-03-31 09:11:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:11:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:14.323352+00:00 (in 9.999203 seconds) 2020-03-31 09:11:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:14 UTC)" (scheduled at 2020-03-31 09:11:04.323352+00:00) 2020-03-31 09:11:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 4, 325233), True, datetime.datetime(2020, 3, 31, 9, 11, 4, 325233), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:11:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 4, 325233), True, datetime.datetime(2020, 3, 31, 9, 11, 4, 325233), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:11:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:11:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:11:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:14 UTC)" executed successfully 2020-03-31 09:11:04,446 [518] [DEBUG] [app] Starting request: urn:request:edfc7d6f-3867-4a90-956b-a7ba44098feb (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:04,446 [518] [DEBUG] [app] Starting request: urn:request:edfc7d6f-3867-4a90-956b-a7ba44098feb (/health/instance) 2020-03-31 09:11:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:04,454 [552] [DEBUG] [app] Starting request: urn:request:568d1e83-4ec1-442c-a47e-df783bd28e2d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:04,454 [552] [DEBUG] [app] Starting request: urn:request:568d1e83-4ec1-442c-a47e-df783bd28e2d (/v1/_internal_ping) 2020-03-31 09:11:04,455 [552] [DEBUG] [app] Ending request: urn:request:568d1e83-4ec1-442c-a47e-df783bd28e2d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:04,455 [552] [DEBUG] [app] Ending request: urn:request:568d1e83-4ec1-442c-a47e-df783bd28e2d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:11:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:04,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:04,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:04,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:04,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:04,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:04,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:04,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:04,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:04,486 [518] [DEBUG] [app] Starting request: urn:request:32ff0dc4-3bd5-43c9-b8d5-0df3ff89d614 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:04,486 [518] [DEBUG] [app] Starting request: urn:request:32ff0dc4-3bd5-43c9-b8d5-0df3ff89d614 (/_internal_ping) 2020-03-31 09:11:04,487 [518] [DEBUG] [app] Ending request: urn:request:32ff0dc4-3bd5-43c9-b8d5-0df3ff89d614 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:04,487 [518] [DEBUG] [app] Ending request: urn:request:32ff0dc4-3bd5-43c9-b8d5-0df3ff89d614 (/_internal_ping) 2020-03-31 09:11:04,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:04,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:11:04,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:04,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:11:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:04,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:11:04,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:11:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:11:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:11:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:04,499 [536] [DEBUG] [app] Starting request: urn:request:33bc0766-605e-4eca-be79-3fccf1e6428e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:04,499 [536] [DEBUG] [app] Starting request: urn:request:33bc0766-605e-4eca-be79-3fccf1e6428e (/c1/_internal_ping) 2020-03-31 09:11:04,500 [536] [DEBUG] [app] Ending request: urn:request:33bc0766-605e-4eca-be79-3fccf1e6428e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:04,500 [536] [DEBUG] [app] Ending request: urn:request:33bc0766-605e-4eca-be79-3fccf1e6428e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:04,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:11:04,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:04,502 [518] [DEBUG] [app] Ending request: urn:request:edfc7d6f-3867-4a90-956b-a7ba44098feb (/health/instance) 2020-03-31 09:11:04,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:04,502 [518] [DEBUG] [app] Ending request: urn:request:edfc7d6f-3867-4a90-956b-a7ba44098feb (/health/instance) 2020-03-31 09:11:04,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:04,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:11:04,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:11:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:11:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:16.924497+00:00 (in 9.999011 seconds) storagereplication stdout | 2020-03-31 09:11:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:16.924497+00:00 (in 9.999011 seconds) 2020-03-31 09:11:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:16 UTC)" (scheduled at 2020-03-31 09:11:06.924497+00:00) storagereplication stdout | 2020-03-31 09:11:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:16 UTC)" (scheduled at 2020-03-31 09:11:06.924497+00:00) 2020-03-31 09:11:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:11:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 6, 927748), True, datetime.datetime(2020, 3, 31, 9, 11, 6, 927748), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:11:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 6, 927748), True, datetime.datetime(2020, 3, 31, 9, 11, 6, 927748), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:11:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:11:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:11:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:16 UTC)" executed successfully 2020-03-31 09:11:08,273 [518] [DEBUG] [app] Starting request: urn:request:52ed566e-b343-48e3-a4ab-e4c5f89968db (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:08,273 [518] [DEBUG] [app] Starting request: urn:request:52ed566e-b343-48e3-a4ab-e4c5f89968db (/health/instance) 2020-03-31 09:11:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:08,278 [550] [DEBUG] [app] Starting request: urn:request:c548c48f-754f-4f92-b367-369fe3b8c104 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:08,278 [550] [DEBUG] [app] Starting request: urn:request:c548c48f-754f-4f92-b367-369fe3b8c104 (/v1/_internal_ping) 2020-03-31 09:11:08,279 [550] [DEBUG] [app] Ending request: urn:request:c548c48f-754f-4f92-b367-369fe3b8c104 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:08,279 [550] [DEBUG] [app] Ending request: urn:request:c548c48f-754f-4f92-b367-369fe3b8c104 (/v1/_internal_ping) 2020-03-31 09:11:08,280 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:11:08,280 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:08,283 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:08,283 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:08,284 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:08,284 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:08,285 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:08,285 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:08,314 [521] [DEBUG] [app] Starting request: urn:request:46ab667c-f195-4076-a100-edf1bb5245bf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:08,314 [521] [DEBUG] [app] Starting request: urn:request:46ab667c-f195-4076-a100-edf1bb5245bf (/_internal_ping) 2020-03-31 09:11:08,315 [521] [DEBUG] [app] Ending request: urn:request:46ab667c-f195-4076-a100-edf1bb5245bf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:08,315 [521] [DEBUG] [app] Ending request: urn:request:46ab667c-f195-4076-a100-edf1bb5245bf (/_internal_ping) 2020-03-31 09:11:08,315 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:11:08,315 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:11:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:11:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:11:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:08,329 [539] [DEBUG] [app] Starting request: urn:request:cd3cfcea-6cf9-47cb-a060-a57af70eca04 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:08,329 [539] [DEBUG] [app] Starting request: urn:request:cd3cfcea-6cf9-47cb-a060-a57af70eca04 (/c1/_internal_ping) 2020-03-31 09:11:08,330 [539] [DEBUG] [app] Ending request: urn:request:cd3cfcea-6cf9-47cb-a060-a57af70eca04 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:08,330 [539] [DEBUG] [app] Ending request: urn:request:cd3cfcea-6cf9-47cb-a060-a57af70eca04 (/c1/_internal_ping) 2020-03-31 09:11:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:08,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:11:08,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:08,333 [518] [DEBUG] [app] Ending request: urn:request:52ed566e-b343-48e3-a4ab-e4c5f89968db (/health/instance) 2020-03-31 09:11:08,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:08,333 [518] [DEBUG] [app] Ending request: urn:request:52ed566e-b343-48e3-a4ab-e4c5f89968db (/health/instance) 2020-03-31 09:11:08,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:11:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:11:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:11:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:21.440707+00:00 (in 9.999080 seconds) 2020-03-31 09:11:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:21 UTC)" (scheduled at 2020-03-31 09:11:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:11:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:21.440707+00:00 (in 9.999080 seconds) 2020-03-31 09:11:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:21 UTC)" (scheduled at 2020-03-31 09:11:11.440707+00:00) 2020-03-31 09:11:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:11:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:11:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:11:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:11:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:11:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:21 UTC)" executed successfully 2020-03-31 09:11:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:11:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:24.323352+00:00 (in 9.999130 seconds) 2020-03-31 09:11:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:24 UTC)" (scheduled at 2020-03-31 09:11:14.323352+00:00) 2020-03-31 09:11:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:11:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:24.323352+00:00 (in 9.999130 seconds) 2020-03-31 09:11:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:24 UTC)" (scheduled at 2020-03-31 09:11:14.323352+00:00) 2020-03-31 09:11:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 14, 325281), True, datetime.datetime(2020, 3, 31, 9, 11, 14, 325281), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:11:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 14, 325281), True, datetime.datetime(2020, 3, 31, 9, 11, 14, 325281), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:11:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:11:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:11:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:24 UTC)" executed successfully 2020-03-31 09:11:14,446 [518] [DEBUG] [app] Starting request: urn:request:ced0ed53-6ed7-4273-94a3-436c60daf8b4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:14,446 [518] [DEBUG] [app] Starting request: urn:request:ced0ed53-6ed7-4273-94a3-436c60daf8b4 (/health/instance) 2020-03-31 09:11:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:14,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:14,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:14,455 [556] [DEBUG] [app] Starting request: urn:request:9c49d1fd-7c52-4f71-b653-b53720fb76b9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:14,455 [556] [DEBUG] [app] Starting request: urn:request:9c49d1fd-7c52-4f71-b653-b53720fb76b9 (/v1/_internal_ping) 2020-03-31 09:11:14,456 [556] [DEBUG] [app] Ending request: urn:request:9c49d1fd-7c52-4f71-b653-b53720fb76b9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:14,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:11:14,456 [556] [DEBUG] [app] Ending request: urn:request:9c49d1fd-7c52-4f71-b653-b53720fb76b9 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:14,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:11:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:14,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:14,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:14,489 [518] [DEBUG] [app] Starting request: urn:request:5eaf5aed-8827-4470-ace3-cf17bc163e65 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:14,489 [518] [DEBUG] [app] Starting request: urn:request:5eaf5aed-8827-4470-ace3-cf17bc163e65 (/_internal_ping) 2020-03-31 09:11:14,490 [518] [DEBUG] [app] Ending request: urn:request:5eaf5aed-8827-4470-ace3-cf17bc163e65 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:14,490 [518] [DEBUG] [app] Ending request: urn:request:5eaf5aed-8827-4470-ace3-cf17bc163e65 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:11:14,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:14,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:14,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:14,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:11:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:11:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:11:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:11:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:14,504 [537] [DEBUG] [app] Starting request: urn:request:12004842-4297-4d19-8cfc-450205a36572 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:14,504 [537] [DEBUG] [app] Starting request: urn:request:12004842-4297-4d19-8cfc-450205a36572 (/c1/_internal_ping) 2020-03-31 09:11:14,505 [537] [DEBUG] [app] Ending request: urn:request:12004842-4297-4d19-8cfc-450205a36572 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:14,505 [537] [DEBUG] [app] Ending request: urn:request:12004842-4297-4d19-8cfc-450205a36572 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:14,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:11:14,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:14,507 [518] [DEBUG] [app] Ending request: urn:request:ced0ed53-6ed7-4273-94a3-436c60daf8b4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:14,507 [518] [DEBUG] [app] Ending request: urn:request:ced0ed53-6ed7-4273-94a3-436c60daf8b4 (/health/instance) 2020-03-31 09:11:14,507 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:11:14,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:11:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:11:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:11:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:45.064514+00:00 (in 29.998974 seconds) 2020-03-31 09:11:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:45 UTC)" (scheduled at 2020-03-31 09:11:15.064514+00:00) gcworker stdout | 2020-03-31 09:11:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:45.064514+00:00 (in 29.998974 seconds) 2020-03-31 09:11:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:45 UTC)" (scheduled at 2020-03-31 09:11:15.064514+00:00) 2020-03-31 09:11:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436275, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:11:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436275, 1209600, 500, 1, 0]) 2020-03-31 09:11:15,086 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:11:15,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:11:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:11:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:45 UTC)" executed successfully 2020-03-31 09:11:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:11:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:26.924497+00:00 (in 9.999251 seconds) 2020-03-31 09:11:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:26 UTC)" (scheduled at 2020-03-31 09:11:16.924497+00:00) 2020-03-31 09:11:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:11:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:26.924497+00:00 (in 9.999251 seconds) 2020-03-31 09:11:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:26 UTC)" (scheduled at 2020-03-31 09:11:16.924497+00:00) 2020-03-31 09:11:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 16, 925971), True, datetime.datetime(2020, 3, 31, 9, 11, 16, 925971), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:11:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 16, 925971), True, datetime.datetime(2020, 3, 31, 9, 11, 16, 925971), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:11:16,953 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:11:16,953 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:16,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:16,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:11:16,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:16,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:26 UTC)" executed successfully 2020-03-31 09:11:18,273 [527] [DEBUG] [app] Starting request: urn:request:245f6978-3f8c-4345-8986-669d8cde1c13 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:18,273 [527] [DEBUG] [app] Starting request: urn:request:245f6978-3f8c-4345-8986-669d8cde1c13 (/health/instance) 2020-03-31 09:11:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:18,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:18,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:18,281 [555] [DEBUG] [app] Starting request: urn:request:77feafd1-9f1a-471b-adf2-0861655e5711 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:18,281 [555] [DEBUG] [app] Starting request: urn:request:77feafd1-9f1a-471b-adf2-0861655e5711 (/v1/_internal_ping) 2020-03-31 09:11:18,282 [555] [DEBUG] [app] Ending request: urn:request:77feafd1-9f1a-471b-adf2-0861655e5711 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:18,282 [555] [DEBUG] [app] Ending request: urn:request:77feafd1-9f1a-471b-adf2-0861655e5711 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:18,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:11:18,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:18,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:18,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:18,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:18,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:18,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:18,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:18,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:18,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:18,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:18,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:18,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:18,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:18,312 [527] [DEBUG] [app] Starting request: urn:request:99021c16-a307-4cce-ad7f-0de064cd1995 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:18,312 [527] [DEBUG] [app] Starting request: urn:request:99021c16-a307-4cce-ad7f-0de064cd1995 (/_internal_ping) 2020-03-31 09:11:18,313 [527] [DEBUG] [app] Ending request: urn:request:99021c16-a307-4cce-ad7f-0de064cd1995 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:18,313 [527] [DEBUG] [app] Ending request: urn:request:99021c16-a307-4cce-ad7f-0de064cd1995 (/_internal_ping) 2020-03-31 09:11:18,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:11:18,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:18,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:11:18,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:18,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:18,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:11:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:11:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:11:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:18,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:18,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:18,327 [543] [DEBUG] [app] Starting request: urn:request:92673002-1d56-4622-a14d-c36b247f5209 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:18,327 [543] [DEBUG] [app] Starting request: urn:request:92673002-1d56-4622-a14d-c36b247f5209 (/c1/_internal_ping) 2020-03-31 09:11:18,328 [543] [DEBUG] [app] Ending request: urn:request:92673002-1d56-4622-a14d-c36b247f5209 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:18,328 [543] [DEBUG] [app] Ending request: urn:request:92673002-1d56-4622-a14d-c36b247f5209 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:18,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:11:18,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:18,330 [527] [DEBUG] [app] Ending request: urn:request:245f6978-3f8c-4345-8986-669d8cde1c13 (/health/instance) 2020-03-31 09:11:18,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:18,330 [527] [DEBUG] [app] Ending request: urn:request:245f6978-3f8c-4345-8986-669d8cde1c13 (/health/instance) 2020-03-31 09:11:18,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:18,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:11:18,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:11:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:11:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:31 UTC)" (scheduled at 2020-03-31 09:11:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:11:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:31 UTC)" (scheduled at 2020-03-31 09:11:21.440707+00:00) 2020-03-31 09:11:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:31.440707+00:00 (in 9.999076 seconds) repositoryactioncounter stdout | 2020-03-31 09:11:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:31.440707+00:00 (in 9.999076 seconds) 2020-03-31 09:11:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:11:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:11:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:11:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:11:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:11:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:31 UTC)" executed successfully 2020-03-31 09:11:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:11:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:52.113221+00:00 (in 29.998976 seconds) 2020-03-31 09:11:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:52 UTC)" (scheduled at 2020-03-31 09:11:22.113221+00:00) securityworker stdout | 2020-03-31 09:11:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:52.113221+00:00 (in 29.998976 seconds) 2020-03-31 09:11:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:52 UTC)" (scheduled at 2020-03-31 09:11:22.113221+00:00) 2020-03-31 09:11:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:11:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:11:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:11:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:11:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:11:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:11:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:11:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:11:52 UTC)" executed successfully 2020-03-31 09:11:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:11:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:34.323352+00:00 (in 9.999191 seconds) 2020-03-31 09:11:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:34 UTC)" (scheduled at 2020-03-31 09:11:24.323352+00:00) notificationworker stdout | 2020-03-31 09:11:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:34.323352+00:00 (in 9.999191 seconds) 2020-03-31 09:11:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:34 UTC)" (scheduled at 2020-03-31 09:11:24.323352+00:00) 2020-03-31 09:11:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:11:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 24, 325223), True, datetime.datetime(2020, 3, 31, 9, 11, 24, 325223), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:11:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 24, 325223), True, datetime.datetime(2020, 3, 31, 9, 11, 24, 325223), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:11:24,352 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:11:24,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:24,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:24,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:11:24,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:24,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:34 UTC)" executed successfully 2020-03-31 09:11:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:11:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:33.330112+00:00 (in 8.918333 seconds) 2020-03-31 09:11:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:24 UTC)" (scheduled at 2020-03-31 09:11:24.410927+00:00) 2020-03-31 09:11:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:11:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:11:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:33.330112+00:00 (in 8.918333 seconds) 2020-03-31 09:11:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:24 UTC)" (scheduled at 2020-03-31 09:11:24.410927+00:00) 2020-03-31 09:11:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:11:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:24 UTC)" executed successfully 2020-03-31 09:11:24,446 [527] [DEBUG] [app] Starting request: urn:request:181f04dc-735d-445e-9485-a2af035419e9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:24,446 [527] [DEBUG] [app] Starting request: urn:request:181f04dc-735d-445e-9485-a2af035419e9 (/health/instance) 2020-03-31 09:11:24,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:24,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:24,455 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:24,455 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:24,457 [538] [DEBUG] [app] Starting request: urn:request:80ba6649-dabd-4cf7-a087-49239f25cc3a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:24,457 [538] [DEBUG] [app] Starting request: urn:request:80ba6649-dabd-4cf7-a087-49239f25cc3a (/v1/_internal_ping) 2020-03-31 09:11:24,458 [538] [DEBUG] [app] Ending request: urn:request:80ba6649-dabd-4cf7-a087-49239f25cc3a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:24,458 [538] [DEBUG] [app] Ending request: urn:request:80ba6649-dabd-4cf7-a087-49239f25cc3a (/v1/_internal_ping) 2020-03-31 09:11:24,459 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:11:24,459 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:24,462 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:24,460 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:24,461 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:11:24,462 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:11:24,460 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:24,461 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:24,466 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:24,466 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:24,467 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:24,467 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:24,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:24,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:24,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:24,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:24,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:24,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:24,492 [525] [DEBUG] [app] Starting request: urn:request:22578f10-4a6f-4ecf-9a83-78906226235d (/_internal_ping) 2020-03-31 09:11:24,494 [525] [DEBUG] [app] Ending request: urn:request:22578f10-4a6f-4ecf-9a83-78906226235d (/_internal_ping) 2020-03-31 09:11:24,494 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:24,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:11:24,492 [525] [DEBUG] [app] Starting request: urn:request:22578f10-4a6f-4ecf-9a83-78906226235d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:24,494 [525] [DEBUG] [app] Ending request: urn:request:22578f10-4a6f-4ecf-9a83-78906226235d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:24,494 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:24,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:24,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:24,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:11:24,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:24,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:24,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:24,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:11:24,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:24,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:24,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:24,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:24,510 [543] [DEBUG] [app] Starting request: urn:request:ab866d79-2d1a-49a7-9ce1-95d2bfca2bc7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:24,510 [543] [DEBUG] [app] Starting request: urn:request:ab866d79-2d1a-49a7-9ce1-95d2bfca2bc7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:24,512 [543] [DEBUG] [app] Ending request: urn:request:ab866d79-2d1a-49a7-9ce1-95d2bfca2bc7 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:24,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:24,512 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:11:24,512 [543] [DEBUG] [app] Ending request: urn:request:ab866d79-2d1a-49a7-9ce1-95d2bfca2bc7 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:24,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:24,514 [527] [DEBUG] [app] Ending request: urn:request:181f04dc-735d-445e-9485-a2af035419e9 (/health/instance) 2020-03-31 09:11:24,515 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:24,515 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-verbs stdout | 2020-03-31 09:11:24,512 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:24,514 [527] [DEBUG] [app] Ending request: urn:request:181f04dc-735d-445e-9485-a2af035419e9 (/health/instance) 2020-03-31 09:11:24,515 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:24,515 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:11:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:11:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:36.924497+00:00 (in 9.999240 seconds) storagereplication stdout | 2020-03-31 09:11:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:36.924497+00:00 (in 9.999240 seconds) 2020-03-31 09:11:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:36 UTC)" (scheduled at 2020-03-31 09:11:26.924497+00:00) storagereplication stdout | 2020-03-31 09:11:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:36 UTC)" (scheduled at 2020-03-31 09:11:26.924497+00:00) 2020-03-31 09:11:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:11:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 26, 927766), True, datetime.datetime(2020, 3, 31, 9, 11, 26, 927766), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:11:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 26, 927766), True, datetime.datetime(2020, 3, 31, 9, 11, 26, 927766), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:11:26,950 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:11:26,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:26,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:26,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:11:26,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:26,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:36 UTC)" executed successfully 2020-03-31 09:11:28,273 [526] [DEBUG] [app] Starting request: urn:request:17e8c4f1-51d4-4126-a5ab-ab3432974a47 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:28,273 [526] [DEBUG] [app] Starting request: urn:request:17e8c4f1-51d4-4126-a5ab-ab3432974a47 (/health/instance) 2020-03-31 09:11:28,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:28,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:28,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:28,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:28,280 [538] [DEBUG] [app] Starting request: urn:request:ff6e5fcf-bbb7-4b6c-867f-3ba7f99dbcef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:28,280 [538] [DEBUG] [app] Starting request: urn:request:ff6e5fcf-bbb7-4b6c-867f-3ba7f99dbcef (/v1/_internal_ping) 2020-03-31 09:11:28,281 [538] [DEBUG] [app] Ending request: urn:request:ff6e5fcf-bbb7-4b6c-867f-3ba7f99dbcef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:28,281 [538] [DEBUG] [app] Ending request: urn:request:ff6e5fcf-bbb7-4b6c-867f-3ba7f99dbcef (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:28,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:28,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:11:28,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:28,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:28,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:28,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:11:28,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:11:28,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:28,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:28,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:28,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:28,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:28,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:28,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:28,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:28,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:28,312 [526] [DEBUG] [app] Starting request: urn:request:c786f021-2096-44b2-ab62-fefdab147f35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:28,312 [526] [DEBUG] [app] Starting request: urn:request:c786f021-2096-44b2-ab62-fefdab147f35 (/_internal_ping) 2020-03-31 09:11:28,313 [526] [DEBUG] [app] Ending request: urn:request:c786f021-2096-44b2-ab62-fefdab147f35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:28,313 [526] [DEBUG] [app] Ending request: urn:request:c786f021-2096-44b2-ab62-fefdab147f35 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:11:28,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:28,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:28,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:28,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:28,317 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:28,317 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:28,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:11:28,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:11:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:11:28,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:28,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:28,326 [535] [DEBUG] [app] Starting request: urn:request:51c6a1fb-d623-4b5a-bf59-cd69b18a9c1c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:28,326 [535] [DEBUG] [app] Starting request: urn:request:51c6a1fb-d623-4b5a-bf59-cd69b18a9c1c (/c1/_internal_ping) 2020-03-31 09:11:28,327 [535] [DEBUG] [app] Ending request: urn:request:51c6a1fb-d623-4b5a-bf59-cd69b18a9c1c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:28,327 [535] [DEBUG] [app] Ending request: urn:request:51c6a1fb-d623-4b5a-bf59-cd69b18a9c1c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:28,327 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:28,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:11:28,327 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:28,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:28,329 [526] [DEBUG] [app] Ending request: urn:request:17e8c4f1-51d4-4126-a5ab-ab3432974a47 (/health/instance) 2020-03-31 09:11:28,329 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:28,329 [526] [DEBUG] [app] Ending request: urn:request:17e8c4f1-51d4-4126-a5ab-ab3432974a47 (/health/instance) 2020-03-31 09:11:28,329 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:28,329 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:11:28,329 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:11:30,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:11:30,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:30,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:00.337699+00:00 (in 29.996067 seconds) buildlogsarchiver stdout | 2020-03-31 09:11:30,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:00.337699+00:00 (in 29.996067 seconds) 2020-03-31 09:11:30,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:00 UTC)" (scheduled at 2020-03-31 09:11:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:11:30,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:00 UTC)" (scheduled at 2020-03-31 09:11:30.337699+00:00) 2020-03-31 09:11:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 11, 30, 342682), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:11:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 11, 30, 342682), False, 50, 1, 0]) 2020-03-31 09:11:30,360 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:11:30,360 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:11:30,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:30,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:11:30,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:30,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:00 UTC)" executed successfully 2020-03-31 09:11:31,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:11:31,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:31,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:41.440707+00:00 (in 9.995365 seconds) 2020-03-31 09:11:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:41 UTC)" (scheduled at 2020-03-31 09:11:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:11:31,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:41.440707+00:00 (in 9.995365 seconds) 2020-03-31 09:11:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:41 UTC)" (scheduled at 2020-03-31 09:11:31.440707+00:00) 2020-03-31 09:11:31,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:11:31,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:11:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:11:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:11:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:11:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:41 UTC)" executed successfully 2020-03-31 09:11:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:11:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:03.330112+00:00 (in 29.999133 seconds) 2020-03-31 09:11:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:03 UTC)" (scheduled at 2020-03-31 09:11:33.330112+00:00) 2020-03-31 09:11:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:11:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:03.330112+00:00 (in 29.999133 seconds) 2020-03-31 09:11:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:03 UTC)" (scheduled at 2020-03-31 09:11:33.330112+00:00) 2020-03-31 09:11:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 33, 331884), True, datetime.datetime(2020, 3, 31, 9, 11, 33, 331884), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:11:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 33, 331884), True, datetime.datetime(2020, 3, 31, 9, 11, 33, 331884), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:11:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:11:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:11:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:03 UTC)" executed successfully 2020-03-31 09:11:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:11:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:43.539999+00:00 (in 10.172556 seconds) 2020-03-31 09:11:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:33 UTC)" (scheduled at 2020-03-31 09:11:33.366791+00:00) 2020-03-31 09:11:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:11:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:43.539999+00:00 (in 10.172556 seconds) 2020-03-31 09:11:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:33 UTC)" (scheduled at 2020-03-31 09:11:33.366791+00:00) 2020-03-31 09:11:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 33, 368254), True, datetime.datetime(2020, 3, 31, 9, 11, 33, 368254), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:11:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 33, 368254), True, datetime.datetime(2020, 3, 31, 9, 11, 33, 368254), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:11:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:11:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:11:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:33 UTC)" executed successfully 2020-03-31 09:11:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:11:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:44.323352+00:00 (in 9.998748 seconds) 2020-03-31 09:11:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:44 UTC)" (scheduled at 2020-03-31 09:11:34.323352+00:00) 2020-03-31 09:11:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:11:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:44.323352+00:00 (in 9.998748 seconds) 2020-03-31 09:11:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:44 UTC)" (scheduled at 2020-03-31 09:11:34.323352+00:00) 2020-03-31 09:11:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 34, 325521), True, datetime.datetime(2020, 3, 31, 9, 11, 34, 325521), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:11:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 34, 325521), True, datetime.datetime(2020, 3, 31, 9, 11, 34, 325521), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:11:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:11:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:11:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:44 UTC)" executed successfully 2020-03-31 09:11:34,447 [526] [DEBUG] [app] Starting request: urn:request:bae861e8-242c-43be-bd71-817ab358aaa0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:34,447 [526] [DEBUG] [app] Starting request: urn:request:bae861e8-242c-43be-bd71-817ab358aaa0 (/health/instance) 2020-03-31 09:11:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:34,454 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:34,454 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:34,455 [548] [DEBUG] [app] Starting request: urn:request:f501b51f-1d26-478b-8525-e2b76fda6652 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:34,455 [548] [DEBUG] [app] Starting request: urn:request:f501b51f-1d26-478b-8525-e2b76fda6652 (/v1/_internal_ping) 2020-03-31 09:11:34,456 [548] [DEBUG] [app] Ending request: urn:request:f501b51f-1d26-478b-8525-e2b76fda6652 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:34,456 [548] [DEBUG] [app] Ending request: urn:request:f501b51f-1d26-478b-8525-e2b76fda6652 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:34,457 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:34,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:11:34,457 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:34,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:34,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:34,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:11:34,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:11:34,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:34,460 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:34,460 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:34,461 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:34,461 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:34,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:34,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:34,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:34,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:34,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:34,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:34,490 [526] [DEBUG] [app] Starting request: urn:request:a4860daa-81af-4054-bff5-e8fddcb97a0d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:34,490 [526] [DEBUG] [app] Starting request: urn:request:a4860daa-81af-4054-bff5-e8fddcb97a0d (/_internal_ping) 2020-03-31 09:11:34,491 [526] [DEBUG] [app] Ending request: urn:request:a4860daa-81af-4054-bff5-e8fddcb97a0d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:34,491 [526] [DEBUG] [app] Ending request: urn:request:a4860daa-81af-4054-bff5-e8fddcb97a0d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:11:34,492 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:34,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:34,492 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:34,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:34,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:34,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:34,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:11:34,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:34,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:34,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:34,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:11:34,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:11:34,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:34,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:34,505 [535] [DEBUG] [app] Starting request: urn:request:3f6733c1-3467-4f63-bdb0-6abb3be4d66f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:34,505 [535] [DEBUG] [app] Starting request: urn:request:3f6733c1-3467-4f63-bdb0-6abb3be4d66f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:34,506 [535] [DEBUG] [app] Ending request: urn:request:3f6733c1-3467-4f63-bdb0-6abb3be4d66f (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:34,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:34,507 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:11:34,506 [535] [DEBUG] [app] Ending request: urn:request:3f6733c1-3467-4f63-bdb0-6abb3be4d66f (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:34,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:34,509 [526] [DEBUG] [app] Ending request: urn:request:bae861e8-242c-43be-bd71-817ab358aaa0 (/health/instance) 2020-03-31 09:11:34,509 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:11:34,507 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:34,510 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:11:34,509 [526] [DEBUG] [app] Ending request: urn:request:bae861e8-242c-43be-bd71-817ab358aaa0 (/health/instance) 2020-03-31 09:11:34,509 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:34,510 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:11:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:11:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:38.010934+00:00 (in 1.085714 seconds) 2020-03-31 09:11:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:46 UTC)" (scheduled at 2020-03-31 09:11:36.924497+00:00) 2020-03-31 09:11:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:11:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:38.010934+00:00 (in 1.085714 seconds) 2020-03-31 09:11:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:46 UTC)" (scheduled at 2020-03-31 09:11:36.924497+00:00) 2020-03-31 09:11:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 36, 925921), True, datetime.datetime(2020, 3, 31, 9, 11, 36, 925921), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:11:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 36, 925921), True, datetime.datetime(2020, 3, 31, 9, 11, 36, 925921), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:11:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:11:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:11:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:46 UTC)" executed successfully 2020-03-31 09:11:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:11:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:46.924497+00:00 (in 8.912806 seconds) 2020-03-31 09:11:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:38 UTC)" (scheduled at 2020-03-31 09:11:38.010934+00:00) 2020-03-31 09:11:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:11:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:46.924497+00:00 (in 8.912806 seconds) 2020-03-31 09:11:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:38 UTC)" (scheduled at 2020-03-31 09:11:38.010934+00:00) 2020-03-31 09:11:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:11:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:11:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:38 UTC)" executed successfully 2020-03-31 09:11:38,273 [527] [DEBUG] [app] Starting request: urn:request:7d80ac83-c5cc-42a0-a206-4b230f835c77 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:38,273 [527] [DEBUG] [app] Starting request: urn:request:7d80ac83-c5cc-42a0-a206-4b230f835c77 (/health/instance) 2020-03-31 09:11:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:38,281 [553] [DEBUG] [app] Starting request: urn:request:8ef04d3d-5d9e-4c9c-8cff-e53e17447cdc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:38,281 [553] [DEBUG] [app] Starting request: urn:request:8ef04d3d-5d9e-4c9c-8cff-e53e17447cdc (/v1/_internal_ping) 2020-03-31 09:11:38,282 [553] [DEBUG] [app] Ending request: urn:request:8ef04d3d-5d9e-4c9c-8cff-e53e17447cdc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:38,282 [553] [DEBUG] [app] Ending request: urn:request:8ef04d3d-5d9e-4c9c-8cff-e53e17447cdc (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:38,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:11:38,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:38,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:38,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:38,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:38,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:38,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:38,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:38,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:38,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:38,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:38,315 [527] [DEBUG] [app] Starting request: urn:request:d9bcae34-249a-488f-a996-4b70e4129dc4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:38,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:38,315 [527] [DEBUG] [app] Starting request: urn:request:d9bcae34-249a-488f-a996-4b70e4129dc4 (/_internal_ping) 2020-03-31 09:11:38,316 [527] [DEBUG] [app] Ending request: urn:request:d9bcae34-249a-488f-a996-4b70e4129dc4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:38,316 [527] [DEBUG] [app] Ending request: urn:request:d9bcae34-249a-488f-a996-4b70e4129dc4 (/_internal_ping) 2020-03-31 09:11:38,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:11:38,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:11:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:11:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:11:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:38,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:38,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:38,331 [543] [DEBUG] [app] Starting request: urn:request:6b2c3d8e-6f7f-402f-95db-e2ad23042099 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:38,331 [543] [DEBUG] [app] Starting request: urn:request:6b2c3d8e-6f7f-402f-95db-e2ad23042099 (/c1/_internal_ping) 2020-03-31 09:11:38,332 [543] [DEBUG] [app] Ending request: urn:request:6b2c3d8e-6f7f-402f-95db-e2ad23042099 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:38,332 [543] [DEBUG] [app] Ending request: urn:request:6b2c3d8e-6f7f-402f-95db-e2ad23042099 (/c1/_internal_ping) 2020-03-31 09:11:38,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:38,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:38,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:38,334 [527] [DEBUG] [app] Ending request: urn:request:7d80ac83-c5cc-42a0-a206-4b230f835c77 (/health/instance) 2020-03-31 09:11:38,334 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:11:38,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:38,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:11:38,334 [527] [DEBUG] [app] Ending request: urn:request:7d80ac83-c5cc-42a0-a206-4b230f835c77 (/health/instance) 2020-03-31 09:11:38,334 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:38,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:11:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:11:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:51.440707+00:00 (in 9.999042 seconds) 2020-03-31 09:11:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:51 UTC)" (scheduled at 2020-03-31 09:11:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:11:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:51.440707+00:00 (in 9.999042 seconds) 2020-03-31 09:11:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:51 UTC)" (scheduled at 2020-03-31 09:11:41.440707+00:00) 2020-03-31 09:11:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:11:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:11:41,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:11:41,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:11:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:11:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:51 UTC)" executed successfully 2020-03-31 09:11:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:11:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:33.366791+00:00 (in 49.825756 seconds) 2020-03-31 09:11:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:43 UTC)" (scheduled at 2020-03-31 09:11:43.539999+00:00) 2020-03-31 09:11:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:11:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:33.366791+00:00 (in 49.825756 seconds) 2020-03-31 09:11:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:43 UTC)" (scheduled at 2020-03-31 09:11:43.539999+00:00) 2020-03-31 09:11:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:11:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:11:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:43 UTC)" executed successfully 2020-03-31 09:11:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:11:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:49.498966+00:00 (in 5.174362 seconds) 2020-03-31 09:11:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:54 UTC)" (scheduled at 2020-03-31 09:11:44.323352+00:00) 2020-03-31 09:11:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:11:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:49.498966+00:00 (in 5.174362 seconds) 2020-03-31 09:11:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:54 UTC)" (scheduled at 2020-03-31 09:11:44.323352+00:00) 2020-03-31 09:11:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 44, 325502), True, datetime.datetime(2020, 3, 31, 9, 11, 44, 325502), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:11:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 44, 325502), True, datetime.datetime(2020, 3, 31, 9, 11, 44, 325502), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:11:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:11:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:11:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:54 UTC)" executed successfully 2020-03-31 09:11:44,446 [518] [DEBUG] [app] Starting request: urn:request:fe8d9983-2801-4664-a4c8-485e811daa92 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:44,446 [518] [DEBUG] [app] Starting request: urn:request:fe8d9983-2801-4664-a4c8-485e811daa92 (/health/instance) 2020-03-31 09:11:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:44,453 [538] [DEBUG] [app] Starting request: urn:request:bc13705d-7d2d-4f2d-8436-fc9403b32a6a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:44,453 [538] [DEBUG] [app] Starting request: urn:request:bc13705d-7d2d-4f2d-8436-fc9403b32a6a (/v1/_internal_ping) 2020-03-31 09:11:44,454 [538] [DEBUG] [app] Ending request: urn:request:bc13705d-7d2d-4f2d-8436-fc9403b32a6a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:44,454 [538] [DEBUG] [app] Ending request: urn:request:bc13705d-7d2d-4f2d-8436-fc9403b32a6a (/v1/_internal_ping) 2020-03-31 09:11:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:11:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:11:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:11:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:44,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:44,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:44,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:44,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:44,490 [526] [DEBUG] [app] Starting request: urn:request:c8db8e70-d1db-4236-bc1a-532d8abc9a80 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:44,490 [526] [DEBUG] [app] Starting request: urn:request:c8db8e70-d1db-4236-bc1a-532d8abc9a80 (/_internal_ping) 2020-03-31 09:11:44,491 [526] [DEBUG] [app] Ending request: urn:request:c8db8e70-d1db-4236-bc1a-532d8abc9a80 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:44,491 [526] [DEBUG] [app] Ending request: urn:request:c8db8e70-d1db-4236-bc1a-532d8abc9a80 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:11:44,492 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:44,492 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:11:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:11:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:11:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:44,504 [536] [DEBUG] [app] Starting request: urn:request:482827eb-544d-448b-b9fe-4c5131e3070f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:44,504 [536] [DEBUG] [app] Starting request: urn:request:482827eb-544d-448b-b9fe-4c5131e3070f (/c1/_internal_ping) 2020-03-31 09:11:44,505 [536] [DEBUG] [app] Ending request: urn:request:482827eb-544d-448b-b9fe-4c5131e3070f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:44,505 [536] [DEBUG] [app] Ending request: urn:request:482827eb-544d-448b-b9fe-4c5131e3070f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:44,505 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:11:44,505 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:44,507 [518] [DEBUG] [app] Ending request: urn:request:fe8d9983-2801-4664-a4c8-485e811daa92 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:44,507 [518] [DEBUG] [app] Ending request: urn:request:fe8d9983-2801-4664-a4c8-485e811daa92 (/health/instance) 2020-03-31 09:11:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:11:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:11:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:11:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:15.064514+00:00 (in 29.999004 seconds) 2020-03-31 09:11:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:15 UTC)" (scheduled at 2020-03-31 09:11:45.064514+00:00) gcworker stdout | 2020-03-31 09:11:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:15.064514+00:00 (in 29.999004 seconds) 2020-03-31 09:11:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:15 UTC)" (scheduled at 2020-03-31 09:11:45.064514+00:00) 2020-03-31 09:11:45,066 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2020-03-31 09:11:45,066 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) 2020-03-31 09:11:45,080 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436305, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:11:45,080 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436305, 1209600, 500, 1, 0]) 2020-03-31 09:11:45,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:11:45,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:11:45,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:45,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:11:45,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:45,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:15 UTC)" executed successfully 2020-03-31 09:11:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:11:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:56.924497+00:00 (in 9.998862 seconds) 2020-03-31 09:11:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:56 UTC)" (scheduled at 2020-03-31 09:11:46.924497+00:00) 2020-03-31 09:11:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:11:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:56.924497+00:00 (in 9.998862 seconds) 2020-03-31 09:11:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:56 UTC)" (scheduled at 2020-03-31 09:11:46.924497+00:00) 2020-03-31 09:11:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 46, 926609), True, datetime.datetime(2020, 3, 31, 9, 11, 46, 926609), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:11:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 46, 926609), True, datetime.datetime(2020, 3, 31, 9, 11, 46, 926609), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:11:46,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:11:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:11:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:11:56 UTC)" executed successfully 2020-03-31 09:11:48,273 [527] [DEBUG] [app] Starting request: urn:request:fd4aa735-3a97-44cb-aa0d-1d4058323c1e (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:48,273 [527] [DEBUG] [app] Starting request: urn:request:fd4aa735-3a97-44cb-aa0d-1d4058323c1e (/health/instance) 2020-03-31 09:11:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:48,281 [556] [DEBUG] [app] Starting request: urn:request:0b0d9fb8-0a44-43bd-b8b1-42f0149003ba (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:48,281 [556] [DEBUG] [app] Starting request: urn:request:0b0d9fb8-0a44-43bd-b8b1-42f0149003ba (/v1/_internal_ping) 2020-03-31 09:11:48,282 [556] [DEBUG] [app] Ending request: urn:request:0b0d9fb8-0a44-43bd-b8b1-42f0149003ba (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:48,282 [556] [DEBUG] [app] Ending request: urn:request:0b0d9fb8-0a44-43bd-b8b1-42f0149003ba (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:48,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:11:48,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:48,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:48,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:48,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:48,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:48,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:48,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:48,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:48,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:48,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:48,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:48,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:48,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:48,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:48,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:48,316 [527] [DEBUG] [app] Starting request: urn:request:6f796311-d1e1-46fd-8373-14178542d28b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:48,316 [527] [DEBUG] [app] Starting request: urn:request:6f796311-d1e1-46fd-8373-14178542d28b (/_internal_ping) 2020-03-31 09:11:48,317 [527] [DEBUG] [app] Ending request: urn:request:6f796311-d1e1-46fd-8373-14178542d28b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:48,317 [527] [DEBUG] [app] Ending request: urn:request:6f796311-d1e1-46fd-8373-14178542d28b (/_internal_ping) 2020-03-31 09:11:48,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:11:48,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:11:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:11:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:11:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:48,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:48,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:48,332 [539] [DEBUG] [app] Starting request: urn:request:1baf2577-4f75-4a56-ae4a-753a771904aa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:48,332 [539] [DEBUG] [app] Starting request: urn:request:1baf2577-4f75-4a56-ae4a-753a771904aa (/c1/_internal_ping) 2020-03-31 09:11:48,333 [539] [DEBUG] [app] Ending request: urn:request:1baf2577-4f75-4a56-ae4a-753a771904aa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:48,333 [539] [DEBUG] [app] Ending request: urn:request:1baf2577-4f75-4a56-ae4a-753a771904aa (/c1/_internal_ping) 2020-03-31 09:11:48,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:48,333 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:48,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:11:48,335 [527] [DEBUG] [app] Ending request: urn:request:fd4aa735-3a97-44cb-aa0d-1d4058323c1e (/health/instance) 2020-03-31 09:11:48,336 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:11:48,333 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:48,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:11:48,335 [527] [DEBUG] [app] Ending request: urn:request:fd4aa735-3a97-44cb-aa0d-1d4058323c1e (/health/instance) 2020-03-31 09:11:48,336 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:48,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:11:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:11:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:49,502 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:54.323352+00:00 (in 4.820907 seconds) 2020-03-31 09:11:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:49 UTC)" (scheduled at 2020-03-31 09:11:49.498966+00:00) 2020-03-31 09:11:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:11:49,503 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:11:49,502 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:54.323352+00:00 (in 4.820907 seconds) 2020-03-31 09:11:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:49 UTC)" (scheduled at 2020-03-31 09:11:49.498966+00:00) 2020-03-31 09:11:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:11:49,503 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:49 UTC)" executed successfully 2020-03-31 09:11:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:11:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:59.560326+00:00 (in 8.448419 seconds) 2020-03-31 09:11:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:51 UTC)" (scheduled at 2020-03-31 09:11:51.110610+00:00) 2020-03-31 09:11:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:11:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:11:59.560326+00:00 (in 8.448419 seconds) 2020-03-31 09:11:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:51 UTC)" (scheduled at 2020-03-31 09:11:51.110610+00:00) 2020-03-31 09:11:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 51, 112929), True, datetime.datetime(2020, 3, 31, 9, 11, 51, 112929), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:11:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 51, 112929), True, datetime.datetime(2020, 3, 31, 9, 11, 51, 112929), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:11:51,133 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:11:51,133 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:51,133 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:11:51,133 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:51 UTC)" executed successfully 2020-03-31 09:11:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:11:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:01.440707+00:00 (in 9.998121 seconds) 2020-03-31 09:11:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:01 UTC)" (scheduled at 2020-03-31 09:11:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:11:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:01.440707+00:00 (in 9.998121 seconds) 2020-03-31 09:11:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:01 UTC)" (scheduled at 2020-03-31 09:11:51.440707+00:00) 2020-03-31 09:11:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:11:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:11:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:11:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:11:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:11:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:01 UTC)" executed successfully 2020-03-31 09:11:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:11:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:22.113221+00:00 (in 29.999169 seconds) 2020-03-31 09:11:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:22 UTC)" (scheduled at 2020-03-31 09:11:52.113221+00:00) securityworker stdout | 2020-03-31 09:11:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:22.113221+00:00 (in 29.999169 seconds) 2020-03-31 09:11:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:22 UTC)" (scheduled at 2020-03-31 09:11:52.113221+00:00) 2020-03-31 09:11:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:11:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:11:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:11:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:11:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:11:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:11:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:11:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:22 UTC)" executed successfully 2020-03-31 09:11:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:11:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:04.323352+00:00 (in 9.999035 seconds) 2020-03-31 09:11:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:04 UTC)" (scheduled at 2020-03-31 09:11:54.323352+00:00) 2020-03-31 09:11:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:11:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:04.323352+00:00 (in 9.999035 seconds) 2020-03-31 09:11:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:04 UTC)" (scheduled at 2020-03-31 09:11:54.323352+00:00) 2020-03-31 09:11:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 54, 325360), True, datetime.datetime(2020, 3, 31, 9, 11, 54, 325360), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:11:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 54, 325360), True, datetime.datetime(2020, 3, 31, 9, 11, 54, 325360), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:11:54,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:11:54,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:54,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:54,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:11:54,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:54,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:04 UTC)" executed successfully 2020-03-31 09:11:54,446 [518] [DEBUG] [app] Starting request: urn:request:6db3b062-f603-4a28-862e-5e34e51a0291 (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:54,446 [518] [DEBUG] [app] Starting request: urn:request:6db3b062-f603-4a28-862e-5e34e51a0291 (/health/instance) 2020-03-31 09:11:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:54,453 [555] [DEBUG] [app] Starting request: urn:request:fa61d519-a67a-4dd5-896e-8485389ed092 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:54,453 [555] [DEBUG] [app] Starting request: urn:request:fa61d519-a67a-4dd5-896e-8485389ed092 (/v1/_internal_ping) 2020-03-31 09:11:54,454 [555] [DEBUG] [app] Ending request: urn:request:fa61d519-a67a-4dd5-896e-8485389ed092 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:54,454 [555] [DEBUG] [app] Ending request: urn:request:fa61d519-a67a-4dd5-896e-8485389ed092 (/v1/_internal_ping) 2020-03-31 09:11:54,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:11:54,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:11:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:11:54,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:11:54,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:54,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:54,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:54,459 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:54,459 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:54,486 [518] [DEBUG] [app] Starting request: urn:request:3bb5b911-8739-4986-85c5-d847982e22e5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:54,486 [518] [DEBUG] [app] Starting request: urn:request:3bb5b911-8739-4986-85c5-d847982e22e5 (/_internal_ping) 2020-03-31 09:11:54,487 [518] [DEBUG] [app] Ending request: urn:request:3bb5b911-8739-4986-85c5-d847982e22e5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:54,487 [518] [DEBUG] [app] Ending request: urn:request:3bb5b911-8739-4986-85c5-d847982e22e5 (/_internal_ping) 2020-03-31 09:11:54,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:11:54,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:11:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:11:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:11:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:54,501 [539] [DEBUG] [app] Starting request: urn:request:7fc3ed1d-7498-48cb-b903-ec2f613d806e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:54,501 [539] [DEBUG] [app] Starting request: urn:request:7fc3ed1d-7498-48cb-b903-ec2f613d806e (/c1/_internal_ping) 2020-03-31 09:11:54,502 [539] [DEBUG] [app] Ending request: urn:request:7fc3ed1d-7498-48cb-b903-ec2f613d806e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:54,502 [539] [DEBUG] [app] Ending request: urn:request:7fc3ed1d-7498-48cb-b903-ec2f613d806e (/c1/_internal_ping) 2020-03-31 09:11:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:54,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:11:54,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:54,505 [518] [DEBUG] [app] Ending request: urn:request:6db3b062-f603-4a28-862e-5e34e51a0291 (/health/instance) 2020-03-31 09:11:54,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:11:54,505 [518] [DEBUG] [app] Ending request: urn:request:6db3b062-f603-4a28-862e-5e34e51a0291 (/health/instance) 2020-03-31 09:11:54,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:11:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:11:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:11:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:03.010318+00:00 (in 6.085023 seconds) 2020-03-31 09:11:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:06 UTC)" (scheduled at 2020-03-31 09:11:56.924497+00:00) 2020-03-31 09:11:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:11:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:03.010318+00:00 (in 6.085023 seconds) 2020-03-31 09:11:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:06 UTC)" (scheduled at 2020-03-31 09:11:56.924497+00:00) 2020-03-31 09:11:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:11:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 56, 926013), True, datetime.datetime(2020, 3, 31, 9, 11, 56, 926013), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:11:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 11, 56, 926013), True, datetime.datetime(2020, 3, 31, 9, 11, 56, 926013), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:11:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:11:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:11:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:11:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:06 UTC)" executed successfully 2020-03-31 09:11:58,273 [518] [DEBUG] [app] Starting request: urn:request:62178512-f6ba-4c49-8435-973b84d1daeb (/health/instance) gunicorn-web stdout | 2020-03-31 09:11:58,273 [518] [DEBUG] [app] Starting request: urn:request:62178512-f6ba-4c49-8435-973b84d1daeb (/health/instance) 2020-03-31 09:11:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:58,277 [538] [DEBUG] [app] Starting request: urn:request:ed0f1a74-4df0-42a0-8c0f-ccc56725562d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:58,277 [538] [DEBUG] [app] Starting request: urn:request:ed0f1a74-4df0-42a0-8c0f-ccc56725562d (/v1/_internal_ping) 2020-03-31 09:11:58,278 [538] [DEBUG] [app] Ending request: urn:request:ed0f1a74-4df0-42a0-8c0f-ccc56725562d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:11:58,278 [538] [DEBUG] [app] Ending request: urn:request:ed0f1a74-4df0-42a0-8c0f-ccc56725562d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:11:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:11:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:58,281 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:11:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:11:58,281 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:11:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:11:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:11:58,283 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:11:58,283 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:11:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:11:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:11:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:11:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:11:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:58,315 [518] [DEBUG] [app] Starting request: urn:request:a170d555-8684-4413-b7ea-09eb9d4ce5c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:58,315 [518] [DEBUG] [app] Starting request: urn:request:a170d555-8684-4413-b7ea-09eb9d4ce5c2 (/_internal_ping) 2020-03-31 09:11:58,316 [518] [DEBUG] [app] Ending request: urn:request:a170d555-8684-4413-b7ea-09eb9d4ce5c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:11:58,316 [518] [DEBUG] [app] Ending request: urn:request:a170d555-8684-4413-b7ea-09eb9d4ce5c2 (/_internal_ping) 2020-03-31 09:11:58,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:11:58,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:11:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:11:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:11:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:11:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:11:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:11:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:11:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:11:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:11:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:11:58,328 [539] [DEBUG] [app] Starting request: urn:request:8fd8d1b8-bfde-485c-99a1-0813575e995c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:58,328 [539] [DEBUG] [app] Starting request: urn:request:8fd8d1b8-bfde-485c-99a1-0813575e995c (/c1/_internal_ping) 2020-03-31 09:11:58,329 [539] [DEBUG] [app] Ending request: urn:request:8fd8d1b8-bfde-485c-99a1-0813575e995c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:11:58,329 [539] [DEBUG] [app] Ending request: urn:request:8fd8d1b8-bfde-485c-99a1-0813575e995c (/c1/_internal_ping) 2020-03-31 09:11:58,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:11:58,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:11:58,331 [518] [DEBUG] [app] Ending request: urn:request:62178512-f6ba-4c49-8435-973b84d1daeb (/health/instance) 2020-03-31 09:11:58,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:58,332 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:58,331 [518] [DEBUG] [app] Ending request: urn:request:62178512-f6ba-4c49-8435-973b84d1daeb (/health/instance) 2020-03-31 09:11:58,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:11:58,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-verbs stdout | 2020-03-31 09:11:58,332 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:11:58,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:11:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:11:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:11:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:11:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:11:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:51.110610+00:00 (in 51.549159 seconds) 2020-03-31 09:11:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:59 UTC)" (scheduled at 2020-03-31 09:11:59.560326+00:00) 2020-03-31 09:11:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:11:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:51.110610+00:00 (in 51.549159 seconds) 2020-03-31 09:11:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:59 UTC)" (scheduled at 2020-03-31 09:11:59.560326+00:00) 2020-03-31 09:11:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:11:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:11:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:12:59 UTC)" executed successfully 2020-03-31 09:12:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:12:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:30.337699+00:00 (in 29.998745 seconds) 2020-03-31 09:12:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:30 UTC)" (scheduled at 2020-03-31 09:12:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:12:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:30.337699+00:00 (in 29.998745 seconds) 2020-03-31 09:12:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:30 UTC)" (scheduled at 2020-03-31 09:12:00.337699+00:00) 2020-03-31 09:12:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 12, 0, 339936), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:12:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 12, 0, 339936), False, 50, 1, 0]) 2020-03-31 09:12:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:12:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:12:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:12:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:30 UTC)" executed successfully 2020-03-31 09:12:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:12:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:11.440707+00:00 (in 9.998777 seconds) 2020-03-31 09:12:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:11 UTC)" (scheduled at 2020-03-31 09:12:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:12:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:11.440707+00:00 (in 9.998777 seconds) 2020-03-31 09:12:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:11 UTC)" (scheduled at 2020-03-31 09:12:01.440707+00:00) 2020-03-31 09:12:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:12:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:12:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:01,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:12:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:01,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:11 UTC)" executed successfully 2020-03-31 09:12:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:06.924497+00:00 (in 3.913467 seconds) 2020-03-31 09:12:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:03 UTC)" (scheduled at 2020-03-31 09:12:03.010318+00:00) storagereplication stdout | 2020-03-31 09:12:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:06.924497+00:00 (in 3.913467 seconds) 2020-03-31 09:12:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:03 UTC)" (scheduled at 2020-03-31 09:12:03.010318+00:00) 2020-03-31 09:12:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:12:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), u'imagestoragereplication/%']) 2020-03-31 09:12:03,027 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), True, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:12:03,027 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), True, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), 0, u'imagestoragereplication/%']) 2020-03-31 09:12:03,031 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), True, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:12:03,031 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), True, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 12, 3, 11945), u'imagestoragereplication/%']) 2020-03-31 09:12:03,035 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:12:03,035 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:03,035 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:03,035 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:03 UTC)" executed successfully 2020-03-31 09:12:03,037 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:12:03,037 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:03,038 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:12:03,038 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:03,039 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 09:12:03,039 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:12:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:12:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:24.410927+00:00 (in 21.079680 seconds) 2020-03-31 09:12:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:33 UTC)" (scheduled at 2020-03-31 09:12:03.330112+00:00) 2020-03-31 09:12:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:12:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:24.410927+00:00 (in 21.079680 seconds) 2020-03-31 09:12:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:33 UTC)" (scheduled at 2020-03-31 09:12:03.330112+00:00) 2020-03-31 09:12:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 3, 332514), True, datetime.datetime(2020, 3, 31, 9, 12, 3, 332514), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:12:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 3, 332514), True, datetime.datetime(2020, 3, 31, 9, 12, 3, 332514), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:12:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:12:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:12:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:33 UTC)" executed successfully 2020-03-31 09:12:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:14.323352+00:00 (in 9.998859 seconds) 2020-03-31 09:12:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:14 UTC)" (scheduled at 2020-03-31 09:12:04.323352+00:00) 2020-03-31 09:12:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:12:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:14.323352+00:00 (in 9.998859 seconds) 2020-03-31 09:12:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:14 UTC)" (scheduled at 2020-03-31 09:12:04.323352+00:00) 2020-03-31 09:12:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 4, 325535), True, datetime.datetime(2020, 3, 31, 9, 12, 4, 325535), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:12:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 4, 325535), True, datetime.datetime(2020, 3, 31, 9, 12, 4, 325535), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:12:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:12:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:14 UTC)" executed successfully 2020-03-31 09:12:04,446 [518] [DEBUG] [app] Starting request: urn:request:90293525-b488-4458-968f-d20d3bafbac1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:04,446 [518] [DEBUG] [app] Starting request: urn:request:90293525-b488-4458-968f-d20d3bafbac1 (/health/instance) 2020-03-31 09:12:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:04,454 [551] [DEBUG] [app] Starting request: urn:request:9eee8bc8-c565-48d1-b457-02a54ec27151 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:04,454 [551] [DEBUG] [app] Starting request: urn:request:9eee8bc8-c565-48d1-b457-02a54ec27151 (/v1/_internal_ping) 2020-03-31 09:12:04,455 [551] [DEBUG] [app] Ending request: urn:request:9eee8bc8-c565-48d1-b457-02a54ec27151 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:04,455 [551] [DEBUG] [app] Ending request: urn:request:9eee8bc8-c565-48d1-b457-02a54ec27151 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:04,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:12:04,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:04,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:12:04,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:04,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:04,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:04,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:04,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:04,487 [518] [DEBUG] [app] Starting request: urn:request:996c03cb-4e1d-48ee-a476-f71f5c6f33eb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:04,487 [518] [DEBUG] [app] Starting request: urn:request:996c03cb-4e1d-48ee-a476-f71f5c6f33eb (/_internal_ping) 2020-03-31 09:12:04,488 [518] [DEBUG] [app] Ending request: urn:request:996c03cb-4e1d-48ee-a476-f71f5c6f33eb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:04,488 [518] [DEBUG] [app] Ending request: urn:request:996c03cb-4e1d-48ee-a476-f71f5c6f33eb (/_internal_ping) 2020-03-31 09:12:04,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:12:04,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:04,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:12:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:12:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:04,505 [532] [DEBUG] [app] Starting request: urn:request:e44eea37-2661-4cd2-bd68-4d700e605c53 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:04,505 [532] [DEBUG] [app] Starting request: urn:request:e44eea37-2661-4cd2-bd68-4d700e605c53 (/c1/_internal_ping) 2020-03-31 09:12:04,506 [532] [DEBUG] [app] Ending request: urn:request:e44eea37-2661-4cd2-bd68-4d700e605c53 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:04,506 [532] [DEBUG] [app] Ending request: urn:request:e44eea37-2661-4cd2-bd68-4d700e605c53 (/c1/_internal_ping) 2020-03-31 09:12:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:04,506 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:12:04,506 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:04,508 [518] [DEBUG] [app] Ending request: urn:request:90293525-b488-4458-968f-d20d3bafbac1 (/health/instance) 2020-03-31 09:12:04,509 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:12:04,508 [518] [DEBUG] [app] Ending request: urn:request:90293525-b488-4458-968f-d20d3bafbac1 (/health/instance) 2020-03-31 09:12:04,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:12:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:12:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:06 UTC)" (scheduled at 2020-03-31 09:12:06.924497+00:00) 2020-03-31 09:12:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:12:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:06 UTC)" (scheduled at 2020-03-31 09:12:06.924497+00:00) 2020-03-31 09:12:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 6, 926662), True, datetime.datetime(2020, 3, 31, 9, 12, 6, 926662), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:12:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 6, 926662), True, datetime.datetime(2020, 3, 31, 9, 12, 6, 926662), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:12:06,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:16.924497+00:00 (in 9.998762 seconds) storagereplication stdout | 2020-03-31 09:12:06,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:16.924497+00:00 (in 9.998762 seconds) 2020-03-31 09:12:06,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:12:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:16 UTC)" executed successfully 2020-03-31 09:12:08,273 [518] [DEBUG] [app] Starting request: urn:request:b6c9d3e8-d6c8-40b2-9fc5-b754782e4128 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:08,273 [518] [DEBUG] [app] Starting request: urn:request:b6c9d3e8-d6c8-40b2-9fc5-b754782e4128 (/health/instance) 2020-03-31 09:12:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:08,279 [538] [DEBUG] [app] Starting request: urn:request:cc0fe400-39f9-451d-90c7-89ef9ec22898 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:08,279 [538] [DEBUG] [app] Starting request: urn:request:cc0fe400-39f9-451d-90c7-89ef9ec22898 (/v1/_internal_ping) 2020-03-31 09:12:08,280 [538] [DEBUG] [app] Ending request: urn:request:cc0fe400-39f9-451d-90c7-89ef9ec22898 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:08,280 [538] [DEBUG] [app] Ending request: urn:request:cc0fe400-39f9-451d-90c7-89ef9ec22898 (/v1/_internal_ping) 2020-03-31 09:12:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:12:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:12:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:12:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:08,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:08,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:08,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:08,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:08,312 [527] [DEBUG] [app] Starting request: urn:request:f5574ebb-4ee7-4ca9-9f56-be3a9f03ece1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:08,312 [527] [DEBUG] [app] Starting request: urn:request:f5574ebb-4ee7-4ca9-9f56-be3a9f03ece1 (/_internal_ping) 2020-03-31 09:12:08,313 [527] [DEBUG] [app] Ending request: urn:request:f5574ebb-4ee7-4ca9-9f56-be3a9f03ece1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:08,313 [527] [DEBUG] [app] Ending request: urn:request:f5574ebb-4ee7-4ca9-9f56-be3a9f03ece1 (/_internal_ping) 2020-03-31 09:12:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:12:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:12:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:08,326 [532] [DEBUG] [app] Starting request: urn:request:1e7d0d4f-4361-43a6-bea1-702681a687a5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:08,326 [532] [DEBUG] [app] Starting request: urn:request:1e7d0d4f-4361-43a6-bea1-702681a687a5 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:08,327 [532] [DEBUG] [app] Ending request: urn:request:1e7d0d4f-4361-43a6-bea1-702681a687a5 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:08,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:08,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:12:08,327 [532] [DEBUG] [app] Ending request: urn:request:1e7d0d4f-4361-43a6-bea1-702681a687a5 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:08,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:12:08,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:08,329 [518] [DEBUG] [app] Ending request: urn:request:b6c9d3e8-d6c8-40b2-9fc5-b754782e4128 (/health/instance) 2020-03-31 09:12:08,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:08,329 [518] [DEBUG] [app] Ending request: urn:request:b6c9d3e8-d6c8-40b2-9fc5-b754782e4128 (/health/instance) 2020-03-31 09:12:08,329 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:12:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:12:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:12:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:12:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:21.440707+00:00 (in 9.998898 seconds) 2020-03-31 09:12:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:21 UTC)" (scheduled at 2020-03-31 09:12:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:12:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:21.440707+00:00 (in 9.998898 seconds) 2020-03-31 09:12:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:21 UTC)" (scheduled at 2020-03-31 09:12:11.440707+00:00) 2020-03-31 09:12:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:12:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:12:11,457 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:12:11,457 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:11,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:11,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:12:11,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:11,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:21 UTC)" executed successfully 2020-03-31 09:12:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:24.323352+00:00 (in 9.998901 seconds) 2020-03-31 09:12:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:24 UTC)" (scheduled at 2020-03-31 09:12:14.323352+00:00) 2020-03-31 09:12:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:12:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:24.323352+00:00 (in 9.998901 seconds) 2020-03-31 09:12:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:24 UTC)" (scheduled at 2020-03-31 09:12:14.323352+00:00) 2020-03-31 09:12:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 14, 325314), True, datetime.datetime(2020, 3, 31, 9, 12, 14, 325314), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:12:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 14, 325314), True, datetime.datetime(2020, 3, 31, 9, 12, 14, 325314), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:12:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:12:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:24 UTC)" executed successfully 2020-03-31 09:12:14,446 [518] [DEBUG] [app] Starting request: urn:request:ae7250f9-dbc8-4666-adee-1cf7235fb8c2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:14,446 [518] [DEBUG] [app] Starting request: urn:request:ae7250f9-dbc8-4666-adee-1cf7235fb8c2 (/health/instance) 2020-03-31 09:12:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:14,453 [550] [DEBUG] [app] Starting request: urn:request:e27a848b-d5d7-432b-ae83-07353a6a446e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:14,453 [550] [DEBUG] [app] Starting request: urn:request:e27a848b-d5d7-432b-ae83-07353a6a446e (/v1/_internal_ping) 2020-03-31 09:12:14,454 [550] [DEBUG] [app] Ending request: urn:request:e27a848b-d5d7-432b-ae83-07353a6a446e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:14,454 [550] [DEBUG] [app] Ending request: urn:request:e27a848b-d5d7-432b-ae83-07353a6a446e (/v1/_internal_ping) 2020-03-31 09:12:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:12:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:14,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:12:14,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:14,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:14,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:14,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:14,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:14,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:14,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:14,494 [527] [DEBUG] [app] Starting request: urn:request:7b699986-b100-4b6d-a093-50fd7b6a08a7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:14,494 [527] [DEBUG] [app] Starting request: urn:request:7b699986-b100-4b6d-a093-50fd7b6a08a7 (/_internal_ping) 2020-03-31 09:12:14,495 [527] [DEBUG] [app] Ending request: urn:request:7b699986-b100-4b6d-a093-50fd7b6a08a7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:14,495 [527] [DEBUG] [app] Ending request: urn:request:7b699986-b100-4b6d-a093-50fd7b6a08a7 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:12:14,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:14,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:14,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:14,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:12:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:12:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:14,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:14,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:14,510 [532] [DEBUG] [app] Starting request: urn:request:77bc03f2-916a-4b5d-9fad-87a92a6781af (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:14,510 [532] [DEBUG] [app] Starting request: urn:request:77bc03f2-916a-4b5d-9fad-87a92a6781af (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:14,512 [532] [DEBUG] [app] Ending request: urn:request:77bc03f2-916a-4b5d-9fad-87a92a6781af (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:14,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:14,512 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:12:14,512 [532] [DEBUG] [app] Ending request: urn:request:77bc03f2-916a-4b5d-9fad-87a92a6781af (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:14,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:14,514 [518] [DEBUG] [app] Ending request: urn:request:ae7250f9-dbc8-4666-adee-1cf7235fb8c2 (/health/instance) 2020-03-31 09:12:14,514 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:14,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-verbs stdout | 2020-03-31 09:12:14,512 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:14,514 [518] [DEBUG] [app] Ending request: urn:request:ae7250f9-dbc8-4666-adee-1cf7235fb8c2 (/health/instance) 2020-03-31 09:12:14,514 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:14,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:12:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:12:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:45.064514+00:00 (in 29.999207 seconds) 2020-03-31 09:12:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:45 UTC)" (scheduled at 2020-03-31 09:12:15.064514+00:00) gcworker stdout | 2020-03-31 09:12:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:45.064514+00:00 (in 29.999207 seconds) 2020-03-31 09:12:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:45 UTC)" (scheduled at 2020-03-31 09:12:15.064514+00:00) 2020-03-31 09:12:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436335, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:12:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436335, 1209600, 500, 1, 0]) 2020-03-31 09:12:15,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:12:15,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:12:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:12:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:45 UTC)" executed successfully 2020-03-31 09:12:16,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:16,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:16,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:26.924497+00:00 (in 9.995435 seconds) 2020-03-31 09:12:16,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:26 UTC)" (scheduled at 2020-03-31 09:12:16.924497+00:00) 2020-03-31 09:12:16,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:12:16,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:26.924497+00:00 (in 9.995435 seconds) 2020-03-31 09:12:16,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:26 UTC)" (scheduled at 2020-03-31 09:12:16.924497+00:00) 2020-03-31 09:12:16,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:16,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 16, 929995), True, datetime.datetime(2020, 3, 31, 9, 12, 16, 929995), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:12:16,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 16, 929995), True, datetime.datetime(2020, 3, 31, 9, 12, 16, 929995), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:12:16,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:12:16,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:16,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:16,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:16,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:16,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:26 UTC)" executed successfully 2020-03-31 09:12:18,273 [525] [DEBUG] [app] Starting request: urn:request:1af52272-3b93-47ff-a22d-9d86438fa1ba (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:18,273 [525] [DEBUG] [app] Starting request: urn:request:1af52272-3b93-47ff-a22d-9d86438fa1ba (/health/instance) 2020-03-31 09:12:18,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:18,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:18,279 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:18,279 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:18,281 [538] [DEBUG] [app] Starting request: urn:request:cb94195a-f6a4-42a4-8791-4a670beee6a3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:18,281 [538] [DEBUG] [app] Starting request: urn:request:cb94195a-f6a4-42a4-8791-4a670beee6a3 (/v1/_internal_ping) 2020-03-31 09:12:18,282 [538] [DEBUG] [app] Ending request: urn:request:cb94195a-f6a4-42a4-8791-4a670beee6a3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:18,282 [538] [DEBUG] [app] Ending request: urn:request:cb94195a-f6a4-42a4-8791-4a670beee6a3 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:18,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:18,283 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:12:18,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:18,284 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:18,283 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:12:18,284 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:12:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:18,294 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:18,294 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:18,310 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:18,310 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:18,314 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:18,314 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:18,315 [525] [DEBUG] [app] Starting request: urn:request:91c8778b-f53b-4fab-a1b9-f58d7be925de (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:18,315 [525] [DEBUG] [app] Starting request: urn:request:91c8778b-f53b-4fab-a1b9-f58d7be925de (/_internal_ping) 2020-03-31 09:12:18,316 [525] [DEBUG] [app] Ending request: urn:request:91c8778b-f53b-4fab-a1b9-f58d7be925de (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:18,316 [525] [DEBUG] [app] Ending request: urn:request:91c8778b-f53b-4fab-a1b9-f58d7be925de (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:12:18,316 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:18,317 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:18,316 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:18,317 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:18,320 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:18,320 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:18,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:12:18,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:12:18,323 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:18,323 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:18,324 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:18,324 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:18,327 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:18,327 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:18,329 [532] [DEBUG] [app] Starting request: urn:request:e60fc57f-8622-47de-aef2-71c61bbdc486 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:18,329 [532] [DEBUG] [app] Starting request: urn:request:e60fc57f-8622-47de-aef2-71c61bbdc486 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:18,330 [532] [DEBUG] [app] Ending request: urn:request:e60fc57f-8622-47de-aef2-71c61bbdc486 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:18,331 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:18,331 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:12:18,330 [532] [DEBUG] [app] Ending request: urn:request:e60fc57f-8622-47de-aef2-71c61bbdc486 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:18,331 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:18,332 [525] [DEBUG] [app] Ending request: urn:request:1af52272-3b93-47ff-a22d-9d86438fa1ba (/health/instance) 2020-03-31 09:12:18,332 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:12:18,331 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:18,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:12:18,332 [525] [DEBUG] [app] Ending request: urn:request:1af52272-3b93-47ff-a22d-9d86438fa1ba (/health/instance) 2020-03-31 09:12:18,332 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:18,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:12:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:12:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:21,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:31.440707+00:00 (in 9.998009 seconds) 2020-03-31 09:12:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:31 UTC)" (scheduled at 2020-03-31 09:12:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:12:21,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:31.440707+00:00 (in 9.998009 seconds) 2020-03-31 09:12:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:31 UTC)" (scheduled at 2020-03-31 09:12:21.440707+00:00) 2020-03-31 09:12:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:12:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:12:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:12:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:12:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:31 UTC)" executed successfully 2020-03-31 09:12:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:12:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:52.113221+00:00 (in 29.998704 seconds) 2020-03-31 09:12:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:52 UTC)" (scheduled at 2020-03-31 09:12:22.113221+00:00) securityworker stdout | 2020-03-31 09:12:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:52.113221+00:00 (in 29.998704 seconds) 2020-03-31 09:12:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:52 UTC)" (scheduled at 2020-03-31 09:12:22.113221+00:00) 2020-03-31 09:12:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:12:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:12:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:12:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:12:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:12:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:12:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:12:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:12:52 UTC)" executed successfully 2020-03-31 09:12:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:31.498397+00:00 (in 7.174168 seconds) 2020-03-31 09:12:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:34 UTC)" (scheduled at 2020-03-31 09:12:24.323352+00:00) 2020-03-31 09:12:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:12:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:31.498397+00:00 (in 7.174168 seconds) 2020-03-31 09:12:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:34 UTC)" (scheduled at 2020-03-31 09:12:24.323352+00:00) 2020-03-31 09:12:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 24, 325214), True, datetime.datetime(2020, 3, 31, 9, 12, 24, 325214), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:12:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 24, 325214), True, datetime.datetime(2020, 3, 31, 9, 12, 24, 325214), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:12:24,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:12:24,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:24,348 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:24,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:24,348 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:24,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:34 UTC)" executed successfully 2020-03-31 09:12:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:12:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:33.330112+00:00 (in 8.917942 seconds) 2020-03-31 09:12:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" (scheduled at 2020-03-31 09:12:24.410927+00:00) 2020-03-31 09:12:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:12:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:33.330112+00:00 (in 8.917942 seconds) 2020-03-31 09:12:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" (scheduled at 2020-03-31 09:12:24.410927+00:00) 2020-03-31 09:12:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" executed successfully 2020-03-31 09:12:24,446 [527] [DEBUG] [app] Starting request: urn:request:c9c4210d-e0bf-4de7-94b9-907182e93406 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:24,446 [527] [DEBUG] [app] Starting request: urn:request:c9c4210d-e0bf-4de7-94b9-907182e93406 (/health/instance) 2020-03-31 09:12:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:24,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:24,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:24,454 [556] [DEBUG] [app] Starting request: urn:request:0c5dd9d1-1ed1-4c99-bf69-55147869729d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:24,454 [556] [DEBUG] [app] Starting request: urn:request:0c5dd9d1-1ed1-4c99-bf69-55147869729d (/v1/_internal_ping) 2020-03-31 09:12:24,455 [556] [DEBUG] [app] Ending request: urn:request:0c5dd9d1-1ed1-4c99-bf69-55147869729d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:24,455 [556] [DEBUG] [app] Ending request: urn:request:0c5dd9d1-1ed1-4c99-bf69-55147869729d (/v1/_internal_ping) 2020-03-31 09:12:24,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:12:24,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:24,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:24,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:24,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:24,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:24,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:12:24,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:24,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:24,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:24,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:24,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:24,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:24,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:24,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:24,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:24,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:24,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:24,485 [527] [DEBUG] [app] Starting request: urn:request:7deeec36-d73c-495c-b8f7-f4713b839b62 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:24,485 [527] [DEBUG] [app] Starting request: urn:request:7deeec36-d73c-495c-b8f7-f4713b839b62 (/_internal_ping) 2020-03-31 09:12:24,486 [527] [DEBUG] [app] Ending request: urn:request:7deeec36-d73c-495c-b8f7-f4713b839b62 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:24,486 [527] [DEBUG] [app] Ending request: urn:request:7deeec36-d73c-495c-b8f7-f4713b839b62 (/_internal_ping) 2020-03-31 09:12:24,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:24,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:24,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:24,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:24,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:24,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:24,499 [532] [DEBUG] [app] Starting request: urn:request:a4d27fba-e70e-4b71-8b33-2192b3ec04c9 (/c1/_internal_ping) 2020-03-31 09:12:24,499 [532] [DEBUG] [app] Ending request: urn:request:a4d27fba-e70e-4b71-8b33-2192b3ec04c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:24,499 [532] [DEBUG] [app] Starting request: urn:request:a4d27fba-e70e-4b71-8b33-2192b3ec04c9 (/c1/_internal_ping) 2020-03-31 09:12:24,499 [532] [DEBUG] [app] Ending request: urn:request:a4d27fba-e70e-4b71-8b33-2192b3ec04c9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:12:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:24,501 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:12:24,501 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:24,503 [527] [DEBUG] [app] Ending request: urn:request:c9c4210d-e0bf-4de7-94b9-907182e93406 (/health/instance) 2020-03-31 09:12:24,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:24,503 [527] [DEBUG] [app] Ending request: urn:request:c9c4210d-e0bf-4de7-94b9-907182e93406 (/health/instance) 2020-03-31 09:12:24,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:24,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:12:24,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:12:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:36.924497+00:00 (in 9.999056 seconds) 2020-03-31 09:12:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:36 UTC)" (scheduled at 2020-03-31 09:12:26.924497+00:00) 2020-03-31 09:12:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:12:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:36.924497+00:00 (in 9.999056 seconds) 2020-03-31 09:12:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:36 UTC)" (scheduled at 2020-03-31 09:12:26.924497+00:00) 2020-03-31 09:12:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 26, 926081), True, datetime.datetime(2020, 3, 31, 9, 12, 26, 926081), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:12:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 26, 926081), True, datetime.datetime(2020, 3, 31, 9, 12, 26, 926081), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:12:26,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:12:26,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:36 UTC)" executed successfully 2020-03-31 09:12:28,273 [518] [DEBUG] [app] Starting request: urn:request:23a1a9f9-9ffe-4d15-a4f0-87cd242474cd (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:28,273 [518] [DEBUG] [app] Starting request: urn:request:23a1a9f9-9ffe-4d15-a4f0-87cd242474cd (/health/instance) 2020-03-31 09:12:28,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:28,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:28,280 [552] [DEBUG] [app] Starting request: urn:request:78695fc7-3b15-45d0-beb3-d8b8ac437c0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:28,280 [552] [DEBUG] [app] Starting request: urn:request:78695fc7-3b15-45d0-beb3-d8b8ac437c0e (/v1/_internal_ping) 2020-03-31 09:12:28,281 [552] [DEBUG] [app] Ending request: urn:request:78695fc7-3b15-45d0-beb3-d8b8ac437c0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:28,281 [552] [DEBUG] [app] Ending request: urn:request:78695fc7-3b15-45d0-beb3-d8b8ac437c0e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:28,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:12:28,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:28,287 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:12:28,287 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:28,289 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:28,289 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:28,290 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:28,290 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:28,315 [527] [DEBUG] [app] Starting request: urn:request:f43a68f6-51b8-419f-8370-660b5f5a0c95 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:28,315 [527] [DEBUG] [app] Starting request: urn:request:f43a68f6-51b8-419f-8370-660b5f5a0c95 (/_internal_ping) 2020-03-31 09:12:28,316 [527] [DEBUG] [app] Ending request: urn:request:f43a68f6-51b8-419f-8370-660b5f5a0c95 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:28,316 [527] [DEBUG] [app] Ending request: urn:request:f43a68f6-51b8-419f-8370-660b5f5a0c95 (/_internal_ping) 2020-03-31 09:12:28,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:12:28,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:12:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:12:28,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:28,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:28,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:28,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:28,330 [532] [DEBUG] [app] Starting request: urn:request:16b627fc-29a4-4b21-b746-0e2eaa626c77 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:28,330 [532] [DEBUG] [app] Starting request: urn:request:16b627fc-29a4-4b21-b746-0e2eaa626c77 (/c1/_internal_ping) 2020-03-31 09:12:28,331 [532] [DEBUG] [app] Ending request: urn:request:16b627fc-29a4-4b21-b746-0e2eaa626c77 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:28,331 [532] [DEBUG] [app] Ending request: urn:request:16b627fc-29a4-4b21-b746-0e2eaa626c77 (/c1/_internal_ping) 2020-03-31 09:12:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:28,332 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:12:28,332 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:28,333 [518] [DEBUG] [app] Ending request: urn:request:23a1a9f9-9ffe-4d15-a4f0-87cd242474cd (/health/instance) 2020-03-31 09:12:28,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:28,333 [518] [DEBUG] [app] Ending request: urn:request:23a1a9f9-9ffe-4d15-a4f0-87cd242474cd (/health/instance) 2020-03-31 09:12:28,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:28,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:12:28,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:12:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:00.337699+00:00 (in 29.999034 seconds) buildlogsarchiver stdout | 2020-03-31 09:12:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:00.337699+00:00 (in 29.999034 seconds) 2020-03-31 09:12:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:00 UTC)" (scheduled at 2020-03-31 09:12:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:12:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:00 UTC)" (scheduled at 2020-03-31 09:12:30.337699+00:00) 2020-03-31 09:12:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 12, 30, 339351), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:12:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 12, 30, 339351), False, 50, 1, 0]) 2020-03-31 09:12:30,360 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:12:30,360 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:12:30,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:30,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:12:30,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:30,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:00 UTC)" executed successfully 2020-03-31 09:12:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:12:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:41 UTC)" (scheduled at 2020-03-31 09:12:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:12:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:41 UTC)" (scheduled at 2020-03-31 09:12:31.440707+00:00) 2020-03-31 09:12:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:41.440707+00:00 (in 9.998809 seconds) repositoryactioncounter stdout | 2020-03-31 09:12:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:41.440707+00:00 (in 9.998809 seconds) 2020-03-31 09:12:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:12:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:12:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:12:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:12:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:41 UTC)" executed successfully 2020-03-31 09:12:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:34.323352+00:00 (in 2.824050 seconds) 2020-03-31 09:12:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:31 UTC)" (scheduled at 2020-03-31 09:12:31.498397+00:00) notificationworker stdout | 2020-03-31 09:12:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:34.323352+00:00 (in 2.824050 seconds) 2020-03-31 09:12:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:31 UTC)" (scheduled at 2020-03-31 09:12:31.498397+00:00) 2020-03-31 09:12:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), u'notification/%']) notificationworker stdout | 2020-03-31 09:12:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), u'notification/%']) 2020-03-31 09:12:31,515 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), True, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:12:31,515 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), True, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), 0, u'notification/%']) 2020-03-31 09:12:31,520 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), True, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), u'notification/%']) notificationworker stdout | 2020-03-31 09:12:31,520 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), True, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 12, 31, 500149), u'notification/%']) 2020-03-31 09:12:31,524 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:12:31,524 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:31,527 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:12:31,527 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:31,527 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:31,527 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:17:31 UTC)" executed successfully 2020-03-31 09:12:31,528 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:12:31,528 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:31,529 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 09:12:31,529 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:12:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:12:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:03.330112+00:00 (in 29.999090 seconds) 2020-03-31 09:12:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:03 UTC)" (scheduled at 2020-03-31 09:12:33.330112+00:00) 2020-03-31 09:12:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:12:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:03.330112+00:00 (in 29.999090 seconds) 2020-03-31 09:12:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:03 UTC)" (scheduled at 2020-03-31 09:12:33.330112+00:00) 2020-03-31 09:12:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 33, 331989), True, datetime.datetime(2020, 3, 31, 9, 12, 33, 331989), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:12:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 33, 331989), True, datetime.datetime(2020, 3, 31, 9, 12, 33, 331989), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:12:33,348 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:12:33,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:12:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:33,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:03 UTC)" executed successfully 2020-03-31 09:12:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:12:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:43.539999+00:00 (in 10.172373 seconds) 2020-03-31 09:12:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:33 UTC)" (scheduled at 2020-03-31 09:12:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:12:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:43.539999+00:00 (in 10.172373 seconds) 2020-03-31 09:12:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:33 UTC)" (scheduled at 2020-03-31 09:12:33.366791+00:00) 2020-03-31 09:12:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:12:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 33, 368684), True, datetime.datetime(2020, 3, 31, 9, 12, 33, 368684), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:12:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 33, 368684), True, datetime.datetime(2020, 3, 31, 9, 12, 33, 368684), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:12:33,382 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:12:33,382 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:33,382 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:33,382 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:12:33,382 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:33,382 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:33 UTC)" executed successfully 2020-03-31 09:12:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:44.323352+00:00 (in 9.999040 seconds) 2020-03-31 09:12:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:44 UTC)" (scheduled at 2020-03-31 09:12:34.323352+00:00) 2020-03-31 09:12:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:12:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:44.323352+00:00 (in 9.999040 seconds) 2020-03-31 09:12:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:44 UTC)" (scheduled at 2020-03-31 09:12:34.323352+00:00) 2020-03-31 09:12:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 34, 325293), True, datetime.datetime(2020, 3, 31, 9, 12, 34, 325293), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:12:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 34, 325293), True, datetime.datetime(2020, 3, 31, 9, 12, 34, 325293), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:12:34,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:12:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:44 UTC)" executed successfully 2020-03-31 09:12:34,446 [518] [DEBUG] [app] Starting request: urn:request:a0ecf7b7-9861-4ae3-8bb7-8e82dc758d29 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:34,446 [518] [DEBUG] [app] Starting request: urn:request:a0ecf7b7-9861-4ae3-8bb7-8e82dc758d29 (/health/instance) 2020-03-31 09:12:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:34,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:34,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:34,456 [538] [DEBUG] [app] Starting request: urn:request:614056e1-7586-46b2-bb2c-744f158ac382 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:34,456 [538] [DEBUG] [app] Starting request: urn:request:614056e1-7586-46b2-bb2c-744f158ac382 (/v1/_internal_ping) 2020-03-31 09:12:34,457 [538] [DEBUG] [app] Ending request: urn:request:614056e1-7586-46b2-bb2c-744f158ac382 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:34,457 [538] [DEBUG] [app] Ending request: urn:request:614056e1-7586-46b2-bb2c-744f158ac382 (/v1/_internal_ping) 2020-03-31 09:12:34,458 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:34,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:12:34,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:12:34,458 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:34,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:12:34,461 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:12:34,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:34,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:34,461 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:34,463 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:34,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:34,463 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:34,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:34,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:34,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:34,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:34,491 [518] [DEBUG] [app] Starting request: urn:request:359211e7-053d-4009-ad56-fdc8860e687e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:34,491 [518] [DEBUG] [app] Starting request: urn:request:359211e7-053d-4009-ad56-fdc8860e687e (/_internal_ping) 2020-03-31 09:12:34,492 [518] [DEBUG] [app] Ending request: urn:request:359211e7-053d-4009-ad56-fdc8860e687e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:34,492 [518] [DEBUG] [app] Ending request: urn:request:359211e7-053d-4009-ad56-fdc8860e687e (/_internal_ping) 2020-03-31 09:12:34,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:12:34,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:12:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:34,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:34,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:34,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:34,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:34,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:34,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:34,510 [535] [DEBUG] [app] Starting request: urn:request:b8d94e16-7692-4c31-afad-2bfe7392e35d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:34,510 [535] [DEBUG] [app] Starting request: urn:request:b8d94e16-7692-4c31-afad-2bfe7392e35d (/c1/_internal_ping) 2020-03-31 09:12:34,511 [535] [DEBUG] [app] Ending request: urn:request:b8d94e16-7692-4c31-afad-2bfe7392e35d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:34,511 [535] [DEBUG] [app] Ending request: urn:request:b8d94e16-7692-4c31-afad-2bfe7392e35d (/c1/_internal_ping) 2020-03-31 09:12:34,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:34,511 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:34,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:34,513 [518] [DEBUG] [app] Ending request: urn:request:a0ecf7b7-9861-4ae3-8bb7-8e82dc758d29 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:12:34,511 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:34,514 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:12:34,513 [518] [DEBUG] [app] Ending request: urn:request:a0ecf7b7-9861-4ae3-8bb7-8e82dc758d29 (/health/instance) 2020-03-31 09:12:34,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:12:34,514 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:12:34,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:12:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:38.010934+00:00 (in 1.085770 seconds) 2020-03-31 09:12:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:46 UTC)" (scheduled at 2020-03-31 09:12:36.924497+00:00) 2020-03-31 09:12:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:12:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:38.010934+00:00 (in 1.085770 seconds) 2020-03-31 09:12:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:46 UTC)" (scheduled at 2020-03-31 09:12:36.924497+00:00) 2020-03-31 09:12:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 36, 926212), True, datetime.datetime(2020, 3, 31, 9, 12, 36, 926212), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:12:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 36, 926212), True, datetime.datetime(2020, 3, 31, 9, 12, 36, 926212), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:12:36,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:12:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:46 UTC)" executed successfully 2020-03-31 09:12:38,012 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:38,012 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:38 UTC)" (scheduled at 2020-03-31 09:12:38.010934+00:00) 2020-03-31 09:12:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:12:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:38 UTC)" (scheduled at 2020-03-31 09:12:38.010934+00:00) 2020-03-31 09:12:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:38,013 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:46.924497+00:00 (in 8.912218 seconds) 2020-03-31 09:12:38,013 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:38,013 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:46.924497+00:00 (in 8.912218 seconds) 2020-03-31 09:12:38,013 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:38 UTC)" executed successfully 2020-03-31 09:12:38,272 [526] [DEBUG] [app] Starting request: urn:request:14964684-cba0-48bb-a871-44b379e3c739 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:38,272 [526] [DEBUG] [app] Starting request: urn:request:14964684-cba0-48bb-a871-44b379e3c739 (/health/instance) 2020-03-31 09:12:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:38,280 [556] [DEBUG] [app] Starting request: urn:request:4981b249-80fc-4691-952c-0e5a9ecf5da1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:38,280 [556] [DEBUG] [app] Starting request: urn:request:4981b249-80fc-4691-952c-0e5a9ecf5da1 (/v1/_internal_ping) 2020-03-31 09:12:38,281 [556] [DEBUG] [app] Ending request: urn:request:4981b249-80fc-4691-952c-0e5a9ecf5da1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:38,281 [556] [DEBUG] [app] Ending request: urn:request:4981b249-80fc-4691-952c-0e5a9ecf5da1 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:38,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:12:38,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:38,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:12:38,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:38,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:38,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:38,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:38,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:38,305 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:38,305 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:38,308 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:38,308 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:38,310 [526] [DEBUG] [app] Starting request: urn:request:f5d4e1c5-2310-4823-b3f7-6d9e13fa7405 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:38,310 [526] [DEBUG] [app] Starting request: urn:request:f5d4e1c5-2310-4823-b3f7-6d9e13fa7405 (/_internal_ping) 2020-03-31 09:12:38,311 [526] [DEBUG] [app] Ending request: urn:request:f5d4e1c5-2310-4823-b3f7-6d9e13fa7405 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:38,311 [526] [DEBUG] [app] Ending request: urn:request:f5d4e1c5-2310-4823-b3f7-6d9e13fa7405 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:12:38,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:38,312 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:38,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:38,312 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:38,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:38,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:12:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:12:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:12:38,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:38,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:38,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:38,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:38,324 [543] [DEBUG] [app] Starting request: urn:request:1c803208-74b8-472e-929b-e8962480894a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:38,324 [543] [DEBUG] [app] Starting request: urn:request:1c803208-74b8-472e-929b-e8962480894a (/c1/_internal_ping) 2020-03-31 09:12:38,325 [543] [DEBUG] [app] Ending request: urn:request:1c803208-74b8-472e-929b-e8962480894a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:38,325 [543] [DEBUG] [app] Ending request: urn:request:1c803208-74b8-472e-929b-e8962480894a (/c1/_internal_ping) 2020-03-31 09:12:38,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:38,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:38,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:38,327 [526] [DEBUG] [app] Ending request: urn:request:14964684-cba0-48bb-a871-44b379e3c739 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:12:38,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:38,328 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:38,328 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:12:38,327 [526] [DEBUG] [app] Ending request: urn:request:14964684-cba0-48bb-a871-44b379e3c739 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:38,328 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:38,328 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:12:41,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:12:41,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:41,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:51 UTC)" (scheduled at 2020-03-31 09:12:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:12:41,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:51 UTC)" (scheduled at 2020-03-31 09:12:41.440707+00:00) 2020-03-31 09:12:41,446 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:51.440707+00:00 (in 9.996061 seconds) repositoryactioncounter stdout | 2020-03-31 09:12:41,446 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:51.440707+00:00 (in 9.996061 seconds) 2020-03-31 09:12:41,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:12:41,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:12:41,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:12:41,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:12:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:51 UTC)" executed successfully 2020-03-31 09:12:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:12:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:33.366791+00:00 (in 49.825665 seconds) 2020-03-31 09:12:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:43 UTC)" (scheduled at 2020-03-31 09:12:43.539999+00:00) 2020-03-31 09:12:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:12:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:33.366791+00:00 (in 49.825665 seconds) 2020-03-31 09:12:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:43 UTC)" (scheduled at 2020-03-31 09:12:43.539999+00:00) 2020-03-31 09:12:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:43 UTC)" executed successfully 2020-03-31 09:12:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:49.498966+00:00 (in 5.174749 seconds) 2020-03-31 09:12:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:54 UTC)" (scheduled at 2020-03-31 09:12:44.323352+00:00) 2020-03-31 09:12:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:12:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:49.498966+00:00 (in 5.174749 seconds) 2020-03-31 09:12:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:54 UTC)" (scheduled at 2020-03-31 09:12:44.323352+00:00) 2020-03-31 09:12:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 44, 325060), True, datetime.datetime(2020, 3, 31, 9, 12, 44, 325060), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:12:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 44, 325060), True, datetime.datetime(2020, 3, 31, 9, 12, 44, 325060), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:12:44,339 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:44,339 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:12:44,339 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:44,339 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:54 UTC)" executed successfully 2020-03-31 09:12:44,446 [527] [DEBUG] [app] Starting request: urn:request:56e7e554-bf27-4b5a-b02b-3d4153db2ee8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:44,446 [527] [DEBUG] [app] Starting request: urn:request:56e7e554-bf27-4b5a-b02b-3d4153db2ee8 (/health/instance) 2020-03-31 09:12:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:44,454 [551] [DEBUG] [app] Starting request: urn:request:3c09d285-d580-4d2a-a349-55a60a9b75fb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:44,454 [551] [DEBUG] [app] Starting request: urn:request:3c09d285-d580-4d2a-a349-55a60a9b75fb (/v1/_internal_ping) 2020-03-31 09:12:44,455 [551] [DEBUG] [app] Ending request: urn:request:3c09d285-d580-4d2a-a349-55a60a9b75fb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:44,455 [551] [DEBUG] [app] Ending request: urn:request:3c09d285-d580-4d2a-a349-55a60a9b75fb (/v1/_internal_ping) 2020-03-31 09:12:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:12:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:44,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:12:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:12:44,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:44,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:44,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:44,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:44,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:44,486 [527] [DEBUG] [app] Starting request: urn:request:f1a32b69-cb33-4ae2-a04d-8681d589eb93 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:44,486 [527] [DEBUG] [app] Starting request: urn:request:f1a32b69-cb33-4ae2-a04d-8681d589eb93 (/_internal_ping) 2020-03-31 09:12:44,487 [527] [DEBUG] [app] Ending request: urn:request:f1a32b69-cb33-4ae2-a04d-8681d589eb93 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:44,487 [527] [DEBUG] [app] Ending request: urn:request:f1a32b69-cb33-4ae2-a04d-8681d589eb93 (/_internal_ping) 2020-03-31 09:12:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:12:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:12:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:12:44,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:44,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:44,500 [537] [DEBUG] [app] Starting request: urn:request:7e6cf69b-5f60-43ee-a5cf-9da7ac8aa8bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:44,500 [537] [DEBUG] [app] Starting request: urn:request:7e6cf69b-5f60-43ee-a5cf-9da7ac8aa8bd (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:44,501 [537] [DEBUG] [app] Ending request: urn:request:7e6cf69b-5f60-43ee-a5cf-9da7ac8aa8bd (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:44,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:12:44,501 [537] [DEBUG] [app] Ending request: urn:request:7e6cf69b-5f60-43ee-a5cf-9da7ac8aa8bd (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:44,503 [527] [DEBUG] [app] Ending request: urn:request:56e7e554-bf27-4b5a-b02b-3d4153db2ee8 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:12:44,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:44,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:44,503 [527] [DEBUG] [app] Ending request: urn:request:56e7e554-bf27-4b5a-b02b-3d4153db2ee8 (/health/instance) 2020-03-31 09:12:44,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:12:44,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:44,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:12:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:12:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:15.064514+00:00 (in 29.999071 seconds) 2020-03-31 09:12:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:15 UTC)" (scheduled at 2020-03-31 09:12:45.064514+00:00) gcworker stdout | 2020-03-31 09:12:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:15.064514+00:00 (in 29.999071 seconds) 2020-03-31 09:12:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:15 UTC)" (scheduled at 2020-03-31 09:12:45.064514+00:00) 2020-03-31 09:12:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436365, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:12:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436365, 1209600, 500, 1, 0]) 2020-03-31 09:12:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:12:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:12:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:12:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:15 UTC)" executed successfully 2020-03-31 09:12:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:46,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:56.924497+00:00 (in 9.998281 seconds) 2020-03-31 09:12:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:56 UTC)" (scheduled at 2020-03-31 09:12:46.924497+00:00) 2020-03-31 09:12:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:12:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:56.924497+00:00 (in 9.998281 seconds) 2020-03-31 09:12:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:56 UTC)" (scheduled at 2020-03-31 09:12:46.924497+00:00) 2020-03-31 09:12:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 46, 927038), True, datetime.datetime(2020, 3, 31, 9, 12, 46, 927038), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:12:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 46, 927038), True, datetime.datetime(2020, 3, 31, 9, 12, 46, 927038), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:12:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:46,945 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:12:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:12:56 UTC)" executed successfully 2020-03-31 09:12:48,273 [518] [DEBUG] [app] Starting request: urn:request:35249390-ccdf-401d-83e2-a0f35d57c6df (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:48,273 [518] [DEBUG] [app] Starting request: urn:request:35249390-ccdf-401d-83e2-a0f35d57c6df (/health/instance) 2020-03-31 09:12:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:48,281 [549] [DEBUG] [app] Starting request: urn:request:e19e0e11-b160-46f3-b5f8-c4d3501729be (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:48,281 [549] [DEBUG] [app] Starting request: urn:request:e19e0e11-b160-46f3-b5f8-c4d3501729be (/v1/_internal_ping) 2020-03-31 09:12:48,282 [549] [DEBUG] [app] Ending request: urn:request:e19e0e11-b160-46f3-b5f8-c4d3501729be (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:48,282 [549] [DEBUG] [app] Ending request: urn:request:e19e0e11-b160-46f3-b5f8-c4d3501729be (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:48,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:12:48,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:12:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:12:48,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:12:48,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:48,287 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:48,287 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:48,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:48,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:48,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:48,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:48,312 [518] [DEBUG] [app] Starting request: urn:request:e37b81aa-d106-4825-ac31-fd2314e29a50 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:48,312 [518] [DEBUG] [app] Starting request: urn:request:e37b81aa-d106-4825-ac31-fd2314e29a50 (/_internal_ping) 2020-03-31 09:12:48,313 [518] [DEBUG] [app] Ending request: urn:request:e37b81aa-d106-4825-ac31-fd2314e29a50 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:48,313 [518] [DEBUG] [app] Ending request: urn:request:e37b81aa-d106-4825-ac31-fd2314e29a50 (/_internal_ping) 2020-03-31 09:12:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:48,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:48,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:48,326 [535] [DEBUG] [app] Starting request: urn:request:ad1bfc47-e804-4c74-a769-9f768752fb72 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:48,326 [535] [DEBUG] [app] Starting request: urn:request:ad1bfc47-e804-4c74-a769-9f768752fb72 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:48,326 [535] [DEBUG] [app] Ending request: urn:request:ad1bfc47-e804-4c74-a769-9f768752fb72 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:12:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:48,327 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:12:48,326 [535] [DEBUG] [app] Ending request: urn:request:ad1bfc47-e804-4c74-a769-9f768752fb72 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:48,329 [518] [DEBUG] [app] Ending request: urn:request:35249390-ccdf-401d-83e2-a0f35d57c6df (/health/instance) 2020-03-31 09:12:48,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:12:48,327 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:48,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:12:48,329 [518] [DEBUG] [app] Ending request: urn:request:35249390-ccdf-401d-83e2-a0f35d57c6df (/health/instance) 2020-03-31 09:12:48,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:48,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:12:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:54.323352+00:00 (in 4.823600 seconds) 2020-03-31 09:12:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:49 UTC)" (scheduled at 2020-03-31 09:12:49.498966+00:00) 2020-03-31 09:12:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:12:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:54.323352+00:00 (in 4.823600 seconds) 2020-03-31 09:12:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:49 UTC)" (scheduled at 2020-03-31 09:12:49.498966+00:00) 2020-03-31 09:12:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:49 UTC)" executed successfully 2020-03-31 09:12:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:12:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:59.560326+00:00 (in 8.448733 seconds) 2020-03-31 09:12:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:51 UTC)" (scheduled at 2020-03-31 09:12:51.110610+00:00) 2020-03-31 09:12:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:12:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:12:59.560326+00:00 (in 8.448733 seconds) 2020-03-31 09:12:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:51 UTC)" (scheduled at 2020-03-31 09:12:51.110610+00:00) 2020-03-31 09:12:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 51, 112386), True, datetime.datetime(2020, 3, 31, 9, 12, 51, 112386), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:12:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 51, 112386), True, datetime.datetime(2020, 3, 31, 9, 12, 51, 112386), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:12:51,352 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:51,353 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:12:51,352 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:51,353 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:51,353 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:12:51,353 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:51 UTC)" executed successfully 2020-03-31 09:12:51,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:12:51,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:01.440707+00:00 (in 9.997356 seconds) 2020-03-31 09:12:51,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:01 UTC)" (scheduled at 2020-03-31 09:12:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:12:51,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:01.440707+00:00 (in 9.997356 seconds) 2020-03-31 09:12:51,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:01 UTC)" (scheduled at 2020-03-31 09:12:51.440707+00:00) 2020-03-31 09:12:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:12:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:12:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:51,459 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:12:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:12:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:12:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:01 UTC)" executed successfully 2020-03-31 09:12:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:12:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:22.113221+00:00 (in 29.998837 seconds) 2020-03-31 09:12:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:22 UTC)" (scheduled at 2020-03-31 09:12:52.113221+00:00) securityworker stdout | 2020-03-31 09:12:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:22.113221+00:00 (in 29.998837 seconds) 2020-03-31 09:12:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:22 UTC)" (scheduled at 2020-03-31 09:12:52.113221+00:00) 2020-03-31 09:12:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:12:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:12:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:12:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:12:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:12:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:12:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:12:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:22 UTC)" executed successfully 2020-03-31 09:12:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:12:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:04.323352+00:00 (in 9.998061 seconds) 2020-03-31 09:12:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:04 UTC)" (scheduled at 2020-03-31 09:12:54.323352+00:00) 2020-03-31 09:12:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:12:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:04.323352+00:00 (in 9.998061 seconds) 2020-03-31 09:12:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:04 UTC)" (scheduled at 2020-03-31 09:12:54.323352+00:00) 2020-03-31 09:12:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 54, 326073), True, datetime.datetime(2020, 3, 31, 9, 12, 54, 326073), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:12:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 54, 326073), True, datetime.datetime(2020, 3, 31, 9, 12, 54, 326073), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:12:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:54,344 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:12:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:12:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:04 UTC)" executed successfully 2020-03-31 09:12:54,446 [518] [DEBUG] [app] Starting request: urn:request:cb7f54d0-5eaf-4fae-917d-6c7b1bf43d0b (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:54,446 [518] [DEBUG] [app] Starting request: urn:request:cb7f54d0-5eaf-4fae-917d-6c7b1bf43d0b (/health/instance) 2020-03-31 09:12:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:54,455 [538] [DEBUG] [app] Starting request: urn:request:822e7bfd-61da-4746-9751-eb14fe507986 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:54,455 [538] [DEBUG] [app] Starting request: urn:request:822e7bfd-61da-4746-9751-eb14fe507986 (/v1/_internal_ping) 2020-03-31 09:12:54,456 [538] [DEBUG] [app] Ending request: urn:request:822e7bfd-61da-4746-9751-eb14fe507986 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:54,456 [538] [DEBUG] [app] Ending request: urn:request:822e7bfd-61da-4746-9751-eb14fe507986 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:12:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:12:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:12:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:54,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:54,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:54,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:54,474 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:54,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:54,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:54,490 [518] [DEBUG] [app] Starting request: urn:request:11fb92ea-598c-434e-a16f-2859c3a22368 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:54,490 [518] [DEBUG] [app] Starting request: urn:request:11fb92ea-598c-434e-a16f-2859c3a22368 (/_internal_ping) 2020-03-31 09:12:54,491 [518] [DEBUG] [app] Ending request: urn:request:11fb92ea-598c-434e-a16f-2859c3a22368 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:54,491 [518] [DEBUG] [app] Ending request: urn:request:11fb92ea-598c-434e-a16f-2859c3a22368 (/_internal_ping) 2020-03-31 09:12:54,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:54,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:12:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:12:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:54,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:54,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:54,504 [532] [DEBUG] [app] Starting request: urn:request:c1cd61c0-39e2-43d2-bede-d5cc752786dc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:54,504 [532] [DEBUG] [app] Starting request: urn:request:c1cd61c0-39e2-43d2-bede-d5cc752786dc (/c1/_internal_ping) 2020-03-31 09:12:54,505 [532] [DEBUG] [app] Ending request: urn:request:c1cd61c0-39e2-43d2-bede-d5cc752786dc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:54,505 [532] [DEBUG] [app] Ending request: urn:request:c1cd61c0-39e2-43d2-bede-d5cc752786dc (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:54,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:54,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:12:54,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:12:54,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:54,507 [518] [DEBUG] [app] Ending request: urn:request:cb7f54d0-5eaf-4fae-917d-6c7b1bf43d0b (/health/instance) 2020-03-31 09:12:54,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:54,507 [518] [DEBUG] [app] Ending request: urn:request:cb7f54d0-5eaf-4fae-917d-6c7b1bf43d0b (/health/instance) 2020-03-31 09:12:54,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:54,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:12:54,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:12:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:12:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:06.924497+00:00 (in 9.999095 seconds) 2020-03-31 09:12:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:06 UTC)" (scheduled at 2020-03-31 09:12:56.924497+00:00) 2020-03-31 09:12:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:12:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 56, 926148), True, datetime.datetime(2020, 3, 31, 9, 12, 56, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:12:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:06.924497+00:00 (in 9.999095 seconds) 2020-03-31 09:12:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:06 UTC)" (scheduled at 2020-03-31 09:12:56.924497+00:00) 2020-03-31 09:12:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:12:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 12, 56, 926148), True, datetime.datetime(2020, 3, 31, 9, 12, 56, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:12:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:12:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:12:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:12:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:12:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:06 UTC)" executed successfully 2020-03-31 09:12:58,273 [518] [DEBUG] [app] Starting request: urn:request:875494d9-e1a4-4bd1-8d21-cafdf14ba57b (/health/instance) gunicorn-web stdout | 2020-03-31 09:12:58,273 [518] [DEBUG] [app] Starting request: urn:request:875494d9-e1a4-4bd1-8d21-cafdf14ba57b (/health/instance) 2020-03-31 09:12:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:58,278 [546] [DEBUG] [app] Starting request: urn:request:5b4f56a6-7885-479a-a0ab-bc55b39fbdb5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:58,278 [546] [DEBUG] [app] Starting request: urn:request:5b4f56a6-7885-479a-a0ab-bc55b39fbdb5 (/v1/_internal_ping) 2020-03-31 09:12:58,279 [546] [DEBUG] [app] Ending request: urn:request:5b4f56a6-7885-479a-a0ab-bc55b39fbdb5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:12:58,279 [546] [DEBUG] [app] Ending request: urn:request:5b4f56a6-7885-479a-a0ab-bc55b39fbdb5 (/v1/_internal_ping) 2020-03-31 09:12:58,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:12:58,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:12:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:12:58,283 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:12:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:12:58,283 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:12:58,284 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:12:58,284 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:12:58,285 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:12:58,285 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:12:58,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:12:58,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:12:58,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:12:58,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:12:58,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:58,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:58,308 [527] [DEBUG] [app] Starting request: urn:request:d5d4ef0e-9108-47f4-aa0f-9756abb272ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:58,308 [527] [DEBUG] [app] Starting request: urn:request:d5d4ef0e-9108-47f4-aa0f-9756abb272ac (/_internal_ping) 2020-03-31 09:12:58,309 [527] [DEBUG] [app] Ending request: urn:request:d5d4ef0e-9108-47f4-aa0f-9756abb272ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:12:58,309 [527] [DEBUG] [app] Ending request: urn:request:d5d4ef0e-9108-47f4-aa0f-9756abb272ac (/_internal_ping) 2020-03-31 09:12:58,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:58,310 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:12:58,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:58,310 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:12:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:58,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:12:58,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:12:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:12:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:12:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:12:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:12:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:12:58,322 [543] [DEBUG] [app] Starting request: urn:request:c979b3d8-5444-48b4-a59d-76e767be8892 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:12:58,322 [543] [DEBUG] [app] Starting request: urn:request:c979b3d8-5444-48b4-a59d-76e767be8892 (/c1/_internal_ping) 2020-03-31 09:12:58,323 [543] [DEBUG] [app] Ending request: urn:request:c979b3d8-5444-48b4-a59d-76e767be8892 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:12:58,323 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:12:58,323 [543] [DEBUG] [app] Ending request: urn:request:c979b3d8-5444-48b4-a59d-76e767be8892 (/c1/_internal_ping) 2020-03-31 09:12:58,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:12:58,323 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:12:58,325 [518] [DEBUG] [app] Ending request: urn:request:875494d9-e1a4-4bd1-8d21-cafdf14ba57b (/health/instance) 2020-03-31 09:12:58,325 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:58,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:12:58,325 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) gunicorn-web stdout | 2020-03-31 09:12:58,325 [518] [DEBUG] [app] Ending request: urn:request:875494d9-e1a4-4bd1-8d21-cafdf14ba57b (/health/instance) 2020-03-31 09:12:58,325 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:12:58,325 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:12:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:12:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) 2020-03-31 09:12:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:12:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:12:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:51.110610+00:00 (in 51.549340 seconds) 2020-03-31 09:12:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:59 UTC)" (scheduled at 2020-03-31 09:12:59.560326+00:00) 2020-03-31 09:12:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:12:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:51.110610+00:00 (in 51.549340 seconds) 2020-03-31 09:12:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:59 UTC)" (scheduled at 2020-03-31 09:12:59.560326+00:00) 2020-03-31 09:12:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:12:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:59 UTC)" executed successfully 2020-03-31 09:13:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:13:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:30.337699+00:00 (in 29.999377 seconds) 2020-03-31 09:13:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:30 UTC)" (scheduled at 2020-03-31 09:13:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:13:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:30.337699+00:00 (in 29.999377 seconds) 2020-03-31 09:13:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:30 UTC)" (scheduled at 2020-03-31 09:13:00.337699+00:00) 2020-03-31 09:13:00,339 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 13, 0, 338984), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:13:00,339 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 13, 0, 338984), False, 50, 1, 0]) 2020-03-31 09:13:00,352 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:13:00,352 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:13:00,353 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:00,353 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:13:00,353 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:00,353 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:30 UTC)" executed successfully 2020-03-31 09:13:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:13:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:11.440707+00:00 (in 9.999067 seconds) 2020-03-31 09:13:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:11 UTC)" (scheduled at 2020-03-31 09:13:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:13:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:11.440707+00:00 (in 9.999067 seconds) 2020-03-31 09:13:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:11 UTC)" (scheduled at 2020-03-31 09:13:01.440707+00:00) 2020-03-31 09:13:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:13:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:13:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:13:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:13:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:11 UTC)" executed successfully 2020-03-31 09:13:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:13:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:24.410927+00:00 (in 21.079664 seconds) 2020-03-31 09:13:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:33 UTC)" (scheduled at 2020-03-31 09:13:03.330112+00:00) 2020-03-31 09:13:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:13:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:24.410927+00:00 (in 21.079664 seconds) 2020-03-31 09:13:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:33 UTC)" (scheduled at 2020-03-31 09:13:03.330112+00:00) 2020-03-31 09:13:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 3, 332088), True, datetime.datetime(2020, 3, 31, 9, 13, 3, 332088), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:13:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 3, 332088), True, datetime.datetime(2020, 3, 31, 9, 13, 3, 332088), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:13:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:13:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:13:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:33 UTC)" executed successfully 2020-03-31 09:13:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:13:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:14.323352+00:00 (in 9.999045 seconds) 2020-03-31 09:13:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:14 UTC)" (scheduled at 2020-03-31 09:13:04.323352+00:00) 2020-03-31 09:13:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:13:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:14.323352+00:00 (in 9.999045 seconds) 2020-03-31 09:13:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:14 UTC)" (scheduled at 2020-03-31 09:13:04.323352+00:00) 2020-03-31 09:13:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 4, 325345), True, datetime.datetime(2020, 3, 31, 9, 13, 4, 325345), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:13:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 4, 325345), True, datetime.datetime(2020, 3, 31, 9, 13, 4, 325345), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:13:04,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:13:04,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:04,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:13:04,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:14 UTC)" executed successfully 2020-03-31 09:13:04,447 [527] [DEBUG] [app] Starting request: urn:request:9b043948-5f6f-4676-bc02-71b0552f250e (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:04,447 [527] [DEBUG] [app] Starting request: urn:request:9b043948-5f6f-4676-bc02-71b0552f250e (/health/instance) 2020-03-31 09:13:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:04,455 [542] [DEBUG] [app] Starting request: urn:request:d4c39c72-d33d-4b92-9452-cc515ef89a72 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:04,455 [542] [DEBUG] [app] Starting request: urn:request:d4c39c72-d33d-4b92-9452-cc515ef89a72 (/v1/_internal_ping) 2020-03-31 09:13:04,456 [542] [DEBUG] [app] Ending request: urn:request:d4c39c72-d33d-4b92-9452-cc515ef89a72 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:04,456 [542] [DEBUG] [app] Ending request: urn:request:d4c39c72-d33d-4b92-9452-cc515ef89a72 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:04,457 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:04,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:13:04,457 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:04,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:04,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:04,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:04,459 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:04,459 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:04,461 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:04,461 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:04,461 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:04,461 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:04,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:04,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:04,489 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:04,489 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:04,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:04,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:04,494 [527] [DEBUG] [app] Starting request: urn:request:b7ff1ea9-25e6-4441-ae78-688e09d203e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:04,494 [527] [DEBUG] [app] Starting request: urn:request:b7ff1ea9-25e6-4441-ae78-688e09d203e3 (/_internal_ping) 2020-03-31 09:13:04,495 [527] [DEBUG] [app] Ending request: urn:request:b7ff1ea9-25e6-4441-ae78-688e09d203e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:04,495 [527] [DEBUG] [app] Ending request: urn:request:b7ff1ea9-25e6-4441-ae78-688e09d203e3 (/_internal_ping) 2020-03-31 09:13:04,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:13:04,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:04,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:13:04,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:04,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:04,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:04,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:13:04,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:13:04,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:04,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:04,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:13:04,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:13:04,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:04,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:04,509 [532] [DEBUG] [app] Starting request: urn:request:c67d6b06-ca7e-43d2-8714-e96dfe18e882 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:04,509 [532] [DEBUG] [app] Starting request: urn:request:c67d6b06-ca7e-43d2-8714-e96dfe18e882 (/c1/_internal_ping) 2020-03-31 09:13:04,510 [532] [DEBUG] [app] Ending request: urn:request:c67d6b06-ca7e-43d2-8714-e96dfe18e882 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:04,510 [532] [DEBUG] [app] Ending request: urn:request:c67d6b06-ca7e-43d2-8714-e96dfe18e882 (/c1/_internal_ping) 2020-03-31 09:13:04,511 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:04,510 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:04,511 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:13:04,510 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:04,512 [527] [DEBUG] [app] Ending request: urn:request:9b043948-5f6f-4676-bc02-71b0552f250e (/health/instance) 2020-03-31 09:13:04,513 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:04,512 [527] [DEBUG] [app] Ending request: urn:request:9b043948-5f6f-4676-bc02-71b0552f250e (/health/instance) 2020-03-31 09:13:04,513 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:04,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:13:04,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:13:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:13:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:16.924497+00:00 (in 9.999165 seconds) 2020-03-31 09:13:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:16 UTC)" (scheduled at 2020-03-31 09:13:06.924497+00:00) 2020-03-31 09:13:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:13:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:16.924497+00:00 (in 9.999165 seconds) 2020-03-31 09:13:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:16 UTC)" (scheduled at 2020-03-31 09:13:06.924497+00:00) 2020-03-31 09:13:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 6, 926125), True, datetime.datetime(2020, 3, 31, 9, 13, 6, 926125), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:13:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 6, 926125), True, datetime.datetime(2020, 3, 31, 9, 13, 6, 926125), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:13:06,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:13:06,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:06,944 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:13:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:16 UTC)" executed successfully 2020-03-31 09:13:08,273 [518] [DEBUG] [app] Starting request: urn:request:23f1b169-d9f7-43ad-a9c2-0ea7f8eff80f (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:08,273 [518] [DEBUG] [app] Starting request: urn:request:23f1b169-d9f7-43ad-a9c2-0ea7f8eff80f (/health/instance) 2020-03-31 09:13:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:08,281 [551] [DEBUG] [app] Starting request: urn:request:0dcf0f50-1ea3-452f-bccb-8cb60d70da42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:08,281 [551] [DEBUG] [app] Starting request: urn:request:0dcf0f50-1ea3-452f-bccb-8cb60d70da42 (/v1/_internal_ping) 2020-03-31 09:13:08,282 [551] [DEBUG] [app] Ending request: urn:request:0dcf0f50-1ea3-452f-bccb-8cb60d70da42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:08,282 [551] [DEBUG] [app] Ending request: urn:request:0dcf0f50-1ea3-452f-bccb-8cb60d70da42 (/v1/_internal_ping) 2020-03-31 09:13:08,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:13:08,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:08,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:13:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:08,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:08,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:08,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:08,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:08,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:08,314 [518] [DEBUG] [app] Starting request: urn:request:3e696f5a-91ab-444d-a512-62f45628a226 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:08,314 [518] [DEBUG] [app] Starting request: urn:request:3e696f5a-91ab-444d-a512-62f45628a226 (/_internal_ping) 2020-03-31 09:13:08,315 [518] [DEBUG] [app] Ending request: urn:request:3e696f5a-91ab-444d-a512-62f45628a226 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:08,315 [518] [DEBUG] [app] Ending request: urn:request:3e696f5a-91ab-444d-a512-62f45628a226 (/_internal_ping) 2020-03-31 09:13:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:13:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:13:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:13:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:13:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:08,327 [532] [DEBUG] [app] Starting request: urn:request:8b9e289c-8b62-47d6-a5da-36f69d157d4b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:08,327 [532] [DEBUG] [app] Starting request: urn:request:8b9e289c-8b62-47d6-a5da-36f69d157d4b (/c1/_internal_ping) 2020-03-31 09:13:08,328 [532] [DEBUG] [app] Ending request: urn:request:8b9e289c-8b62-47d6-a5da-36f69d157d4b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:08,328 [532] [DEBUG] [app] Ending request: urn:request:8b9e289c-8b62-47d6-a5da-36f69d157d4b (/c1/_internal_ping) 2020-03-31 09:13:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:08,329 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:08,330 [518] [DEBUG] [app] Ending request: urn:request:23f1b169-d9f7-43ad-a9c2-0ea7f8eff80f (/health/instance) 2020-03-31 09:13:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:13:08,329 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:13:08,330 [518] [DEBUG] [app] Ending request: urn:request:23f1b169-d9f7-43ad-a9c2-0ea7f8eff80f (/health/instance) 2020-03-31 09:13:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:13:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:13:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:21.440707+00:00 (in 9.999231 seconds) 2020-03-31 09:13:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:21 UTC)" (scheduled at 2020-03-31 09:13:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:13:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:21.440707+00:00 (in 9.999231 seconds) 2020-03-31 09:13:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:21 UTC)" (scheduled at 2020-03-31 09:13:11.440707+00:00) 2020-03-31 09:13:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:13:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:13:11,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:13:11,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:11,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:11,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:13:11,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:11,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:21 UTC)" executed successfully 2020-03-31 09:13:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:13:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:24.323352+00:00 (in 9.999028 seconds) 2020-03-31 09:13:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:24 UTC)" (scheduled at 2020-03-31 09:13:14.323352+00:00) 2020-03-31 09:13:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:13:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:24.323352+00:00 (in 9.999028 seconds) 2020-03-31 09:13:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:24 UTC)" (scheduled at 2020-03-31 09:13:14.323352+00:00) 2020-03-31 09:13:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 14, 325442), True, datetime.datetime(2020, 3, 31, 9, 13, 14, 325442), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:13:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 14, 325442), True, datetime.datetime(2020, 3, 31, 9, 13, 14, 325442), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:13:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:13:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:13:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:24 UTC)" executed successfully 2020-03-31 09:13:14,446 [526] [DEBUG] [app] Starting request: urn:request:7621eec6-b437-407d-8891-4ca1ec2a6491 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:14,446 [526] [DEBUG] [app] Starting request: urn:request:7621eec6-b437-407d-8891-4ca1ec2a6491 (/health/instance) 2020-03-31 09:13:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:14,453 [538] [DEBUG] [app] Starting request: urn:request:44740896-d704-41a8-960a-85bac8461812 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:14,453 [538] [DEBUG] [app] Starting request: urn:request:44740896-d704-41a8-960a-85bac8461812 (/v1/_internal_ping) 2020-03-31 09:13:14,454 [538] [DEBUG] [app] Ending request: urn:request:44740896-d704-41a8-960a-85bac8461812 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:14,454 [538] [DEBUG] [app] Ending request: urn:request:44740896-d704-41a8-960a-85bac8461812 (/v1/_internal_ping) 2020-03-31 09:13:14,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:13:14,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:14,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:14,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:14,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:14,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:14,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:14,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:14,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:14,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:14,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:14,480 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:14,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:14,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:14,487 [526] [DEBUG] [app] Starting request: urn:request:d425b102-3366-40fc-a5f2-35afc84c7123 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:14,487 [526] [DEBUG] [app] Starting request: urn:request:d425b102-3366-40fc-a5f2-35afc84c7123 (/_internal_ping) 2020-03-31 09:13:14,488 [526] [DEBUG] [app] Ending request: urn:request:d425b102-3366-40fc-a5f2-35afc84c7123 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:14,488 [526] [DEBUG] [app] Ending request: urn:request:d425b102-3366-40fc-a5f2-35afc84c7123 (/_internal_ping) 2020-03-31 09:13:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:13:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:14,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:13:14,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:14,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:14,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:13:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:13:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:14,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:13:14,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:13:14,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:14,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:14,503 [532] [DEBUG] [app] Starting request: urn:request:9b95f3fa-64f9-4f96-95f1-02dc9f4b5a42 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:14,503 [532] [DEBUG] [app] Starting request: urn:request:9b95f3fa-64f9-4f96-95f1-02dc9f4b5a42 (/c1/_internal_ping) 2020-03-31 09:13:14,503 [532] [DEBUG] [app] Ending request: urn:request:9b95f3fa-64f9-4f96-95f1-02dc9f4b5a42 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:14,503 [532] [DEBUG] [app] Ending request: urn:request:9b95f3fa-64f9-4f96-95f1-02dc9f4b5a42 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:14,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:14,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:13:14,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:14,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:14,506 [526] [DEBUG] [app] Ending request: urn:request:7621eec6-b437-407d-8891-4ca1ec2a6491 (/health/instance) 2020-03-31 09:13:14,506 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:14,506 [526] [DEBUG] [app] Ending request: urn:request:7621eec6-b437-407d-8891-4ca1ec2a6491 (/health/instance) 2020-03-31 09:13:14,506 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:14,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:13:14,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:13:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:13:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:45.064514+00:00 (in 29.998998 seconds) 2020-03-31 09:13:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:45 UTC)" (scheduled at 2020-03-31 09:13:15.064514+00:00) gcworker stdout | 2020-03-31 09:13:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:45.064514+00:00 (in 29.998998 seconds) 2020-03-31 09:13:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:45 UTC)" (scheduled at 2020-03-31 09:13:15.064514+00:00) 2020-03-31 09:13:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436395, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:13:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436395, 1209600, 500, 1, 0]) 2020-03-31 09:13:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:13:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:13:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:13:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:45 UTC)" executed successfully 2020-03-31 09:13:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:13:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:26.924497+00:00 (in 9.999236 seconds) 2020-03-31 09:13:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:26 UTC)" (scheduled at 2020-03-31 09:13:16.924497+00:00) 2020-03-31 09:13:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:13:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:26.924497+00:00 (in 9.999236 seconds) 2020-03-31 09:13:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:26 UTC)" (scheduled at 2020-03-31 09:13:16.924497+00:00) 2020-03-31 09:13:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 16, 925944), True, datetime.datetime(2020, 3, 31, 9, 13, 16, 925944), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:13:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 16, 925944), True, datetime.datetime(2020, 3, 31, 9, 13, 16, 925944), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:13:16,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:13:16,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:13:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:26 UTC)" executed successfully 2020-03-31 09:13:18,273 [518] [DEBUG] [app] Starting request: urn:request:a5e7b415-ee1d-49d6-9436-719093bd3b92 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:18,273 [518] [DEBUG] [app] Starting request: urn:request:a5e7b415-ee1d-49d6-9436-719093bd3b92 (/health/instance) 2020-03-31 09:13:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:18,281 [542] [DEBUG] [app] Starting request: urn:request:c50b2a79-22e6-412a-8f48-c30405afe647 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:18,281 [542] [DEBUG] [app] Starting request: urn:request:c50b2a79-22e6-412a-8f48-c30405afe647 (/v1/_internal_ping) 2020-03-31 09:13:18,282 [542] [DEBUG] [app] Ending request: urn:request:c50b2a79-22e6-412a-8f48-c30405afe647 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:18,282 [542] [DEBUG] [app] Ending request: urn:request:c50b2a79-22e6-412a-8f48-c30405afe647 (/v1/_internal_ping) 2020-03-31 09:13:18,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:13:18,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:18,286 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:18,286 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:18,287 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:18,287 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:18,288 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:18,288 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:18,313 [521] [DEBUG] [app] Starting request: urn:request:1410b18d-7019-4022-9d4d-42336555a655 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:18,313 [521] [DEBUG] [app] Starting request: urn:request:1410b18d-7019-4022-9d4d-42336555a655 (/_internal_ping) 2020-03-31 09:13:18,314 [521] [DEBUG] [app] Ending request: urn:request:1410b18d-7019-4022-9d4d-42336555a655 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:18,314 [521] [DEBUG] [app] Ending request: urn:request:1410b18d-7019-4022-9d4d-42336555a655 (/_internal_ping) 2020-03-31 09:13:18,315 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:13:18,315 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:13:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:13:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:13:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:18,329 [532] [DEBUG] [app] Starting request: urn:request:c9467485-5de2-41ae-9da4-ddbe60078047 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:18,329 [532] [DEBUG] [app] Starting request: urn:request:c9467485-5de2-41ae-9da4-ddbe60078047 (/c1/_internal_ping) 2020-03-31 09:13:18,330 [532] [DEBUG] [app] Ending request: urn:request:c9467485-5de2-41ae-9da4-ddbe60078047 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:18,330 [532] [DEBUG] [app] Ending request: urn:request:c9467485-5de2-41ae-9da4-ddbe60078047 (/c1/_internal_ping) 2020-03-31 09:13:18,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:18,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:18,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:13:18,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:18,332 [518] [DEBUG] [app] Ending request: urn:request:a5e7b415-ee1d-49d6-9436-719093bd3b92 (/health/instance) 2020-03-31 09:13:18,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:18,332 [518] [DEBUG] [app] Ending request: urn:request:a5e7b415-ee1d-49d6-9436-719093bd3b92 (/health/instance) 2020-03-31 09:13:18,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:18,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:13:18,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:13:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:13:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:31.440707+00:00 (in 9.998982 seconds) 2020-03-31 09:13:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:31 UTC)" (scheduled at 2020-03-31 09:13:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:13:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:31.440707+00:00 (in 9.998982 seconds) 2020-03-31 09:13:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:31 UTC)" (scheduled at 2020-03-31 09:13:21.440707+00:00) 2020-03-31 09:13:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:13:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:13:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:13:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:31 UTC)" executed successfully 2020-03-31 09:13:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:13:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:22 UTC)" (scheduled at 2020-03-31 09:13:22.113221+00:00) securityworker stdout | 2020-03-31 09:13:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:22 UTC)" (scheduled at 2020-03-31 09:13:22.113221+00:00) 2020-03-31 09:13:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:13:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:52.113221+00:00 (in 29.999178 seconds) securityworker stdout | 2020-03-31 09:13:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:13:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:52.113221+00:00 (in 29.999178 seconds) 2020-03-31 09:13:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:13:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:13:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:13:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:13:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:13:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:13:52 UTC)" executed successfully 2020-03-31 09:13:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:13:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:34.323352+00:00 (in 9.998812 seconds) 2020-03-31 09:13:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:34 UTC)" (scheduled at 2020-03-31 09:13:24.323352+00:00) 2020-03-31 09:13:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:13:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:34.323352+00:00 (in 9.998812 seconds) 2020-03-31 09:13:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:34 UTC)" (scheduled at 2020-03-31 09:13:24.323352+00:00) 2020-03-31 09:13:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 24, 325302), True, datetime.datetime(2020, 3, 31, 9, 13, 24, 325302), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:13:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 24, 325302), True, datetime.datetime(2020, 3, 31, 9, 13, 24, 325302), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:13:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:13:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:13:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:34 UTC)" executed successfully 2020-03-31 09:13:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:13:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" (scheduled at 2020-03-31 09:13:24.410927+00:00) 2020-03-31 09:13:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:13:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" (scheduled at 2020-03-31 09:13:24.410927+00:00) 2020-03-31 09:13:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:13:24 UTC)" executed successfully 2020-03-31 09:13:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:33.330112+00:00 (in 8.918173 seconds) security_notification_worker stdout | 2020-03-31 09:13:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:33.330112+00:00 (in 8.918173 seconds) 2020-03-31 09:13:24,445 [518] [DEBUG] [app] Starting request: urn:request:e9792f1b-35cc-4be9-8328-1e88c1761561 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:24,445 [518] [DEBUG] [app] Starting request: urn:request:e9792f1b-35cc-4be9-8328-1e88c1761561 (/health/instance) 2020-03-31 09:13:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:24,453 [538] [DEBUG] [app] Starting request: urn:request:e257c65d-29e7-4b5d-b6f6-970d691a55b6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:24,453 [538] [DEBUG] [app] Starting request: urn:request:e257c65d-29e7-4b5d-b6f6-970d691a55b6 (/v1/_internal_ping) 2020-03-31 09:13:24,454 [538] [DEBUG] [app] Ending request: urn:request:e257c65d-29e7-4b5d-b6f6-970d691a55b6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:24,454 [538] [DEBUG] [app] Ending request: urn:request:e257c65d-29e7-4b5d-b6f6-970d691a55b6 (/v1/_internal_ping) 2020-03-31 09:13:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:13:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:24,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:24,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:24,488 [527] [DEBUG] [app] Starting request: urn:request:38deb81e-2b55-4626-b926-3dec06a365e0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:24,488 [527] [DEBUG] [app] Starting request: urn:request:38deb81e-2b55-4626-b926-3dec06a365e0 (/_internal_ping) 2020-03-31 09:13:24,489 [527] [DEBUG] [app] Ending request: urn:request:38deb81e-2b55-4626-b926-3dec06a365e0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:24,489 [527] [DEBUG] [app] Ending request: urn:request:38deb81e-2b55-4626-b926-3dec06a365e0 (/_internal_ping) 2020-03-31 09:13:24,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:13:24,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:13:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:24,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:13:24,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:13:24,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:24,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:13:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:24,503 [532] [DEBUG] [app] Starting request: urn:request:5d3e8977-b8db-4c47-b17a-910ccf0bdbb4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:24,503 [532] [DEBUG] [app] Starting request: urn:request:5d3e8977-b8db-4c47-b17a-910ccf0bdbb4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:24,504 [532] [DEBUG] [app] Ending request: urn:request:5d3e8977-b8db-4c47-b17a-910ccf0bdbb4 (/c1/_internal_ping) 2020-03-31 09:13:24,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:24,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:13:24,504 [532] [DEBUG] [app] Ending request: urn:request:5d3e8977-b8db-4c47-b17a-910ccf0bdbb4 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:24,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:24,507 [518] [DEBUG] [app] Ending request: urn:request:e9792f1b-35cc-4be9-8328-1e88c1761561 (/health/instance) 2020-03-31 09:13:24,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:13:24,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:13:24,507 [518] [DEBUG] [app] Ending request: urn:request:e9792f1b-35cc-4be9-8328-1e88c1761561 (/health/instance) 2020-03-31 09:13:24,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:13:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:13:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:13:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:36.924497+00:00 (in 9.999143 seconds) 2020-03-31 09:13:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:36 UTC)" (scheduled at 2020-03-31 09:13:26.924497+00:00) 2020-03-31 09:13:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:13:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:36.924497+00:00 (in 9.999143 seconds) 2020-03-31 09:13:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:36 UTC)" (scheduled at 2020-03-31 09:13:26.924497+00:00) 2020-03-31 09:13:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 26, 926385), True, datetime.datetime(2020, 3, 31, 9, 13, 26, 926385), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:13:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 26, 926385), True, datetime.datetime(2020, 3, 31, 9, 13, 26, 926385), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:13:26,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:13:26,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:13:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:36 UTC)" executed successfully 2020-03-31 09:13:28,278 [526] [DEBUG] [app] Starting request: urn:request:25b1fdb8-16c9-4e56-b745-b1c247ea5701 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:28,278 [526] [DEBUG] [app] Starting request: urn:request:25b1fdb8-16c9-4e56-b745-b1c247ea5701 (/health/instance) 2020-03-31 09:13:28,280 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:28,280 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:28,285 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:28,285 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:28,286 [547] [DEBUG] [app] Starting request: urn:request:bf96c5ec-8f85-4390-96b9-f06a2ba9411d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:28,286 [547] [DEBUG] [app] Starting request: urn:request:bf96c5ec-8f85-4390-96b9-f06a2ba9411d (/v1/_internal_ping) 2020-03-31 09:13:28,288 [547] [DEBUG] [app] Ending request: urn:request:bf96c5ec-8f85-4390-96b9-f06a2ba9411d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:28,288 [547] [DEBUG] [app] Ending request: urn:request:bf96c5ec-8f85-4390-96b9-f06a2ba9411d (/v1/_internal_ping) 2020-03-31 09:13:28,289 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:13:28,289 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:28,289 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:28,290 [526] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:13:28,289 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:28,290 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:28,292 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:28,292 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:28,293 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:28,293 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:28,294 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:28,294 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:28,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:28,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:28,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:28,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:28,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:28,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:28,317 [526] [DEBUG] [app] Starting request: urn:request:fed64067-0846-407a-b8e5-edad904a7078 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:28,317 [526] [DEBUG] [app] Starting request: urn:request:fed64067-0846-407a-b8e5-edad904a7078 (/_internal_ping) 2020-03-31 09:13:28,317 [526] [DEBUG] [app] Ending request: urn:request:fed64067-0846-407a-b8e5-edad904a7078 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:28,317 [526] [DEBUG] [app] Ending request: urn:request:fed64067-0846-407a-b8e5-edad904a7078 (/_internal_ping) 2020-03-31 09:13:28,318 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:28,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:13:28,318 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:28,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:13:28,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:28,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:28,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:13:28,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:13:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:28,329 [532] [DEBUG] [app] Starting request: urn:request:667ba292-65ad-4dbc-86cd-da2a132a2773 (/c1/_internal_ping) 2020-03-31 09:13:28,330 [532] [DEBUG] [app] Ending request: urn:request:667ba292-65ad-4dbc-86cd-da2a132a2773 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:28,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:28,332 [526] [DEBUG] [app] Ending request: urn:request:25b1fdb8-16c9-4e56-b745-b1c247ea5701 (/health/instance) 2020-03-31 09:13:28,333 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:28,333 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-verbs stdout | 2020-03-31 09:13:28,329 [532] [DEBUG] [app] Starting request: urn:request:667ba292-65ad-4dbc-86cd-da2a132a2773 (/c1/_internal_ping) 2020-03-31 09:13:28,332 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:13:28,330 [532] [DEBUG] [app] Ending request: urn:request:667ba292-65ad-4dbc-86cd-da2a132a2773 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:28,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:28,332 [526] [DEBUG] [app] Ending request: urn:request:25b1fdb8-16c9-4e56-b745-b1c247ea5701 (/health/instance) 2020-03-31 09:13:28,333 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:28,333 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:13:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-verbs stdout | 2020-03-31 09:13:28,332 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:13:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:00.337699+00:00 (in 29.999145 seconds) 2020-03-31 09:13:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:00 UTC)" (scheduled at 2020-03-31 09:13:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:13:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:00.337699+00:00 (in 29.999145 seconds) 2020-03-31 09:13:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:00 UTC)" (scheduled at 2020-03-31 09:13:30.337699+00:00) 2020-03-31 09:13:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 13, 30, 339342), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:13:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 13, 30, 339342), False, 50, 1, 0]) 2020-03-31 09:13:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:13:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:13:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:13:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:00 UTC)" executed successfully 2020-03-31 09:13:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:13:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:41 UTC)" (scheduled at 2020-03-31 09:13:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:13:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:41 UTC)" (scheduled at 2020-03-31 09:13:31.440707+00:00) 2020-03-31 09:13:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:41.440707+00:00 (in 9.998935 seconds) repositoryactioncounter stdout | 2020-03-31 09:13:31,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:41.440707+00:00 (in 9.998935 seconds) 2020-03-31 09:13:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:13:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:13:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:13:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:13:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:41 UTC)" executed successfully 2020-03-31 09:13:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:13:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:03 UTC)" (scheduled at 2020-03-31 09:13:33.330112+00:00) 2020-03-31 09:13:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:03.330112+00:00 (in 29.998663 seconds) security_notification_worker stdout | 2020-03-31 09:13:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:03 UTC)" (scheduled at 2020-03-31 09:13:33.330112+00:00) 2020-03-31 09:13:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:03.330112+00:00 (in 29.998663 seconds) 2020-03-31 09:13:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 33, 332401), True, datetime.datetime(2020, 3, 31, 9, 13, 33, 332401), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:13:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 33, 332401), True, datetime.datetime(2020, 3, 31, 9, 13, 33, 332401), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:13:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:33,349 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:13:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:33,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:13:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:03 UTC)" executed successfully 2020-03-31 09:13:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:13:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:43.539999+00:00 (in 10.172282 seconds) 2020-03-31 09:13:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:33 UTC)" (scheduled at 2020-03-31 09:13:33.366791+00:00) 2020-03-31 09:13:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:13:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:43.539999+00:00 (in 10.172282 seconds) 2020-03-31 09:13:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:33 UTC)" (scheduled at 2020-03-31 09:13:33.366791+00:00) 2020-03-31 09:13:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 33, 368389), True, datetime.datetime(2020, 3, 31, 9, 13, 33, 368389), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:13:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 33, 368389), True, datetime.datetime(2020, 3, 31, 9, 13, 33, 368389), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:13:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:13:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:13:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:33 UTC)" executed successfully 2020-03-31 09:13:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:13:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:44.323352+00:00 (in 9.999225 seconds) 2020-03-31 09:13:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:44 UTC)" (scheduled at 2020-03-31 09:13:34.323352+00:00) 2020-03-31 09:13:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:13:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:44.323352+00:00 (in 9.999225 seconds) 2020-03-31 09:13:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:44 UTC)" (scheduled at 2020-03-31 09:13:34.323352+00:00) 2020-03-31 09:13:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 34, 324996), True, datetime.datetime(2020, 3, 31, 9, 13, 34, 324996), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:13:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 34, 324996), True, datetime.datetime(2020, 3, 31, 9, 13, 34, 324996), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:13:34,338 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:13:34,338 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:34,338 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:34,339 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:13:34,338 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:34,339 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:44 UTC)" executed successfully 2020-03-31 09:13:34,446 [518] [DEBUG] [app] Starting request: urn:request:e97674e3-a799-40be-94b6-3cf9cfcabc98 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:34,446 [518] [DEBUG] [app] Starting request: urn:request:e97674e3-a799-40be-94b6-3cf9cfcabc98 (/health/instance) 2020-03-31 09:13:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:34,454 [538] [DEBUG] [app] Starting request: urn:request:0d264566-099e-4ebe-8934-16182211f42e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:34,454 [538] [DEBUG] [app] Starting request: urn:request:0d264566-099e-4ebe-8934-16182211f42e (/v1/_internal_ping) 2020-03-31 09:13:34,455 [538] [DEBUG] [app] Ending request: urn:request:0d264566-099e-4ebe-8934-16182211f42e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:34,455 [538] [DEBUG] [app] Ending request: urn:request:0d264566-099e-4ebe-8934-16182211f42e (/v1/_internal_ping) 2020-03-31 09:13:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:13:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:13:34,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:13:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:13:34,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:34,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:34,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:34,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:34,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:34,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:34,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:34,492 [527] [DEBUG] [app] Starting request: urn:request:c7f10d65-976c-4192-99d5-a5ee3caecede (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:34,492 [527] [DEBUG] [app] Starting request: urn:request:c7f10d65-976c-4192-99d5-a5ee3caecede (/_internal_ping) 2020-03-31 09:13:34,493 [527] [DEBUG] [app] Ending request: urn:request:c7f10d65-976c-4192-99d5-a5ee3caecede (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:34,493 [527] [DEBUG] [app] Ending request: urn:request:c7f10d65-976c-4192-99d5-a5ee3caecede (/_internal_ping) 2020-03-31 09:13:34,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:13:34,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:13:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:13:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:13:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:13:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:34,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:34,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:34,506 [537] [DEBUG] [app] Starting request: urn:request:2c6fb20a-cc18-443a-8545-7c31d60b6d90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:34,506 [537] [DEBUG] [app] Starting request: urn:request:2c6fb20a-cc18-443a-8545-7c31d60b6d90 (/c1/_internal_ping) 2020-03-31 09:13:34,507 [537] [DEBUG] [app] Ending request: urn:request:2c6fb20a-cc18-443a-8545-7c31d60b6d90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:34,507 [537] [DEBUG] [app] Ending request: urn:request:2c6fb20a-cc18-443a-8545-7c31d60b6d90 (/c1/_internal_ping) 2020-03-31 09:13:34,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:34,508 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:34,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:34,509 [518] [DEBUG] [app] Ending request: urn:request:e97674e3-a799-40be-94b6-3cf9cfcabc98 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:13:34,508 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:34,509 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:13:34,509 [518] [DEBUG] [app] Ending request: urn:request:e97674e3-a799-40be-94b6-3cf9cfcabc98 (/health/instance) 2020-03-31 09:13:34,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:13:34,509 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:13:34,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:13:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:38.010934+00:00 (in 1.085675 seconds) 2020-03-31 09:13:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:46 UTC)" (scheduled at 2020-03-31 09:13:36.924497+00:00) 2020-03-31 09:13:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:13:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:13:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:38.010934+00:00 (in 1.085675 seconds) 2020-03-31 09:13:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:46 UTC)" (scheduled at 2020-03-31 09:13:36.924497+00:00) 2020-03-31 09:13:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 36, 926025), True, datetime.datetime(2020, 3, 31, 9, 13, 36, 926025), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:13:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 36, 926025), True, datetime.datetime(2020, 3, 31, 9, 13, 36, 926025), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:13:36,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:13:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:13:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:46 UTC)" executed successfully 2020-03-31 09:13:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:13:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:46.924497+00:00 (in 8.912749 seconds) 2020-03-31 09:13:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:38 UTC)" (scheduled at 2020-03-31 09:13:38.010934+00:00) 2020-03-31 09:13:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:13:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:46.924497+00:00 (in 8.912749 seconds) 2020-03-31 09:13:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:38 UTC)" (scheduled at 2020-03-31 09:13:38.010934+00:00) 2020-03-31 09:13:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:38 UTC)" executed successfully 2020-03-31 09:13:38,273 [527] [DEBUG] [app] Starting request: urn:request:791215ce-94a7-44e4-b7b2-3b33690b9bf8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:38,273 [527] [DEBUG] [app] Starting request: urn:request:791215ce-94a7-44e4-b7b2-3b33690b9bf8 (/health/instance) 2020-03-31 09:13:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:38,280 [548] [DEBUG] [app] Starting request: urn:request:5e11d4bd-0a8e-44ef-85e5-24e07169a572 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:38,280 [548] [DEBUG] [app] Starting request: urn:request:5e11d4bd-0a8e-44ef-85e5-24e07169a572 (/v1/_internal_ping) 2020-03-31 09:13:38,281 [548] [DEBUG] [app] Ending request: urn:request:5e11d4bd-0a8e-44ef-85e5-24e07169a572 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:38,281 [548] [DEBUG] [app] Ending request: urn:request:5e11d4bd-0a8e-44ef-85e5-24e07169a572 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:38,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:13:38,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:38,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:38,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:38,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:38,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:38,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:38,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:38,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:38,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:38,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:38,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:38,316 [527] [DEBUG] [app] Starting request: urn:request:eae33914-179b-4926-b5b2-e47476e0128c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:38,316 [527] [DEBUG] [app] Starting request: urn:request:eae33914-179b-4926-b5b2-e47476e0128c (/_internal_ping) 2020-03-31 09:13:38,317 [527] [DEBUG] [app] Ending request: urn:request:eae33914-179b-4926-b5b2-e47476e0128c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:38,317 [527] [DEBUG] [app] Ending request: urn:request:eae33914-179b-4926-b5b2-e47476e0128c (/_internal_ping) 2020-03-31 09:13:38,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:38,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:13:38,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:38,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:13:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:13:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:13:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:38,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:38,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:38,330 [537] [DEBUG] [app] Starting request: urn:request:60c55029-12b2-4b7b-99a0-52baa62316c0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:38,330 [537] [DEBUG] [app] Starting request: urn:request:60c55029-12b2-4b7b-99a0-52baa62316c0 (/c1/_internal_ping) 2020-03-31 09:13:38,331 [537] [DEBUG] [app] Ending request: urn:request:60c55029-12b2-4b7b-99a0-52baa62316c0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:38,331 [537] [DEBUG] [app] Ending request: urn:request:60c55029-12b2-4b7b-99a0-52baa62316c0 (/c1/_internal_ping) 2020-03-31 09:13:38,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:38,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:38,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:38,333 [527] [DEBUG] [app] Ending request: urn:request:791215ce-94a7-44e4-b7b2-3b33690b9bf8 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:13:38,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:38,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:38,333 [527] [DEBUG] [app] Ending request: urn:request:791215ce-94a7-44e4-b7b2-3b33690b9bf8 (/health/instance) 2020-03-31 09:13:38,334 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:13:38,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:38,334 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:13:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:13:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:51.440707+00:00 (in 9.997936 seconds) 2020-03-31 09:13:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:51 UTC)" (scheduled at 2020-03-31 09:13:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:13:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:51.440707+00:00 (in 9.997936 seconds) 2020-03-31 09:13:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:51 UTC)" (scheduled at 2020-03-31 09:13:41.440707+00:00) 2020-03-31 09:13:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:13:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:13:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:41,462 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:13:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:13:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:51 UTC)" executed successfully 2020-03-31 09:13:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:13:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:43,540 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:43 UTC)" (scheduled at 2020-03-31 09:13:43.539999+00:00) 2020-03-31 09:13:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:33.366791+00:00 (in 49.826092 seconds) 2020-03-31 09:13:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:13:43,540 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:43 UTC)" (scheduled at 2020-03-31 09:13:43.539999+00:00) 2020-03-31 09:13:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:33.366791+00:00 (in 49.826092 seconds) 2020-03-31 09:13:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:13:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:43 UTC)" executed successfully 2020-03-31 09:13:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:13:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:49.498966+00:00 (in 5.174695 seconds) 2020-03-31 09:13:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:54 UTC)" (scheduled at 2020-03-31 09:13:44.323352+00:00) 2020-03-31 09:13:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:13:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:49.498966+00:00 (in 5.174695 seconds) 2020-03-31 09:13:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:54 UTC)" (scheduled at 2020-03-31 09:13:44.323352+00:00) 2020-03-31 09:13:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 44, 325292), True, datetime.datetime(2020, 3, 31, 9, 13, 44, 325292), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:13:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 44, 325292), True, datetime.datetime(2020, 3, 31, 9, 13, 44, 325292), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:13:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:44,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:13:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:13:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:54 UTC)" executed successfully 2020-03-31 09:13:44,446 [527] [DEBUG] [app] Starting request: urn:request:70087484-456b-4429-a0b0-a94999c28ab1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:44,446 [527] [DEBUG] [app] Starting request: urn:request:70087484-456b-4429-a0b0-a94999c28ab1 (/health/instance) 2020-03-31 09:13:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:44,453 [554] [DEBUG] [app] Starting request: urn:request:e287c665-f941-484d-8efc-e62feffa0c36 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:44,453 [554] [DEBUG] [app] Starting request: urn:request:e287c665-f941-484d-8efc-e62feffa0c36 (/v1/_internal_ping) 2020-03-31 09:13:44,454 [554] [DEBUG] [app] Ending request: urn:request:e287c665-f941-484d-8efc-e62feffa0c36 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:44,454 [554] [DEBUG] [app] Ending request: urn:request:e287c665-f941-484d-8efc-e62feffa0c36 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:44,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:13:44,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:44,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:44,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:44,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:44,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:44,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:44,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:44,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:44,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:44,485 [527] [DEBUG] [app] Starting request: urn:request:0a37145b-b9cd-4dba-b59f-0063d19ef65b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:44,485 [527] [DEBUG] [app] Starting request: urn:request:0a37145b-b9cd-4dba-b59f-0063d19ef65b (/_internal_ping) 2020-03-31 09:13:44,486 [527] [DEBUG] [app] Ending request: urn:request:0a37145b-b9cd-4dba-b59f-0063d19ef65b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:44,486 [527] [DEBUG] [app] Ending request: urn:request:0a37145b-b9cd-4dba-b59f-0063d19ef65b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:13:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:44,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:44,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:13:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:13:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:13:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:44,499 [537] [DEBUG] [app] Starting request: urn:request:ecb6def4-f922-4da9-a22d-891aec38cf8b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:44,499 [537] [DEBUG] [app] Starting request: urn:request:ecb6def4-f922-4da9-a22d-891aec38cf8b (/c1/_internal_ping) 2020-03-31 09:13:44,500 [537] [DEBUG] [app] Ending request: urn:request:ecb6def4-f922-4da9-a22d-891aec38cf8b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:44,500 [537] [DEBUG] [app] Ending request: urn:request:ecb6def4-f922-4da9-a22d-891aec38cf8b (/c1/_internal_ping) 2020-03-31 09:13:44,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:44,500 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:44,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:13:44,500 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:44,502 [527] [DEBUG] [app] Ending request: urn:request:70087484-456b-4429-a0b0-a94999c28ab1 (/health/instance) 2020-03-31 09:13:44,502 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:44,502 [527] [DEBUG] [app] Ending request: urn:request:70087484-456b-4429-a0b0-a94999c28ab1 (/health/instance) 2020-03-31 09:13:44,502 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:44,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:13:44,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:13:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:13:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:15.064514+00:00 (in 29.999037 seconds) 2020-03-31 09:13:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:15 UTC)" (scheduled at 2020-03-31 09:13:45.064514+00:00) gcworker stdout | 2020-03-31 09:13:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:15.064514+00:00 (in 29.999037 seconds) 2020-03-31 09:13:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:15 UTC)" (scheduled at 2020-03-31 09:13:45.064514+00:00) 2020-03-31 09:13:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436425, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:13:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436425, 1209600, 500, 1, 0]) 2020-03-31 09:13:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:13:45,084 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:13:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:13:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:13:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:15 UTC)" executed successfully 2020-03-31 09:13:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:13:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:56.924497+00:00 (in 9.999054 seconds) 2020-03-31 09:13:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:56 UTC)" (scheduled at 2020-03-31 09:13:46.924497+00:00) 2020-03-31 09:13:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:13:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:56.924497+00:00 (in 9.999054 seconds) 2020-03-31 09:13:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:56 UTC)" (scheduled at 2020-03-31 09:13:46.924497+00:00) 2020-03-31 09:13:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 46, 926381), True, datetime.datetime(2020, 3, 31, 9, 13, 46, 926381), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:13:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 46, 926381), True, datetime.datetime(2020, 3, 31, 9, 13, 46, 926381), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:13:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:13:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:13:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:13:56 UTC)" executed successfully 2020-03-31 09:13:48,273 [527] [DEBUG] [app] Starting request: urn:request:3a42223c-7553-4476-beb3-c3e2fd762012 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:48,273 [527] [DEBUG] [app] Starting request: urn:request:3a42223c-7553-4476-beb3-c3e2fd762012 (/health/instance) 2020-03-31 09:13:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:48,277 [548] [DEBUG] [app] Starting request: urn:request:cb408975-4b96-404a-9697-84f3d83c613f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:48,277 [548] [DEBUG] [app] Starting request: urn:request:cb408975-4b96-404a-9697-84f3d83c613f (/v1/_internal_ping) 2020-03-31 09:13:48,278 [548] [DEBUG] [app] Ending request: urn:request:cb408975-4b96-404a-9697-84f3d83c613f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:48,278 [548] [DEBUG] [app] Ending request: urn:request:cb408975-4b96-404a-9697-84f3d83c613f (/v1/_internal_ping) 2020-03-31 09:13:48,279 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:13:48,279 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:48,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:48,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:13:48,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:48,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:13:48,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:13:48,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:48,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:48,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:48,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:48,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:48,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:48,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:48,302 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:48,302 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:48,306 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:48,306 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:48,307 [527] [DEBUG] [app] Starting request: urn:request:34531af0-70bf-406e-93e9-c767b0f43a35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:48,307 [527] [DEBUG] [app] Starting request: urn:request:34531af0-70bf-406e-93e9-c767b0f43a35 (/_internal_ping) 2020-03-31 09:13:48,308 [527] [DEBUG] [app] Ending request: urn:request:34531af0-70bf-406e-93e9-c767b0f43a35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:48,308 [527] [DEBUG] [app] Ending request: urn:request:34531af0-70bf-406e-93e9-c767b0f43a35 (/_internal_ping) 2020-03-31 09:13:48,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:48,309 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:13:48,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:48,309 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:13:48,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:48,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:13:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:13:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:13:48,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:48,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:48,321 [537] [DEBUG] [app] Starting request: urn:request:dbb08dd7-b10e-4cb7-a09a-523a5e3d8fae (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:48,321 [537] [DEBUG] [app] Starting request: urn:request:dbb08dd7-b10e-4cb7-a09a-523a5e3d8fae (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:48,322 [537] [DEBUG] [app] Ending request: urn:request:dbb08dd7-b10e-4cb7-a09a-523a5e3d8fae (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:48,323 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:13:48,322 [537] [DEBUG] [app] Ending request: urn:request:dbb08dd7-b10e-4cb7-a09a-523a5e3d8fae (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:13:48,323 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:48,324 [527] [DEBUG] [app] Ending request: urn:request:3a42223c-7553-4476-beb3-c3e2fd762012 (/health/instance) 2020-03-31 09:13:48,325 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:48,324 [527] [DEBUG] [app] Ending request: urn:request:3a42223c-7553-4476-beb3-c3e2fd762012 (/health/instance) 2020-03-31 09:13:48,325 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:48,325 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) gunicorn-web stdout | 2020-03-31 09:13:48,325 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) 2020-03-31 09:13:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:13:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:54.323352+00:00 (in 4.823102 seconds) 2020-03-31 09:13:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:49 UTC)" (scheduled at 2020-03-31 09:13:49.498966+00:00) 2020-03-31 09:13:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:13:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:54.323352+00:00 (in 4.823102 seconds) 2020-03-31 09:13:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:49 UTC)" (scheduled at 2020-03-31 09:13:49.498966+00:00) 2020-03-31 09:13:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:49 UTC)" executed successfully 2020-03-31 09:13:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:13:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:59.560326+00:00 (in 8.448750 seconds) 2020-03-31 09:13:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:51 UTC)" (scheduled at 2020-03-31 09:13:51.110610+00:00) 2020-03-31 09:13:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:13:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:13:59.560326+00:00 (in 8.448750 seconds) 2020-03-31 09:13:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:51 UTC)" (scheduled at 2020-03-31 09:13:51.110610+00:00) 2020-03-31 09:13:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 51, 112539), True, datetime.datetime(2020, 3, 31, 9, 13, 51, 112539), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:13:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 51, 112539), True, datetime.datetime(2020, 3, 31, 9, 13, 51, 112539), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:13:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:13:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:13:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:51 UTC)" executed successfully 2020-03-31 09:13:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:13:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:01.440707+00:00 (in 9.999063 seconds) 2020-03-31 09:13:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:01 UTC)" (scheduled at 2020-03-31 09:13:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:13:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:01.440707+00:00 (in 9.999063 seconds) 2020-03-31 09:13:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:01 UTC)" (scheduled at 2020-03-31 09:13:51.440707+00:00) 2020-03-31 09:13:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:13:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:13:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:13:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:13:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:13:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:01 UTC)" executed successfully 2020-03-31 09:13:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:13:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:22.113221+00:00 (in 29.997535 seconds) 2020-03-31 09:13:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:22 UTC)" (scheduled at 2020-03-31 09:13:52.113221+00:00) securityworker stdout | 2020-03-31 09:13:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:22.113221+00:00 (in 29.997535 seconds) 2020-03-31 09:13:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:22 UTC)" (scheduled at 2020-03-31 09:13:52.113221+00:00) 2020-03-31 09:13:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:13:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:13:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:13:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:13:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:13:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:13:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:13:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:22 UTC)" executed successfully 2020-03-31 09:13:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:13:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:04.323352+00:00 (in 9.999170 seconds) 2020-03-31 09:13:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:04 UTC)" (scheduled at 2020-03-31 09:13:54.323352+00:00) 2020-03-31 09:13:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:13:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:04.323352+00:00 (in 9.999170 seconds) 2020-03-31 09:13:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:04 UTC)" (scheduled at 2020-03-31 09:13:54.323352+00:00) 2020-03-31 09:13:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 54, 325396), True, datetime.datetime(2020, 3, 31, 9, 13, 54, 325396), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:13:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 54, 325396), True, datetime.datetime(2020, 3, 31, 9, 13, 54, 325396), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:13:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:13:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:13:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:04 UTC)" executed successfully 2020-03-31 09:13:54,446 [518] [DEBUG] [app] Starting request: urn:request:44aff239-6bd3-422d-8896-fd490b4c1b2e (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:54,446 [518] [DEBUG] [app] Starting request: urn:request:44aff239-6bd3-422d-8896-fd490b4c1b2e (/health/instance) 2020-03-31 09:13:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:54,454 [544] [DEBUG] [app] Starting request: urn:request:101a14df-c169-4193-9ace-451aca2b938f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:54,454 [544] [DEBUG] [app] Starting request: urn:request:101a14df-c169-4193-9ace-451aca2b938f (/v1/_internal_ping) 2020-03-31 09:13:54,455 [544] [DEBUG] [app] Ending request: urn:request:101a14df-c169-4193-9ace-451aca2b938f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:54,455 [544] [DEBUG] [app] Ending request: urn:request:101a14df-c169-4193-9ace-451aca2b938f (/v1/_internal_ping) 2020-03-31 09:13:54,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:13:54,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:13:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:54,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:54,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:54,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:54,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:54,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:54,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:54,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:54,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:54,488 [518] [DEBUG] [app] Starting request: urn:request:187a3ae3-d16d-4e42-8a91-4e3b461a12f7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:54,488 [518] [DEBUG] [app] Starting request: urn:request:187a3ae3-d16d-4e42-8a91-4e3b461a12f7 (/_internal_ping) 2020-03-31 09:13:54,489 [518] [DEBUG] [app] Ending request: urn:request:187a3ae3-d16d-4e42-8a91-4e3b461a12f7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:54,489 [518] [DEBUG] [app] Ending request: urn:request:187a3ae3-d16d-4e42-8a91-4e3b461a12f7 (/_internal_ping) 2020-03-31 09:13:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:13:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:13:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:13:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:13:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:13:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:13:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:13:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:54,501 [543] [DEBUG] [app] Starting request: urn:request:150f7e37-220f-4773-9b35-f01bfcae0714 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:54,501 [543] [DEBUG] [app] Starting request: urn:request:150f7e37-220f-4773-9b35-f01bfcae0714 (/c1/_internal_ping) 2020-03-31 09:13:54,502 [543] [DEBUG] [app] Ending request: urn:request:150f7e37-220f-4773-9b35-f01bfcae0714 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:54,502 [543] [DEBUG] [app] Ending request: urn:request:150f7e37-220f-4773-9b35-f01bfcae0714 (/c1/_internal_ping) 2020-03-31 09:13:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:54,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:13:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:13:54,505 [518] [DEBUG] [app] Ending request: urn:request:44aff239-6bd3-422d-8896-fd490b4c1b2e (/health/instance) 2020-03-31 09:13:54,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:13:54,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:54,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:13:54,505 [518] [DEBUG] [app] Ending request: urn:request:44aff239-6bd3-422d-8896-fd490b4c1b2e (/health/instance) 2020-03-31 09:13:54,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:13:54,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:13:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:13:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:06.924497+00:00 (in 9.999191 seconds) 2020-03-31 09:13:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:06 UTC)" (scheduled at 2020-03-31 09:13:56.924497+00:00) 2020-03-31 09:13:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:13:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:06.924497+00:00 (in 9.999191 seconds) 2020-03-31 09:13:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:06 UTC)" (scheduled at 2020-03-31 09:13:56.924497+00:00) 2020-03-31 09:13:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:13:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 56, 925921), True, datetime.datetime(2020, 3, 31, 9, 13, 56, 925921), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:13:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 13, 56, 925921), True, datetime.datetime(2020, 3, 31, 9, 13, 56, 925921), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:13:56,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:13:56,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:13:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:13:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:13:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:06 UTC)" executed successfully 2020-03-31 09:13:58,273 [526] [DEBUG] [app] Starting request: urn:request:c67727ec-052c-49dc-bb4e-c9973207e810 (/health/instance) gunicorn-web stdout | 2020-03-31 09:13:58,273 [526] [DEBUG] [app] Starting request: urn:request:c67727ec-052c-49dc-bb4e-c9973207e810 (/health/instance) 2020-03-31 09:13:58,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:58,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:58,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:58,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:58,281 [553] [DEBUG] [app] Starting request: urn:request:f3521486-0bd5-41e0-bc98-2905bc559257 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:58,281 [553] [DEBUG] [app] Starting request: urn:request:f3521486-0bd5-41e0-bc98-2905bc559257 (/v1/_internal_ping) 2020-03-31 09:13:58,282 [553] [DEBUG] [app] Ending request: urn:request:f3521486-0bd5-41e0-bc98-2905bc559257 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:13:58,282 [553] [DEBUG] [app] Ending request: urn:request:f3521486-0bd5-41e0-bc98-2905bc559257 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:58,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:58,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:13:58,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:58,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:13:58,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:58,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:13:58,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:13:58,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:13:58,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:13:58,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:13:58,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:13:58,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:13:58,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:13:58,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:13:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:13:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:13:58,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:58,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:58,313 [526] [DEBUG] [app] Starting request: urn:request:0558ffe6-96eb-4ddd-a338-c5ecf17a68df (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:58,313 [526] [DEBUG] [app] Starting request: urn:request:0558ffe6-96eb-4ddd-a338-c5ecf17a68df (/_internal_ping) 2020-03-31 09:13:58,313 [526] [DEBUG] [app] Ending request: urn:request:0558ffe6-96eb-4ddd-a338-c5ecf17a68df (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:58,313 [526] [DEBUG] [app] Ending request: urn:request:0558ffe6-96eb-4ddd-a338-c5ecf17a68df (/_internal_ping) 2020-03-31 09:13:58,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:13:58,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:13:58,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:13:58,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:58,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:58,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:58,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:13:58,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:13:58,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:58,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:58,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:13:58,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:13:58,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:13:58,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:13:58,327 [537] [DEBUG] [app] Starting request: urn:request:197f1394-7ac4-4f7e-8140-57cab6a5ef6b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:13:58,327 [537] [DEBUG] [app] Starting request: urn:request:197f1394-7ac4-4f7e-8140-57cab6a5ef6b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:58,328 [537] [DEBUG] [app] Ending request: urn:request:197f1394-7ac4-4f7e-8140-57cab6a5ef6b (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:13:58,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:58,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:13:58,328 [537] [DEBUG] [app] Ending request: urn:request:197f1394-7ac4-4f7e-8140-57cab6a5ef6b (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:13:58,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:13:58,331 [526] [DEBUG] [app] Ending request: urn:request:c67727ec-052c-49dc-bb4e-c9973207e810 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:13:58,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:13:58,331 [526] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:13:58,331 [526] [DEBUG] [app] Ending request: urn:request:c67727ec-052c-49dc-bb4e-c9973207e810 (/health/instance) 2020-03-31 09:13:58,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:13:58,331 [526] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:13:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:13:58,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:13:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:13:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:13:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:13:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:51.110610+00:00 (in 51.549437 seconds) 2020-03-31 09:13:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:59 UTC)" (scheduled at 2020-03-31 09:13:59.560326+00:00) 2020-03-31 09:13:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:13:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:51.110610+00:00 (in 51.549437 seconds) 2020-03-31 09:13:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:59 UTC)" (scheduled at 2020-03-31 09:13:59.560326+00:00) 2020-03-31 09:13:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:13:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:13:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:59 UTC)" executed successfully 2020-03-31 09:14:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:14:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:30.337699+00:00 (in 29.998973 seconds) 2020-03-31 09:14:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:30 UTC)" (scheduled at 2020-03-31 09:14:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:14:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:30.337699+00:00 (in 29.998973 seconds) 2020-03-31 09:14:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:30 UTC)" (scheduled at 2020-03-31 09:14:00.337699+00:00) 2020-03-31 09:14:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 14, 0, 339468), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:14:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 14, 0, 339468), False, 50, 1, 0]) 2020-03-31 09:14:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:14:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:14:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:14:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:30 UTC)" executed successfully 2020-03-31 09:14:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:14:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:11.440707+00:00 (in 9.998963 seconds) 2020-03-31 09:14:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:11 UTC)" (scheduled at 2020-03-31 09:14:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:14:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:11.440707+00:00 (in 9.998963 seconds) 2020-03-31 09:14:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:11 UTC)" (scheduled at 2020-03-31 09:14:01.440707+00:00) 2020-03-31 09:14:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:14:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:14:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:14:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:14:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:14:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:11 UTC)" executed successfully 2020-03-31 09:14:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:14:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:23.410371+00:00 (in 20.078978 seconds) 2020-03-31 09:14:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:33 UTC)" (scheduled at 2020-03-31 09:14:03.330112+00:00) 2020-03-31 09:14:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:14:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:23.410371+00:00 (in 20.078978 seconds) 2020-03-31 09:14:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:33 UTC)" (scheduled at 2020-03-31 09:14:03.330112+00:00) 2020-03-31 09:14:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 3, 332361), True, datetime.datetime(2020, 3, 31, 9, 14, 3, 332361), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:14:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 3, 332361), True, datetime.datetime(2020, 3, 31, 9, 14, 3, 332361), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:14:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:03,350 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:14:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:14:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:33 UTC)" executed successfully 2020-03-31 09:14:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:14:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:14.323352+00:00 (in 9.998648 seconds) 2020-03-31 09:14:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:14 UTC)" (scheduled at 2020-03-31 09:14:04.323352+00:00) 2020-03-31 09:14:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:14:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:14.323352+00:00 (in 9.998648 seconds) 2020-03-31 09:14:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:14 UTC)" (scheduled at 2020-03-31 09:14:04.323352+00:00) 2020-03-31 09:14:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 4, 325661), True, datetime.datetime(2020, 3, 31, 9, 14, 4, 325661), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:14:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 4, 325661), True, datetime.datetime(2020, 3, 31, 9, 14, 4, 325661), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:14:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:14:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:14:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:14 UTC)" executed successfully 2020-03-31 09:14:04,446 [518] [DEBUG] [app] Starting request: urn:request:9e37be01-9d37-4585-9ad6-ac2ef1e7066a (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:04,446 [518] [DEBUG] [app] Starting request: urn:request:9e37be01-9d37-4585-9ad6-ac2ef1e7066a (/health/instance) 2020-03-31 09:14:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:04,454 [546] [DEBUG] [app] Starting request: urn:request:d191dc32-29e7-4ede-92df-825f25e242d4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:04,454 [546] [DEBUG] [app] Starting request: urn:request:d191dc32-29e7-4ede-92df-825f25e242d4 (/v1/_internal_ping) 2020-03-31 09:14:04,455 [546] [DEBUG] [app] Ending request: urn:request:d191dc32-29e7-4ede-92df-825f25e242d4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:04,455 [546] [DEBUG] [app] Ending request: urn:request:d191dc32-29e7-4ede-92df-825f25e242d4 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:04,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:14:04,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:04,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:14:04,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:04,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:04,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:04,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:04,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:04,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:04,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:04,492 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:04,492 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:04,498 [518] [DEBUG] [app] Starting request: urn:request:afccdbf0-a5de-4186-b9ca-8fe01abeadd9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:04,498 [518] [DEBUG] [app] Starting request: urn:request:afccdbf0-a5de-4186-b9ca-8fe01abeadd9 (/_internal_ping) 2020-03-31 09:14:04,499 [518] [DEBUG] [app] Ending request: urn:request:afccdbf0-a5de-4186-b9ca-8fe01abeadd9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:04,499 [518] [DEBUG] [app] Ending request: urn:request:afccdbf0-a5de-4186-b9ca-8fe01abeadd9 (/_internal_ping) 2020-03-31 09:14:04,500 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:14:04,500 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:14:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:04,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:14:04,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:04,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:04,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:04,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:04,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:04,514 [543] [DEBUG] [app] Starting request: urn:request:abc596dc-73e9-4b91-b664-6b5577d9194e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:04,514 [543] [DEBUG] [app] Starting request: urn:request:abc596dc-73e9-4b91-b664-6b5577d9194e (/c1/_internal_ping) 2020-03-31 09:14:04,515 [543] [DEBUG] [app] Ending request: urn:request:abc596dc-73e9-4b91-b664-6b5577d9194e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:04,515 [543] [DEBUG] [app] Ending request: urn:request:abc596dc-73e9-4b91-b664-6b5577d9194e (/c1/_internal_ping) 2020-03-31 09:14:04,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:04,516 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:04,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:04,518 [518] [DEBUG] [app] Ending request: urn:request:9e37be01-9d37-4585-9ad6-ac2ef1e7066a (/health/instance) 2020-03-31 09:14:04,518 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:14:04,516 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:04,519 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:14:04,518 [518] [DEBUG] [app] Ending request: urn:request:9e37be01-9d37-4585-9ad6-ac2ef1e7066a (/health/instance) 2020-03-31 09:14:04,518 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:04,519 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:14:06,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:16.924497+00:00 (in 9.999343 seconds) 2020-03-31 09:14:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:16 UTC)" (scheduled at 2020-03-31 09:14:06.924497+00:00) 2020-03-31 09:14:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:14:06,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:14:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:16.924497+00:00 (in 9.999343 seconds) 2020-03-31 09:14:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:16 UTC)" (scheduled at 2020-03-31 09:14:06.924497+00:00) 2020-03-31 09:14:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 6, 925888), True, datetime.datetime(2020, 3, 31, 9, 14, 6, 925888), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:14:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 6, 925888), True, datetime.datetime(2020, 3, 31, 9, 14, 6, 925888), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:14:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:14:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:14:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:16 UTC)" executed successfully 2020-03-31 09:14:08,273 [518] [DEBUG] [app] Starting request: urn:request:952ead84-0c0a-4151-8d44-a9b79d16302c (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:08,273 [518] [DEBUG] [app] Starting request: urn:request:952ead84-0c0a-4151-8d44-a9b79d16302c (/health/instance) 2020-03-31 09:14:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:08,278 [555] [DEBUG] [app] Starting request: urn:request:880a4ad7-c03e-45a2-91f3-54b1d47d8495 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:08,278 [555] [DEBUG] [app] Starting request: urn:request:880a4ad7-c03e-45a2-91f3-54b1d47d8495 (/v1/_internal_ping) 2020-03-31 09:14:08,279 [555] [DEBUG] [app] Ending request: urn:request:880a4ad7-c03e-45a2-91f3-54b1d47d8495 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:08,279 [555] [DEBUG] [app] Ending request: urn:request:880a4ad7-c03e-45a2-91f3-54b1d47d8495 (/v1/_internal_ping) 2020-03-31 09:14:08,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:14:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:14:08,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:14:08,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:14:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:14:08,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:08,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:08,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:08,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:08,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:08,315 [527] [DEBUG] [app] Starting request: urn:request:ffef7dea-210b-4cdd-85a7-494ed286dd5d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:08,315 [527] [DEBUG] [app] Starting request: urn:request:ffef7dea-210b-4cdd-85a7-494ed286dd5d (/_internal_ping) 2020-03-31 09:14:08,316 [527] [DEBUG] [app] Ending request: urn:request:ffef7dea-210b-4cdd-85a7-494ed286dd5d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:08,316 [527] [DEBUG] [app] Ending request: urn:request:ffef7dea-210b-4cdd-85a7-494ed286dd5d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:14:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:14:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:14:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:08,330 [543] [DEBUG] [app] Starting request: urn:request:9f3641a8-af51-4de9-af2f-14bd6fbc8f0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:08,330 [543] [DEBUG] [app] Starting request: urn:request:9f3641a8-af51-4de9-af2f-14bd6fbc8f0b (/c1/_internal_ping) 2020-03-31 09:14:08,330 [543] [DEBUG] [app] Ending request: urn:request:9f3641a8-af51-4de9-af2f-14bd6fbc8f0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:08,330 [543] [DEBUG] [app] Ending request: urn:request:9f3641a8-af51-4de9-af2f-14bd6fbc8f0b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:08,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:14:08,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:08,333 [518] [DEBUG] [app] Ending request: urn:request:952ead84-0c0a-4151-8d44-a9b79d16302c (/health/instance) 2020-03-31 09:14:08,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:08,333 [518] [DEBUG] [app] Ending request: urn:request:952ead84-0c0a-4151-8d44-a9b79d16302c (/health/instance) 2020-03-31 09:14:08,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:14:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:14:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:14:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:21.440707+00:00 (in 9.998938 seconds) 2020-03-31 09:14:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:21 UTC)" (scheduled at 2020-03-31 09:14:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:14:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:21.440707+00:00 (in 9.998938 seconds) 2020-03-31 09:14:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:21 UTC)" (scheduled at 2020-03-31 09:14:11.440707+00:00) 2020-03-31 09:14:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:14:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:14:11,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:14:11,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:14:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:11,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:14:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:11,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:21 UTC)" executed successfully 2020-03-31 09:14:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:14:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:24.323352+00:00 (in 9.998077 seconds) 2020-03-31 09:14:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:24 UTC)" (scheduled at 2020-03-31 09:14:14.323352+00:00) 2020-03-31 09:14:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:14:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:24.323352+00:00 (in 9.998077 seconds) 2020-03-31 09:14:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:24 UTC)" (scheduled at 2020-03-31 09:14:14.323352+00:00) 2020-03-31 09:14:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 14, 326283), True, datetime.datetime(2020, 3, 31, 9, 14, 14, 326283), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:14:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 14, 326283), True, datetime.datetime(2020, 3, 31, 9, 14, 14, 326283), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:14:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:14:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:14:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:24 UTC)" executed successfully 2020-03-31 09:14:14,447 [527] [DEBUG] [app] Starting request: urn:request:84c11224-137a-46f2-9c95-1f892b378df1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:14,447 [527] [DEBUG] [app] Starting request: urn:request:84c11224-137a-46f2-9c95-1f892b378df1 (/health/instance) 2020-03-31 09:14:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:14,455 [538] [DEBUG] [app] Starting request: urn:request:a288e6c7-da84-462a-95d5-12f51fe7a984 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:14,455 [538] [DEBUG] [app] Starting request: urn:request:a288e6c7-da84-462a-95d5-12f51fe7a984 (/v1/_internal_ping) 2020-03-31 09:14:14,456 [538] [DEBUG] [app] Ending request: urn:request:a288e6c7-da84-462a-95d5-12f51fe7a984 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:14,456 [538] [DEBUG] [app] Ending request: urn:request:a288e6c7-da84-462a-95d5-12f51fe7a984 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:14:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:14,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:14:14,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:14,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:14,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:14,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:14,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:14,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:14,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:14,486 [518] [DEBUG] [app] Starting request: urn:request:aa8b838c-9252-41a7-9c74-81488a49a617 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:14,486 [518] [DEBUG] [app] Starting request: urn:request:aa8b838c-9252-41a7-9c74-81488a49a617 (/_internal_ping) 2020-03-31 09:14:14,487 [518] [DEBUG] [app] Ending request: urn:request:aa8b838c-9252-41a7-9c74-81488a49a617 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:14,487 [518] [DEBUG] [app] Ending request: urn:request:aa8b838c-9252-41a7-9c74-81488a49a617 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:14:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:14,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:14,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:14,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:14,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:14:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:14:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:14,502 [539] [DEBUG] [app] Starting request: urn:request:9fd75a06-c9f0-4c51-a5da-d12a126c7df3 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:14:14,502 [539] [DEBUG] [app] Starting request: urn:request:9fd75a06-c9f0-4c51-a5da-d12a126c7df3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:14:14,504 [539] [DEBUG] [app] Ending request: urn:request:9fd75a06-c9f0-4c51-a5da-d12a126c7df3 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:14:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:14,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:14:14,504 [539] [DEBUG] [app] Ending request: urn:request:9fd75a06-c9f0-4c51-a5da-d12a126c7df3 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:14:14,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:14,507 [527] [DEBUG] [app] Ending request: urn:request:84c11224-137a-46f2-9c95-1f892b378df1 (/health/instance) 2020-03-31 09:14:14,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:14,507 [527] [DEBUG] [app] Ending request: urn:request:84c11224-137a-46f2-9c95-1f892b378df1 (/health/instance) 2020-03-31 09:14:14,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:14,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:14:14,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:14:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:14:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:45.064514+00:00 (in 29.998960 seconds) 2020-03-31 09:14:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:45 UTC)" (scheduled at 2020-03-31 09:14:15.064514+00:00) gcworker stdout | 2020-03-31 09:14:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:45.064514+00:00 (in 29.998960 seconds) 2020-03-31 09:14:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:45 UTC)" (scheduled at 2020-03-31 09:14:15.064514+00:00) 2020-03-31 09:14:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436455, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:14:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436455, 1209600, 500, 1, 0]) 2020-03-31 09:14:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:14:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:14:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:14:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:45 UTC)" executed successfully 2020-03-31 09:14:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:14:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:26.924497+00:00 (in 9.999235 seconds) 2020-03-31 09:14:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:26 UTC)" (scheduled at 2020-03-31 09:14:16.924497+00:00) 2020-03-31 09:14:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:14:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:26.924497+00:00 (in 9.999235 seconds) 2020-03-31 09:14:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:26 UTC)" (scheduled at 2020-03-31 09:14:16.924497+00:00) 2020-03-31 09:14:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 16, 925929), True, datetime.datetime(2020, 3, 31, 9, 14, 16, 925929), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:14:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 16, 925929), True, datetime.datetime(2020, 3, 31, 9, 14, 16, 925929), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:14:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:14:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:14:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:26 UTC)" executed successfully 2020-03-31 09:14:18,273 [527] [DEBUG] [app] Starting request: urn:request:63ceb2ca-34b5-4201-847e-19f9afa49e51 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:18,273 [527] [DEBUG] [app] Starting request: urn:request:63ceb2ca-34b5-4201-847e-19f9afa49e51 (/health/instance) 2020-03-31 09:14:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:18,278 [551] [DEBUG] [app] Starting request: urn:request:5786db0c-f560-4401-a04f-f64140e80131 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:18,278 [551] [DEBUG] [app] Starting request: urn:request:5786db0c-f560-4401-a04f-f64140e80131 (/v1/_internal_ping) 2020-03-31 09:14:18,280 [551] [DEBUG] [app] Ending request: urn:request:5786db0c-f560-4401-a04f-f64140e80131 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:18,280 [551] [DEBUG] [app] Ending request: urn:request:5786db0c-f560-4401-a04f-f64140e80131 (/v1/_internal_ping) 2020-03-31 09:14:18,281 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:14:18,281 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:18,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:14:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:18,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:18,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:14:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:14:18,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:18,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:18,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:18,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:18,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:18,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:18,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:18,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:18,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:18,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:18,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:18,316 [527] [DEBUG] [app] Starting request: urn:request:90e5bdd6-b1eb-44b9-ad66-946dee306b47 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:18,316 [527] [DEBUG] [app] Starting request: urn:request:90e5bdd6-b1eb-44b9-ad66-946dee306b47 (/_internal_ping) 2020-03-31 09:14:18,317 [527] [DEBUG] [app] Ending request: urn:request:90e5bdd6-b1eb-44b9-ad66-946dee306b47 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:18,317 [527] [DEBUG] [app] Ending request: urn:request:90e5bdd6-b1eb-44b9-ad66-946dee306b47 (/_internal_ping) 2020-03-31 09:14:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:14:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:14:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:14:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:14:18,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:18,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:18,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:18,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:18,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:18,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:18,330 [537] [DEBUG] [app] Starting request: urn:request:a077c51e-7f3f-4aa9-8c22-6008edb560f2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:18,330 [537] [DEBUG] [app] Starting request: urn:request:a077c51e-7f3f-4aa9-8c22-6008edb560f2 (/c1/_internal_ping) 2020-03-31 09:14:18,331 [537] [DEBUG] [app] Ending request: urn:request:a077c51e-7f3f-4aa9-8c22-6008edb560f2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:18,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:14:18,331 [537] [DEBUG] [app] Ending request: urn:request:a077c51e-7f3f-4aa9-8c22-6008edb560f2 (/c1/_internal_ping) 2020-03-31 09:14:18,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:14:18,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:18,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:18,333 [527] [DEBUG] [app] Ending request: urn:request:63ceb2ca-34b5-4201-847e-19f9afa49e51 (/health/instance) 2020-03-31 09:14:18,334 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:18,334 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:14:18,333 [527] [DEBUG] [app] Ending request: urn:request:63ceb2ca-34b5-4201-847e-19f9afa49e51 (/health/instance) 2020-03-31 09:14:18,334 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:18,334 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:14:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:14:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:31.440707+00:00 (in 9.999111 seconds) 2020-03-31 09:14:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:31 UTC)" (scheduled at 2020-03-31 09:14:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:14:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:31.440707+00:00 (in 9.999111 seconds) 2020-03-31 09:14:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:31 UTC)" (scheduled at 2020-03-31 09:14:21.440707+00:00) 2020-03-31 09:14:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:14:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:14:21,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:14:21,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:14:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:14:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:31 UTC)" executed successfully 2020-03-31 09:14:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:14:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:52.113221+00:00 (in 29.999080 seconds) 2020-03-31 09:14:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:52 UTC)" (scheduled at 2020-03-31 09:14:22.113221+00:00) securityworker stdout | 2020-03-31 09:14:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:52.113221+00:00 (in 29.999080 seconds) 2020-03-31 09:14:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:52 UTC)" (scheduled at 2020-03-31 09:14:22.113221+00:00) 2020-03-31 09:14:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:14:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:14:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:14:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:14:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:14:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:14:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:14:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:14:52 UTC)" executed successfully 2020-03-31 09:14:23,413 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:14:23,413 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:23,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:24.410927+00:00 (in 0.997505 seconds) 2020-03-31 09:14:23,414 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:23 UTC)" (scheduled at 2020-03-31 09:14:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:14:23,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:24.410927+00:00 (in 0.997505 seconds) 2020-03-31 09:14:23,414 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:23 UTC)" (scheduled at 2020-03-31 09:14:23.410371+00:00) 2020-03-31 09:14:23,415 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:14:23,415 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), u'security_notification/%']) 2020-03-31 09:14:23,429 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), True, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:14:23,429 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), True, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), 0, u'security_notification/%']) 2020-03-31 09:14:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), True, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:14:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), True, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 14, 23, 414526), u'security_notification/%']) 2020-03-31 09:14:23,437 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:14:23,437 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:23,438 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:14:23,438 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:23 UTC)" executed successfully 2020-03-31 09:14:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:14:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:14:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:14:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:14:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:34.323352+00:00 (in 9.999221 seconds) 2020-03-31 09:14:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:34 UTC)" (scheduled at 2020-03-31 09:14:24.323352+00:00) 2020-03-31 09:14:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:14:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:14:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:34.323352+00:00 (in 9.999221 seconds) 2020-03-31 09:14:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:34 UTC)" (scheduled at 2020-03-31 09:14:24.323352+00:00) 2020-03-31 09:14:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 24, 324858), True, datetime.datetime(2020, 3, 31, 9, 14, 24, 324858), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:14:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 24, 324858), True, datetime.datetime(2020, 3, 31, 9, 14, 24, 324858), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:14:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:14:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:34 UTC)" executed successfully 2020-03-31 09:14:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:14:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:33.330112+00:00 (in 8.918468 seconds) 2020-03-31 09:14:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:24 UTC)" (scheduled at 2020-03-31 09:14:24.410927+00:00) 2020-03-31 09:14:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:14:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:33.330112+00:00 (in 8.918468 seconds) 2020-03-31 09:14:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:24 UTC)" (scheduled at 2020-03-31 09:14:24.410927+00:00) 2020-03-31 09:14:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:24 UTC)" executed successfully 2020-03-31 09:14:24,446 [518] [DEBUG] [app] Starting request: urn:request:aa5c55c2-aa1d-4c81-843a-fa121d875466 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:24,446 [518] [DEBUG] [app] Starting request: urn:request:aa5c55c2-aa1d-4c81-843a-fa121d875466 (/health/instance) 2020-03-31 09:14:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:24,456 [556] [DEBUG] [app] Starting request: urn:request:d1ac8932-4f8c-447b-9b9d-7f8184ed88e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:24,456 [556] [DEBUG] [app] Starting request: urn:request:d1ac8932-4f8c-447b-9b9d-7f8184ed88e9 (/v1/_internal_ping) 2020-03-31 09:14:24,457 [556] [DEBUG] [app] Ending request: urn:request:d1ac8932-4f8c-447b-9b9d-7f8184ed88e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:24,457 [556] [DEBUG] [app] Ending request: urn:request:d1ac8932-4f8c-447b-9b9d-7f8184ed88e9 (/v1/_internal_ping) 2020-03-31 09:14:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:14:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:14:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:14:24,461 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:14:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:24,462 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:24,461 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:24,464 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:24,462 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:24,464 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:24,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:24,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:24,500 [518] [DEBUG] [app] Starting request: urn:request:e79ed5df-9710-4217-b894-ecf312500422 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:24,500 [518] [DEBUG] [app] Starting request: urn:request:e79ed5df-9710-4217-b894-ecf312500422 (/_internal_ping) 2020-03-31 09:14:24,501 [518] [DEBUG] [app] Ending request: urn:request:e79ed5df-9710-4217-b894-ecf312500422 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:14:24,501 [518] [DEBUG] [app] Ending request: urn:request:e79ed5df-9710-4217-b894-ecf312500422 (/_internal_ping) 2020-03-31 09:14:24,502 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:14:24,502 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:24,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:24,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:14:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:14:24,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:24,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:14:24,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:24,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:24,515 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:24,515 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:24,517 [537] [DEBUG] [app] Starting request: urn:request:2b003a81-5315-4df2-9ec3-666d775a9e7a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:24,517 [537] [DEBUG] [app] Starting request: urn:request:2b003a81-5315-4df2-9ec3-666d775a9e7a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:24,518 [537] [DEBUG] [app] Ending request: urn:request:2b003a81-5315-4df2-9ec3-666d775a9e7a (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:24,519 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:14:24,518 [537] [DEBUG] [app] Ending request: urn:request:2b003a81-5315-4df2-9ec3-666d775a9e7a (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:24,519 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:24,521 [518] [DEBUG] [app] Ending request: urn:request:aa5c55c2-aa1d-4c81-843a-fa121d875466 (/health/instance) 2020-03-31 09:14:24,521 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:24,521 [518] [DEBUG] [app] Ending request: urn:request:aa5c55c2-aa1d-4c81-843a-fa121d875466 (/health/instance) 2020-03-31 09:14:24,521 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) 2020-03-31 09:14:24,522 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:14:24,522 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:24,522 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:14:24,522 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:36.924497+00:00 (in 9.998051 seconds) 2020-03-31 09:14:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:36 UTC)" (scheduled at 2020-03-31 09:14:26.924497+00:00) 2020-03-31 09:14:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 26, 927296), True, datetime.datetime(2020, 3, 31, 9, 14, 26, 927296), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:14:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:36.924497+00:00 (in 9.998051 seconds) storagereplication stdout | 2020-03-31 09:14:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:36 UTC)" (scheduled at 2020-03-31 09:14:26.924497+00:00) 2020-03-31 09:14:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:14:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 26, 927296), True, datetime.datetime(2020, 3, 31, 9, 14, 26, 927296), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:14:26,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:14:26,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:26,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:26,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:14:26,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:26,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:36 UTC)" executed successfully 2020-03-31 09:14:28,272 [527] [DEBUG] [app] Starting request: urn:request:319c92e3-d8c9-4004-a6ba-1d573ef1d419 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:28,272 [527] [DEBUG] [app] Starting request: urn:request:319c92e3-d8c9-4004-a6ba-1d573ef1d419 (/health/instance) 2020-03-31 09:14:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:28,280 [555] [DEBUG] [app] Starting request: urn:request:e2564d48-25e6-4e8c-80fa-012a006e5016 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:28,280 [555] [DEBUG] [app] Starting request: urn:request:e2564d48-25e6-4e8c-80fa-012a006e5016 (/v1/_internal_ping) 2020-03-31 09:14:28,282 [555] [DEBUG] [app] Ending request: urn:request:e2564d48-25e6-4e8c-80fa-012a006e5016 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:28,282 [555] [DEBUG] [app] Ending request: urn:request:e2564d48-25e6-4e8c-80fa-012a006e5016 (/v1/_internal_ping) 2020-03-31 09:14:28,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-registry stdout | 2020-03-31 09:14:28,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:14:28,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:14:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:14:28,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:28,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:28,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:28,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:28,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:28,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:28,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:28,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:28,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:28,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:28,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:28,310 [527] [DEBUG] [app] Starting request: urn:request:3a993ba6-c644-4b06-b77f-847d7e0fed99 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:28,310 [527] [DEBUG] [app] Starting request: urn:request:3a993ba6-c644-4b06-b77f-847d7e0fed99 (/_internal_ping) 2020-03-31 09:14:28,311 [527] [DEBUG] [app] Ending request: urn:request:3a993ba6-c644-4b06-b77f-847d7e0fed99 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:28,311 [527] [DEBUG] [app] Ending request: urn:request:3a993ba6-c644-4b06-b77f-847d7e0fed99 (/_internal_ping) 2020-03-31 09:14:28,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:14:28,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:28,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:14:28,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:28,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:14:28,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:28,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:28,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:28,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:28,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:28,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:28,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:28,325 [543] [DEBUG] [app] Starting request: urn:request:85de2c2f-bfcc-46d3-b3a1-ee3172783624 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:28,325 [543] [DEBUG] [app] Starting request: urn:request:85de2c2f-bfcc-46d3-b3a1-ee3172783624 (/c1/_internal_ping) 2020-03-31 09:14:28,326 [543] [DEBUG] [app] Ending request: urn:request:85de2c2f-bfcc-46d3-b3a1-ee3172783624 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:28,326 [543] [DEBUG] [app] Ending request: urn:request:85de2c2f-bfcc-46d3-b3a1-ee3172783624 (/c1/_internal_ping) 2020-03-31 09:14:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:28,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:28,328 [527] [DEBUG] [app] Ending request: urn:request:319c92e3-d8c9-4004-a6ba-1d573ef1d419 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:14:28,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:28,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:28,328 [527] [DEBUG] [app] Ending request: urn:request:319c92e3-d8c9-4004-a6ba-1d573ef1d419 (/health/instance) 2020-03-31 09:14:28,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:14:28,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:28,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:14:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:14:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:00 UTC)" (scheduled at 2020-03-31 09:14:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:14:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:00 UTC)" (scheduled at 2020-03-31 09:14:30.337699+00:00) 2020-03-31 09:14:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 14, 30, 339510), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:14:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 14, 30, 339510), False, 50, 1, 0]) 2020-03-31 09:14:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:00.337699+00:00 (in 29.999142 seconds) buildlogsarchiver stdout | 2020-03-31 09:14:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:00.337699+00:00 (in 29.999142 seconds) 2020-03-31 09:14:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:14:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:14:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:14:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:00 UTC)" executed successfully 2020-03-31 09:14:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:14:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:41.440707+00:00 (in 9.999149 seconds) 2020-03-31 09:14:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:41 UTC)" (scheduled at 2020-03-31 09:14:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:14:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:41.440707+00:00 (in 9.999149 seconds) 2020-03-31 09:14:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:41 UTC)" (scheduled at 2020-03-31 09:14:31.440707+00:00) 2020-03-31 09:14:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:14:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:14:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:14:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:14:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:14:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:41 UTC)" executed successfully 2020-03-31 09:14:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:14:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:03.330112+00:00 (in 29.999211 seconds) 2020-03-31 09:14:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:03 UTC)" (scheduled at 2020-03-31 09:14:33.330112+00:00) 2020-03-31 09:14:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:14:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:03.330112+00:00 (in 29.999211 seconds) 2020-03-31 09:14:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:03 UTC)" (scheduled at 2020-03-31 09:14:33.330112+00:00) 2020-03-31 09:14:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 33, 331928), True, datetime.datetime(2020, 3, 31, 9, 14, 33, 331928), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:14:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 33, 331928), True, datetime.datetime(2020, 3, 31, 9, 14, 33, 331928), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:14:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:14:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:14:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:03 UTC)" executed successfully 2020-03-31 09:14:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:14:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:43.539999+00:00 (in 10.171825 seconds) 2020-03-31 09:14:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:33 UTC)" (scheduled at 2020-03-31 09:14:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:14:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:43.539999+00:00 (in 10.171825 seconds) 2020-03-31 09:14:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:33 UTC)" (scheduled at 2020-03-31 09:14:33.366791+00:00) 2020-03-31 09:14:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:14:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 33, 369036), True, datetime.datetime(2020, 3, 31, 9, 14, 33, 369036), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:14:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 33, 369036), True, datetime.datetime(2020, 3, 31, 9, 14, 33, 369036), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:14:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:14:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:14:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:33 UTC)" executed successfully 2020-03-31 09:14:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:14:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:44.323352+00:00 (in 9.999202 seconds) 2020-03-31 09:14:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:44 UTC)" (scheduled at 2020-03-31 09:14:34.323352+00:00) 2020-03-31 09:14:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:14:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:44.323352+00:00 (in 9.999202 seconds) 2020-03-31 09:14:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:44 UTC)" (scheduled at 2020-03-31 09:14:34.323352+00:00) 2020-03-31 09:14:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 34, 325276), True, datetime.datetime(2020, 3, 31, 9, 14, 34, 325276), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:14:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 34, 325276), True, datetime.datetime(2020, 3, 31, 9, 14, 34, 325276), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:14:34,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:14:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:14:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:44 UTC)" executed successfully 2020-03-31 09:14:34,446 [527] [DEBUG] [app] Starting request: urn:request:99e63526-b346-43ea-bd92-b3639498c561 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:34,446 [527] [DEBUG] [app] Starting request: urn:request:99e63526-b346-43ea-bd92-b3639498c561 (/health/instance) 2020-03-31 09:14:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:34,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:34,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:34,453 [548] [DEBUG] [app] Starting request: urn:request:ec35cfc3-7496-4f91-83e2-995143ebf60c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:34,453 [548] [DEBUG] [app] Starting request: urn:request:ec35cfc3-7496-4f91-83e2-995143ebf60c (/v1/_internal_ping) 2020-03-31 09:14:34,454 [548] [DEBUG] [app] Ending request: urn:request:ec35cfc3-7496-4f91-83e2-995143ebf60c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:34,454 [548] [DEBUG] [app] Ending request: urn:request:ec35cfc3-7496-4f91-83e2-995143ebf60c (/v1/_internal_ping) 2020-03-31 09:14:34,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:14:34,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:34,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:34,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:34,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:34,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:34,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:14:34,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:34,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:34,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:34,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:34,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:34,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:34,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:34,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:34,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:34,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:34,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:34,487 [527] [DEBUG] [app] Starting request: urn:request:73d6045d-dff7-43b0-963c-5b91d2f83399 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:34,487 [527] [DEBUG] [app] Starting request: urn:request:73d6045d-dff7-43b0-963c-5b91d2f83399 (/_internal_ping) 2020-03-31 09:14:34,488 [527] [DEBUG] [app] Ending request: urn:request:73d6045d-dff7-43b0-963c-5b91d2f83399 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:34,488 [527] [DEBUG] [app] Ending request: urn:request:73d6045d-dff7-43b0-963c-5b91d2f83399 (/_internal_ping) 2020-03-31 09:14:34,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:14:34,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:34,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:14:34,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:34,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:34,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:14:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:14:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:34,501 [535] [DEBUG] [app] Starting request: urn:request:3c967dc7-c3f7-4108-919f-62b105ac7cdf (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:34,501 [535] [DEBUG] [app] Starting request: urn:request:3c967dc7-c3f7-4108-919f-62b105ac7cdf (/c1/_internal_ping) 2020-03-31 09:14:34,501 [535] [DEBUG] [app] Ending request: urn:request:3c967dc7-c3f7-4108-919f-62b105ac7cdf (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:34,501 [535] [DEBUG] [app] Ending request: urn:request:3c967dc7-c3f7-4108-919f-62b105ac7cdf (/c1/_internal_ping) 2020-03-31 09:14:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:34,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:34,503 [527] [DEBUG] [app] Ending request: urn:request:99e63526-b346-43ea-bd92-b3639498c561 (/health/instance) 2020-03-31 09:14:34,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:14:34,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:14:34,503 [527] [DEBUG] [app] Ending request: urn:request:99e63526-b346-43ea-bd92-b3639498c561 (/health/instance) 2020-03-31 09:14:34,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:14:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:38.010934+00:00 (in 1.085597 seconds) storagereplication stdout | 2020-03-31 09:14:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:38.010934+00:00 (in 1.085597 seconds) 2020-03-31 09:14:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:46 UTC)" (scheduled at 2020-03-31 09:14:36.924497+00:00) 2020-03-31 09:14:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:14:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:46 UTC)" (scheduled at 2020-03-31 09:14:36.924497+00:00) 2020-03-31 09:14:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 36, 926121), True, datetime.datetime(2020, 3, 31, 9, 14, 36, 926121), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:14:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 36, 926121), True, datetime.datetime(2020, 3, 31, 9, 14, 36, 926121), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:14:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:14:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:14:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:46 UTC)" executed successfully 2020-03-31 09:14:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:14:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:46.924497+00:00 (in 8.912819 seconds) 2020-03-31 09:14:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:38 UTC)" (scheduled at 2020-03-31 09:14:38.010934+00:00) 2020-03-31 09:14:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:14:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:46.924497+00:00 (in 8.912819 seconds) 2020-03-31 09:14:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:38 UTC)" (scheduled at 2020-03-31 09:14:38.010934+00:00) 2020-03-31 09:14:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:38 UTC)" executed successfully 2020-03-31 09:14:38,272 [526] [DEBUG] [app] Starting request: urn:request:36d67e91-39d5-45e4-b6a2-0426f4a6cbf2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:38,272 [526] [DEBUG] [app] Starting request: urn:request:36d67e91-39d5-45e4-b6a2-0426f4a6cbf2 (/health/instance) 2020-03-31 09:14:38,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:38,274 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:38,279 [553] [DEBUG] [app] Starting request: urn:request:d74e76c6-f12a-45c1-b4e3-36ee6cd626ec (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:38,279 [553] [DEBUG] [app] Starting request: urn:request:d74e76c6-f12a-45c1-b4e3-36ee6cd626ec (/v1/_internal_ping) 2020-03-31 09:14:38,280 [553] [DEBUG] [app] Ending request: urn:request:d74e76c6-f12a-45c1-b4e3-36ee6cd626ec (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:38,280 [553] [DEBUG] [app] Ending request: urn:request:d74e76c6-f12a-45c1-b4e3-36ee6cd626ec (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:38,281 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:38,281 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:14:38,281 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:38,282 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:38,281 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:38,282 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:38,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:14:38,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:38,285 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:38,285 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:38,286 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:38,286 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:38,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:38,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:38,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:38,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:38,309 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:38,309 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:38,311 [525] [DEBUG] [app] Starting request: urn:request:bd6d659a-c3e5-4e87-af68-fcefb59eb3fc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:38,311 [525] [DEBUG] [app] Starting request: urn:request:bd6d659a-c3e5-4e87-af68-fcefb59eb3fc (/_internal_ping) 2020-03-31 09:14:38,312 [525] [DEBUG] [app] Ending request: urn:request:bd6d659a-c3e5-4e87-af68-fcefb59eb3fc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:38,312 [525] [DEBUG] [app] Ending request: urn:request:bd6d659a-c3e5-4e87-af68-fcefb59eb3fc (/_internal_ping) 2020-03-31 09:14:38,313 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:38,313 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:14:38,313 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:38,313 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:14:38,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:38,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:14:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:14:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:38,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:38,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:38,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:38,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:38,325 [535] [DEBUG] [app] Starting request: urn:request:4841ef15-4760-49ce-a93c-b2a7addd8a93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:38,325 [535] [DEBUG] [app] Starting request: urn:request:4841ef15-4760-49ce-a93c-b2a7addd8a93 (/c1/_internal_ping) 2020-03-31 09:14:38,326 [535] [DEBUG] [app] Ending request: urn:request:4841ef15-4760-49ce-a93c-b2a7addd8a93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:38,326 [535] [DEBUG] [app] Ending request: urn:request:4841ef15-4760-49ce-a93c-b2a7addd8a93 (/c1/_internal_ping) 2020-03-31 09:14:38,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:38,326 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:38,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:14:38,326 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:38,328 [526] [DEBUG] [app] Ending request: urn:request:36d67e91-39d5-45e4-b6a2-0426f4a6cbf2 (/health/instance) 2020-03-31 09:14:38,328 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:38,328 [526] [DEBUG] [app] Ending request: urn:request:36d67e91-39d5-45e4-b6a2-0426f4a6cbf2 (/health/instance) 2020-03-31 09:14:38,328 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:38,329 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:14:38,329 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:14:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:14:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:51 UTC)" (scheduled at 2020-03-31 09:14:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:14:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:51 UTC)" (scheduled at 2020-03-31 09:14:41.440707+00:00) 2020-03-31 09:14:41,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:51.440707+00:00 (in 9.998842 seconds) repositoryactioncounter stdout | 2020-03-31 09:14:41,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:51.440707+00:00 (in 9.998842 seconds) 2020-03-31 09:14:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:14:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:14:41,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:14:41,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:14:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:14:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:51 UTC)" executed successfully 2020-03-31 09:14:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:14:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:43 UTC)" (scheduled at 2020-03-31 09:14:43.539999+00:00) 2020-03-31 09:14:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:14.539408+00:00 (in 30.998494 seconds) 2020-03-31 09:14:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:14:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:14:43 UTC)" (scheduled at 2020-03-31 09:14:43.539999+00:00) 2020-03-31 09:14:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:14.539408+00:00 (in 30.998494 seconds) 2020-03-31 09:14:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:14:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:43 UTC)" executed successfully 2020-03-31 09:14:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:14:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:54 UTC)" (scheduled at 2020-03-31 09:14:44.323352+00:00) 2020-03-31 09:14:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:49.498966+00:00 (in 5.174383 seconds) 2020-03-31 09:14:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:14:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:54 UTC)" (scheduled at 2020-03-31 09:14:44.323352+00:00) 2020-03-31 09:14:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:49.498966+00:00 (in 5.174383 seconds) 2020-03-31 09:14:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 44, 325605), True, datetime.datetime(2020, 3, 31, 9, 14, 44, 325605), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:14:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 44, 325605), True, datetime.datetime(2020, 3, 31, 9, 14, 44, 325605), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:14:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:14:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:14:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:54 UTC)" executed successfully 2020-03-31 09:14:44,445 [527] [DEBUG] [app] Starting request: urn:request:947246a0-d2e3-4f22-b853-7a3686f05cd3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:44,445 [527] [DEBUG] [app] Starting request: urn:request:947246a0-d2e3-4f22-b853-7a3686f05cd3 (/health/instance) 2020-03-31 09:14:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:44,454 [554] [DEBUG] [app] Starting request: urn:request:e1c419d0-14f2-4a1f-840d-517d64619c38 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:44,454 [554] [DEBUG] [app] Starting request: urn:request:e1c419d0-14f2-4a1f-840d-517d64619c38 (/v1/_internal_ping) 2020-03-31 09:14:44,455 [554] [DEBUG] [app] Ending request: urn:request:e1c419d0-14f2-4a1f-840d-517d64619c38 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:44,455 [554] [DEBUG] [app] Ending request: urn:request:e1c419d0-14f2-4a1f-840d-517d64619c38 (/v1/_internal_ping) 2020-03-31 09:14:44,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:14:44,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:44,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:14:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:44,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:44,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:44,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:44,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:44,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:44,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:44,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:44,488 [518] [DEBUG] [app] Starting request: urn:request:fde39cef-b9fd-4c49-a717-54489907cef4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:44,488 [518] [DEBUG] [app] Starting request: urn:request:fde39cef-b9fd-4c49-a717-54489907cef4 (/_internal_ping) 2020-03-31 09:14:44,489 [518] [DEBUG] [app] Ending request: urn:request:fde39cef-b9fd-4c49-a717-54489907cef4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:44,489 [518] [DEBUG] [app] Ending request: urn:request:fde39cef-b9fd-4c49-a717-54489907cef4 (/_internal_ping) 2020-03-31 09:14:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:14:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:14:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:14:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:14:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:14:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:44,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:44,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:44,502 [541] [DEBUG] [app] Starting request: urn:request:60345a58-942b-43a6-a12b-c1de332eee95 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:44,502 [541] [DEBUG] [app] Starting request: urn:request:60345a58-942b-43a6-a12b-c1de332eee95 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:44,503 [541] [DEBUG] [app] Ending request: urn:request:60345a58-942b-43a6-a12b-c1de332eee95 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:44,503 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:14:44,503 [541] [DEBUG] [app] Ending request: urn:request:60345a58-942b-43a6-a12b-c1de332eee95 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:44,505 [527] [DEBUG] [app] Ending request: urn:request:947246a0-d2e3-4f22-b853-7a3686f05cd3 (/health/instance) 2020-03-31 09:14:44,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:14:44,503 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:14:44,505 [527] [DEBUG] [app] Ending request: urn:request:947246a0-d2e3-4f22-b853-7a3686f05cd3 (/health/instance) 2020-03-31 09:14:44,506 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:44,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:14:44,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:14:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:14:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:15.064514+00:00 (in 29.999151 seconds) 2020-03-31 09:14:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:15 UTC)" (scheduled at 2020-03-31 09:14:45.064514+00:00) gcworker stdout | 2020-03-31 09:14:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:15.064514+00:00 (in 29.999151 seconds) 2020-03-31 09:14:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:15 UTC)" (scheduled at 2020-03-31 09:14:45.064514+00:00) 2020-03-31 09:14:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436485, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:14:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436485, 1209600, 500, 1, 0]) 2020-03-31 09:14:45,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:14:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:14:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:14:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:15 UTC)" executed successfully 2020-03-31 09:14:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:14:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:56.924497+00:00 (in 9.999173 seconds) 2020-03-31 09:14:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:56 UTC)" (scheduled at 2020-03-31 09:14:46.924497+00:00) 2020-03-31 09:14:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:14:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:56.924497+00:00 (in 9.999173 seconds) 2020-03-31 09:14:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:56 UTC)" (scheduled at 2020-03-31 09:14:46.924497+00:00) 2020-03-31 09:14:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 46, 926180), True, datetime.datetime(2020, 3, 31, 9, 14, 46, 926180), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:14:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 46, 926180), True, datetime.datetime(2020, 3, 31, 9, 14, 46, 926180), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:14:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:14:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:14:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:14:56 UTC)" executed successfully 2020-03-31 09:14:48,273 [518] [DEBUG] [app] Starting request: urn:request:a3fb0a7e-1436-4b99-a23f-000283ae5c03 (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:48,273 [518] [DEBUG] [app] Starting request: urn:request:a3fb0a7e-1436-4b99-a23f-000283ae5c03 (/health/instance) 2020-03-31 09:14:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:48,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:48,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:48,280 [548] [DEBUG] [app] Starting request: urn:request:3148c568-b3ee-46d7-8a5a-f0965d4ec028 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:48,280 [548] [DEBUG] [app] Starting request: urn:request:3148c568-b3ee-46d7-8a5a-f0965d4ec028 (/v1/_internal_ping) 2020-03-31 09:14:48,281 [548] [DEBUG] [app] Ending request: urn:request:3148c568-b3ee-46d7-8a5a-f0965d4ec028 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:48,281 [548] [DEBUG] [app] Ending request: urn:request:3148c568-b3ee-46d7-8a5a-f0965d4ec028 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:48,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:48,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:14:48,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:48,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:48,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:48,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:48,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:14:48,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:48,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:48,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:48,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:48,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:48,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:48,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:48,313 [518] [DEBUG] [app] Starting request: urn:request:cb449513-c059-4ec8-9713-ed0de2e274bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:48,313 [518] [DEBUG] [app] Starting request: urn:request:cb449513-c059-4ec8-9713-ed0de2e274bc (/_internal_ping) 2020-03-31 09:14:48,313 [518] [DEBUG] [app] Ending request: urn:request:cb449513-c059-4ec8-9713-ed0de2e274bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:48,313 [518] [DEBUG] [app] Ending request: urn:request:cb449513-c059-4ec8-9713-ed0de2e274bc (/_internal_ping) 2020-03-31 09:14:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:48,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:14:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:48,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:14:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:14:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:48,326 [541] [DEBUG] [app] Starting request: urn:request:8244618d-f645-45a2-bf70-0f655111b42f (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:14:48,326 [541] [DEBUG] [app] Starting request: urn:request:8244618d-f645-45a2-bf70-0f655111b42f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:48,327 [541] [DEBUG] [app] Ending request: urn:request:8244618d-f645-45a2-bf70-0f655111b42f (/c1/_internal_ping) 2020-03-31 09:14:48,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:14:48,327 [541] [DEBUG] [app] Ending request: urn:request:8244618d-f645-45a2-bf70-0f655111b42f (/c1/_internal_ping) 2020-03-31 09:14:48,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:48,329 [518] [DEBUG] [app] Ending request: urn:request:a3fb0a7e-1436-4b99-a23f-000283ae5c03 (/health/instance) 2020-03-31 09:14:48,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:48,329 [518] [DEBUG] [app] Ending request: urn:request:a3fb0a7e-1436-4b99-a23f-000283ae5c03 (/health/instance) 2020-03-31 09:14:48,329 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:14:48,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:14:48,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:14:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:14:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:54.323352+00:00 (in 4.823039 seconds) 2020-03-31 09:14:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:49 UTC)" (scheduled at 2020-03-31 09:14:49.498966+00:00) 2020-03-31 09:14:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:14:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:54.323352+00:00 (in 4.823039 seconds) 2020-03-31 09:14:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:49 UTC)" (scheduled at 2020-03-31 09:14:49.498966+00:00) 2020-03-31 09:14:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:49 UTC)" executed successfully 2020-03-31 09:14:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:14:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:51 UTC)" (scheduled at 2020-03-31 09:14:51.110610+00:00) 2020-03-31 09:14:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:53.556282+00:00 (in 2.444404 seconds) namespacegcworker stdout | 2020-03-31 09:14:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:51 UTC)" (scheduled at 2020-03-31 09:14:51.110610+00:00) 2020-03-31 09:14:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:53.556282+00:00 (in 2.444404 seconds) 2020-03-31 09:14:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 51, 112797), True, datetime.datetime(2020, 3, 31, 9, 14, 51, 112797), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:14:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 51, 112797), True, datetime.datetime(2020, 3, 31, 9, 14, 51, 112797), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:14:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:14:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:14:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:51 UTC)" executed successfully 2020-03-31 09:14:51,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:14:51,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:01.440707+00:00 (in 9.996722 seconds) 2020-03-31 09:14:51,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:01 UTC)" (scheduled at 2020-03-31 09:14:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:14:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:01.440707+00:00 (in 9.996722 seconds) 2020-03-31 09:14:51,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:01 UTC)" (scheduled at 2020-03-31 09:14:51.440707+00:00) 2020-03-31 09:14:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:14:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:14:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:14:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:14:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:14:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:01 UTC)" executed successfully 2020-03-31 09:14:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:14:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:22.113221+00:00 (in 29.999248 seconds) 2020-03-31 09:14:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:22 UTC)" (scheduled at 2020-03-31 09:14:52.113221+00:00) securityworker stdout | 2020-03-31 09:14:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:22.113221+00:00 (in 29.999248 seconds) 2020-03-31 09:14:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:22 UTC)" (scheduled at 2020-03-31 09:14:52.113221+00:00) 2020-03-31 09:14:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:14:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:14:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:14:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:14:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:14:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:14:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:14:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:22 UTC)" executed successfully 2020-03-31 09:14:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:14:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:59.560326+00:00 (in 6.002885 seconds) 2020-03-31 09:14:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:53 UTC)" (scheduled at 2020-03-31 09:14:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:14:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:14:59.560326+00:00 (in 6.002885 seconds) 2020-03-31 09:14:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:53 UTC)" (scheduled at 2020-03-31 09:14:53.556282+00:00) 2020-03-31 09:14:53,560 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:14:53,560 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), u'namespacegc/%']) 2020-03-31 09:14:53,575 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), True, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:14:53,575 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), True, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), 0, u'namespacegc/%']) 2020-03-31 09:14:53,579 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), True, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:14:53,579 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), True, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 14, 53, 558300), u'namespacegc/%']) 2020-03-31 09:14:53,583 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:14:53,583 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:53,585 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:14:53,585 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:53,585 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:53 UTC)" executed successfully 2020-03-31 09:14:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:14:53,585 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:19:53 UTC)" executed successfully 2020-03-31 09:14:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:53,587 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:14:53,587 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:14:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:14:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:04.323352+00:00 (in 9.998697 seconds) 2020-03-31 09:14:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:04 UTC)" (scheduled at 2020-03-31 09:14:54.323352+00:00) 2020-03-31 09:14:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:14:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:04.323352+00:00 (in 9.998697 seconds) 2020-03-31 09:14:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:04 UTC)" (scheduled at 2020-03-31 09:14:54.323352+00:00) 2020-03-31 09:14:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 54, 325579), True, datetime.datetime(2020, 3, 31, 9, 14, 54, 325579), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:14:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 54, 325579), True, datetime.datetime(2020, 3, 31, 9, 14, 54, 325579), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:14:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:14:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:14:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:04 UTC)" executed successfully 2020-03-31 09:14:54,446 [518] [DEBUG] [app] Starting request: urn:request:6a418995-9835-49ac-a985-703200850ada (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:54,446 [518] [DEBUG] [app] Starting request: urn:request:6a418995-9835-49ac-a985-703200850ada (/health/instance) 2020-03-31 09:14:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:54,453 [538] [DEBUG] [app] Starting request: urn:request:b197cec6-5c5c-46a8-8f05-8f48a7709f0d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:54,453 [538] [DEBUG] [app] Starting request: urn:request:b197cec6-5c5c-46a8-8f05-8f48a7709f0d (/v1/_internal_ping) 2020-03-31 09:14:54,454 [538] [DEBUG] [app] Ending request: urn:request:b197cec6-5c5c-46a8-8f05-8f48a7709f0d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:54,454 [538] [DEBUG] [app] Ending request: urn:request:b197cec6-5c5c-46a8-8f05-8f48a7709f0d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:14:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:14:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:54,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:14:54,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:54,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:54,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:54,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:54,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:54,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:54,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:54,486 [518] [DEBUG] [app] Starting request: urn:request:e562ee60-e001-4472-8b90-d009ef20e585 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:54,486 [518] [DEBUG] [app] Starting request: urn:request:e562ee60-e001-4472-8b90-d009ef20e585 (/_internal_ping) 2020-03-31 09:14:54,487 [518] [DEBUG] [app] Ending request: urn:request:e562ee60-e001-4472-8b90-d009ef20e585 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:54,487 [518] [DEBUG] [app] Ending request: urn:request:e562ee60-e001-4472-8b90-d009ef20e585 (/_internal_ping) 2020-03-31 09:14:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:14:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:14:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:14:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:14:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:54,500 [535] [DEBUG] [app] Starting request: urn:request:4b66a9ac-54a2-4402-9142-618d557ab165 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:54,500 [535] [DEBUG] [app] Starting request: urn:request:4b66a9ac-54a2-4402-9142-618d557ab165 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:54,501 [535] [DEBUG] [app] Ending request: urn:request:4b66a9ac-54a2-4402-9142-618d557ab165 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:14:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:54,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:14:54,501 [535] [DEBUG] [app] Ending request: urn:request:4b66a9ac-54a2-4402-9142-618d557ab165 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:54,503 [518] [DEBUG] [app] Ending request: urn:request:6a418995-9835-49ac-a985-703200850ada (/health/instance) 2020-03-31 09:14:54,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:14:54,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:54,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:14:54,503 [518] [DEBUG] [app] Ending request: urn:request:6a418995-9835-49ac-a985-703200850ada (/health/instance) 2020-03-31 09:14:54,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:54,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:14:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:14:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:06.924497+00:00 (in 9.999258 seconds) 2020-03-31 09:14:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:06 UTC)" (scheduled at 2020-03-31 09:14:56.924497+00:00) 2020-03-31 09:14:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:14:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:06.924497+00:00 (in 9.999258 seconds) 2020-03-31 09:14:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:06 UTC)" (scheduled at 2020-03-31 09:14:56.924497+00:00) 2020-03-31 09:14:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:14:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 56, 925936), True, datetime.datetime(2020, 3, 31, 9, 14, 56, 925936), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:14:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 14, 56, 925936), True, datetime.datetime(2020, 3, 31, 9, 14, 56, 925936), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:14:56,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:14:56,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:14:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:14:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:14:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:06 UTC)" executed successfully 2020-03-31 09:14:58,273 [527] [DEBUG] [app] Starting request: urn:request:a0668f20-bb2d-4c27-bc41-940b88f29aad (/health/instance) gunicorn-web stdout | 2020-03-31 09:14:58,273 [527] [DEBUG] [app] Starting request: urn:request:a0668f20-bb2d-4c27-bc41-940b88f29aad (/health/instance) 2020-03-31 09:14:58,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:58,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:58,282 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:58,282 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:58,283 [553] [DEBUG] [app] Starting request: urn:request:536a42ea-04f5-4927-a571-4eaa14a22b4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:58,283 [553] [DEBUG] [app] Starting request: urn:request:536a42ea-04f5-4927-a571-4eaa14a22b4c (/v1/_internal_ping) 2020-03-31 09:14:58,284 [553] [DEBUG] [app] Ending request: urn:request:536a42ea-04f5-4927-a571-4eaa14a22b4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:14:58,284 [553] [DEBUG] [app] Ending request: urn:request:536a42ea-04f5-4927-a571-4eaa14a22b4c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:58,285 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:58,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:14:58,285 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:58,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:14:58,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:58,287 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:14:58,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:14:58,287 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:14:58,288 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:14:58,288 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:14:58,289 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:14:58,289 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:14:58,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:14:58,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:14:58,315 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:14:58,315 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:14:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:58,320 [526] [DEBUG] [app] Starting request: urn:request:2b90b3ea-9881-48b7-8988-50bd3c2f2d28 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:58,320 [526] [DEBUG] [app] Starting request: urn:request:2b90b3ea-9881-48b7-8988-50bd3c2f2d28 (/_internal_ping) 2020-03-31 09:14:58,321 [526] [DEBUG] [app] Ending request: urn:request:2b90b3ea-9881-48b7-8988-50bd3c2f2d28 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:14:58,321 [526] [DEBUG] [app] Ending request: urn:request:2b90b3ea-9881-48b7-8988-50bd3c2f2d28 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:14:58,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:14:58,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:14:58,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:58,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:14:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:14:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:14:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:14:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:14:58,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:14:58,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:14:58,333 [543] [DEBUG] [app] Starting request: urn:request:fec18f4e-353b-420a-9b48-92db07f03032 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:58,333 [543] [DEBUG] [app] Starting request: urn:request:fec18f4e-353b-420a-9b48-92db07f03032 (/c1/_internal_ping) 2020-03-31 09:14:58,334 [543] [DEBUG] [app] Ending request: urn:request:fec18f4e-353b-420a-9b48-92db07f03032 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:14:58,334 [543] [DEBUG] [app] Ending request: urn:request:fec18f4e-353b-420a-9b48-92db07f03032 (/c1/_internal_ping) 2020-03-31 09:14:58,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:14:58,335 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:14:58,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:14:58,335 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:14:58,336 [527] [DEBUG] [app] Ending request: urn:request:a0668f20-bb2d-4c27-bc41-940b88f29aad (/health/instance) 2020-03-31 09:14:58,337 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:14:58,336 [527] [DEBUG] [app] Ending request: urn:request:a0668f20-bb2d-4c27-bc41-940b88f29aad (/health/instance) 2020-03-31 09:14:58,337 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:14:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:14:58,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:14:58,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:14:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:14:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:14:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:14:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:59 UTC)" (scheduled at 2020-03-31 09:14:59.560326+00:00) 2020-03-31 09:14:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:14:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:59 UTC)" (scheduled at 2020-03-31 09:14:59.560326+00:00) 2020-03-31 09:14:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:14:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:15:59 UTC)" executed successfully 2020-03-31 09:14:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:51.110610+00:00 (in 51.549046 seconds) namespacegcworker stdout | 2020-03-31 09:14:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:51.110610+00:00 (in 51.549046 seconds) 2020-03-31 09:15:00,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:15:00,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:30.337699+00:00 (in 29.996620 seconds) 2020-03-31 09:15:00,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:30 UTC)" (scheduled at 2020-03-31 09:15:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:15:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:30.337699+00:00 (in 29.996620 seconds) 2020-03-31 09:15:00,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:30 UTC)" (scheduled at 2020-03-31 09:15:00.337699+00:00) 2020-03-31 09:15:00,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 15, 0, 341868), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:15:00,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 15, 0, 341868), False, 50, 1, 0]) 2020-03-31 09:15:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:15:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:15:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:15:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:30 UTC)" executed successfully 2020-03-31 09:15:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:15:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:11.440707+00:00 (in 9.997835 seconds) 2020-03-31 09:15:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:11 UTC)" (scheduled at 2020-03-31 09:15:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:15:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:11.440707+00:00 (in 9.997835 seconds) 2020-03-31 09:15:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:11 UTC)" (scheduled at 2020-03-31 09:15:01.440707+00:00) 2020-03-31 09:15:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:15:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:15:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:15:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:15:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:15:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:11 UTC)" executed successfully 2020-03-31 09:15:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:15:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:24.410927+00:00 (in 21.079660 seconds) 2020-03-31 09:15:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:33 UTC)" (scheduled at 2020-03-31 09:15:03.330112+00:00) 2020-03-31 09:15:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:15:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:24.410927+00:00 (in 21.079660 seconds) 2020-03-31 09:15:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:33 UTC)" (scheduled at 2020-03-31 09:15:03.330112+00:00) 2020-03-31 09:15:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 3, 332350), True, datetime.datetime(2020, 3, 31, 9, 15, 3, 332350), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:15:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 3, 332350), True, datetime.datetime(2020, 3, 31, 9, 15, 3, 332350), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:15:03,355 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:15:03,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:15:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:33 UTC)" executed successfully 2020-03-31 09:15:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:15:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:14.323352+00:00 (in 9.999112 seconds) 2020-03-31 09:15:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:14 UTC)" (scheduled at 2020-03-31 09:15:04.323352+00:00) 2020-03-31 09:15:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:15:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:14.323352+00:00 (in 9.999112 seconds) 2020-03-31 09:15:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:14 UTC)" (scheduled at 2020-03-31 09:15:04.323352+00:00) 2020-03-31 09:15:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 4, 324955), True, datetime.datetime(2020, 3, 31, 9, 15, 4, 324955), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:15:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 4, 324955), True, datetime.datetime(2020, 3, 31, 9, 15, 4, 324955), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:15:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:04,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:15:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:15:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:14 UTC)" executed successfully 2020-03-31 09:15:04,446 [518] [DEBUG] [app] Starting request: urn:request:226fd679-edad-4011-80f0-d46a0961c705 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:04,446 [518] [DEBUG] [app] Starting request: urn:request:226fd679-edad-4011-80f0-d46a0961c705 (/health/instance) 2020-03-31 09:15:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:04,455 [546] [DEBUG] [app] Starting request: urn:request:d805695f-d87b-4d23-bdc7-cae8bc21a44a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:04,455 [546] [DEBUG] [app] Starting request: urn:request:d805695f-d87b-4d23-bdc7-cae8bc21a44a (/v1/_internal_ping) 2020-03-31 09:15:04,457 [546] [DEBUG] [app] Ending request: urn:request:d805695f-d87b-4d23-bdc7-cae8bc21a44a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:04,457 [546] [DEBUG] [app] Ending request: urn:request:d805695f-d87b-4d23-bdc7-cae8bc21a44a (/v1/_internal_ping) 2020-03-31 09:15:04,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:04,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:15:04,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:04,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:04,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:04,460 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:15:04,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:04,461 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:04,460 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:04,461 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:04,462 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:04,462 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:04,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:04,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:04,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:04,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:04,495 [527] [DEBUG] [app] Starting request: urn:request:01f0449e-3a77-4e16-a055-6b358f771534 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:04,495 [527] [DEBUG] [app] Starting request: urn:request:01f0449e-3a77-4e16-a055-6b358f771534 (/_internal_ping) 2020-03-31 09:15:04,496 [527] [DEBUG] [app] Ending request: urn:request:01f0449e-3a77-4e16-a055-6b358f771534 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:04,496 [527] [DEBUG] [app] Ending request: urn:request:01f0449e-3a77-4e16-a055-6b358f771534 (/_internal_ping) 2020-03-31 09:15:04,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:15:04,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:15:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:15:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:15:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:15:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:15:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:04,508 [543] [DEBUG] [app] Starting request: urn:request:302c9039-6a61-4c0b-8305-4e0081436293 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:04,508 [543] [DEBUG] [app] Starting request: urn:request:302c9039-6a61-4c0b-8305-4e0081436293 (/c1/_internal_ping) 2020-03-31 09:15:04,509 [543] [DEBUG] [app] Ending request: urn:request:302c9039-6a61-4c0b-8305-4e0081436293 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:04,509 [543] [DEBUG] [app] Ending request: urn:request:302c9039-6a61-4c0b-8305-4e0081436293 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:04,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:04,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:15:04,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:04,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:04,511 [518] [DEBUG] [app] Ending request: urn:request:226fd679-edad-4011-80f0-d46a0961c705 (/health/instance) 2020-03-31 09:15:04,511 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:04,511 [518] [DEBUG] [app] Ending request: urn:request:226fd679-edad-4011-80f0-d46a0961c705 (/health/instance) 2020-03-31 09:15:04,511 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:04,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:15:04,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:15:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:15:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:16.924497+00:00 (in 9.999260 seconds) 2020-03-31 09:15:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:16 UTC)" (scheduled at 2020-03-31 09:15:06.924497+00:00) 2020-03-31 09:15:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:15:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:16.924497+00:00 (in 9.999260 seconds) 2020-03-31 09:15:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:16 UTC)" (scheduled at 2020-03-31 09:15:06.924497+00:00) 2020-03-31 09:15:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 6, 926172), True, datetime.datetime(2020, 3, 31, 9, 15, 6, 926172), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:15:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 6, 926172), True, datetime.datetime(2020, 3, 31, 9, 15, 6, 926172), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:15:06,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:15:06,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:15:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:16 UTC)" executed successfully 2020-03-31 09:15:08,273 [518] [DEBUG] [app] Starting request: urn:request:bba3d53f-8150-4b57-9b7f-a627d9bbc1e4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:08,273 [518] [DEBUG] [app] Starting request: urn:request:bba3d53f-8150-4b57-9b7f-a627d9bbc1e4 (/health/instance) 2020-03-31 09:15:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:08,279 [547] [DEBUG] [app] Starting request: urn:request:089b989f-591f-491b-b234-2eb08e6fd100 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:08,279 [547] [DEBUG] [app] Starting request: urn:request:089b989f-591f-491b-b234-2eb08e6fd100 (/v1/_internal_ping) 2020-03-31 09:15:08,280 [547] [DEBUG] [app] Ending request: urn:request:089b989f-591f-491b-b234-2eb08e6fd100 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:08,280 [547] [DEBUG] [app] Ending request: urn:request:089b989f-591f-491b-b234-2eb08e6fd100 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:08,281 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:15:08,281 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:15:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:08,286 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:08,286 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:08,288 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:08,288 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:08,289 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:08,289 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:08,314 [518] [DEBUG] [app] Starting request: urn:request:a38c20c5-cb46-465a-bc8d-329aa8bee116 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:08,314 [518] [DEBUG] [app] Starting request: urn:request:a38c20c5-cb46-465a-bc8d-329aa8bee116 (/_internal_ping) 2020-03-31 09:15:08,315 [518] [DEBUG] [app] Ending request: urn:request:a38c20c5-cb46-465a-bc8d-329aa8bee116 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:08,315 [518] [DEBUG] [app] Ending request: urn:request:a38c20c5-cb46-465a-bc8d-329aa8bee116 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:15:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:15:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:15:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:15:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:08,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:08,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:08,331 [543] [DEBUG] [app] Starting request: urn:request:cec393f0-f53a-46b4-b74f-e15a0819d162 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:08,331 [543] [DEBUG] [app] Starting request: urn:request:cec393f0-f53a-46b4-b74f-e15a0819d162 (/c1/_internal_ping) 2020-03-31 09:15:08,332 [543] [DEBUG] [app] Ending request: urn:request:cec393f0-f53a-46b4-b74f-e15a0819d162 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:08,332 [543] [DEBUG] [app] Ending request: urn:request:cec393f0-f53a-46b4-b74f-e15a0819d162 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:08,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:08,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:08,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:15:08,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:08,335 [518] [DEBUG] [app] Ending request: urn:request:bba3d53f-8150-4b57-9b7f-a627d9bbc1e4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:08,335 [518] [DEBUG] [app] Ending request: urn:request:bba3d53f-8150-4b57-9b7f-a627d9bbc1e4 (/health/instance) 2020-03-31 09:15:08,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:08,336 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:15:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:15:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:15:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:21 UTC)" (scheduled at 2020-03-31 09:15:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:15:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:21 UTC)" (scheduled at 2020-03-31 09:15:11.440707+00:00) 2020-03-31 09:15:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:15:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:15:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:21.440707+00:00 (in 9.998846 seconds) repositoryactioncounter stdout | 2020-03-31 09:15:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:21.440707+00:00 (in 9.998846 seconds) 2020-03-31 09:15:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:15:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:15:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:15:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:21 UTC)" executed successfully 2020-03-31 09:15:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:15:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:24.323352+00:00 (in 9.999174 seconds) 2020-03-31 09:15:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:24 UTC)" (scheduled at 2020-03-31 09:15:14.323352+00:00) 2020-03-31 09:15:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:15:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:24.323352+00:00 (in 9.999174 seconds) 2020-03-31 09:15:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:24 UTC)" (scheduled at 2020-03-31 09:15:14.323352+00:00) 2020-03-31 09:15:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 14, 325118), True, datetime.datetime(2020, 3, 31, 9, 15, 14, 325118), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:15:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 14, 325118), True, datetime.datetime(2020, 3, 31, 9, 15, 14, 325118), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:15:14,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:15:14,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:14,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:14,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:15:14,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:14,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:24 UTC)" executed successfully 2020-03-31 09:15:14,446 [518] [DEBUG] [app] Starting request: urn:request:fd949dd6-3131-49a3-80b9-43af3e52f6aa (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:14,446 [518] [DEBUG] [app] Starting request: urn:request:fd949dd6-3131-49a3-80b9-43af3e52f6aa (/health/instance) 2020-03-31 09:15:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:14,454 [550] [DEBUG] [app] Starting request: urn:request:2ef1c972-80c9-42c6-8d96-3c6474fd9aa7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:14,454 [550] [DEBUG] [app] Starting request: urn:request:2ef1c972-80c9-42c6-8d96-3c6474fd9aa7 (/v1/_internal_ping) 2020-03-31 09:15:14,455 [550] [DEBUG] [app] Ending request: urn:request:2ef1c972-80c9-42c6-8d96-3c6474fd9aa7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:14,455 [550] [DEBUG] [app] Ending request: urn:request:2ef1c972-80c9-42c6-8d96-3c6474fd9aa7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:15:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:15:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:14,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:14,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:14,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:14,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:14,488 [518] [DEBUG] [app] Starting request: urn:request:88db2af9-4685-4c8e-8234-859d4edcdb4d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:14,488 [518] [DEBUG] [app] Starting request: urn:request:88db2af9-4685-4c8e-8234-859d4edcdb4d (/_internal_ping) 2020-03-31 09:15:14,489 [518] [DEBUG] [app] Ending request: urn:request:88db2af9-4685-4c8e-8234-859d4edcdb4d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:14,489 [518] [DEBUG] [app] Ending request: urn:request:88db2af9-4685-4c8e-8234-859d4edcdb4d (/_internal_ping) 2020-03-31 09:15:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:15:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:15:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:15:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:15:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:15:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:14,503 [543] [DEBUG] [app] Starting request: urn:request:00b8f4bf-c64b-4e1e-888f-2ed38324e661 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:14,503 [543] [DEBUG] [app] Starting request: urn:request:00b8f4bf-c64b-4e1e-888f-2ed38324e661 (/c1/_internal_ping) 2020-03-31 09:15:14,504 [543] [DEBUG] [app] Ending request: urn:request:00b8f4bf-c64b-4e1e-888f-2ed38324e661 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:14,504 [543] [DEBUG] [app] Ending request: urn:request:00b8f4bf-c64b-4e1e-888f-2ed38324e661 (/c1/_internal_ping) 2020-03-31 09:15:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:14,505 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:14,506 [518] [DEBUG] [app] Ending request: urn:request:fd949dd6-3131-49a3-80b9-43af3e52f6aa (/health/instance) 2020-03-31 09:15:14,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:15:14,505 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:14,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:15:14,506 [518] [DEBUG] [app] Ending request: urn:request:fd949dd6-3131-49a3-80b9-43af3e52f6aa (/health/instance) 2020-03-31 09:15:14,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:14,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:15:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:15:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:33.366791+00:00 (in 18.826127 seconds) 2020-03-31 09:15:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:20:14 UTC)" (scheduled at 2020-03-31 09:15:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:15:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:33.366791+00:00 (in 18.826127 seconds) 2020-03-31 09:15:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:20:14 UTC)" (scheduled at 2020-03-31 09:15:14.539408+00:00) 2020-03-31 09:15:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:15:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), u'exportactionlogs/%']) 2020-03-31 09:15:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), True, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:15:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), True, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), 0, u'exportactionlogs/%']) 2020-03-31 09:15:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), True, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:15:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), True, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 15, 14, 541421), u'exportactionlogs/%']) 2020-03-31 09:15:14,564 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:15:14,564 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:14,565 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:20:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:15:14,565 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:20:14 UTC)" executed successfully 2020-03-31 09:15:14,567 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:15:14,567 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:15:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:14,569 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:15:14,569 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:15:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:15:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:45.064514+00:00 (in 29.999061 seconds) 2020-03-31 09:15:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:45 UTC)" (scheduled at 2020-03-31 09:15:15.064514+00:00) gcworker stdout | 2020-03-31 09:15:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:45.064514+00:00 (in 29.999061 seconds) 2020-03-31 09:15:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:45 UTC)" (scheduled at 2020-03-31 09:15:15.064514+00:00) 2020-03-31 09:15:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436515, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:15:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436515, 1209600, 500, 1, 0]) 2020-03-31 09:15:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:15:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:15:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:15:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:45 UTC)" executed successfully 2020-03-31 09:15:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:15:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:26.924497+00:00 (in 9.999174 seconds) 2020-03-31 09:15:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:26 UTC)" (scheduled at 2020-03-31 09:15:16.924497+00:00) 2020-03-31 09:15:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:15:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:26.924497+00:00 (in 9.999174 seconds) 2020-03-31 09:15:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:26 UTC)" (scheduled at 2020-03-31 09:15:16.924497+00:00) 2020-03-31 09:15:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 16, 926005), True, datetime.datetime(2020, 3, 31, 9, 15, 16, 926005), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:15:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 16, 926005), True, datetime.datetime(2020, 3, 31, 9, 15, 16, 926005), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:15:16,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:15:16,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:15:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:26 UTC)" executed successfully 2020-03-31 09:15:18,273 [518] [DEBUG] [app] Starting request: urn:request:9fed87d9-43c9-4f5e-aa8e-915b676291f5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:18,273 [518] [DEBUG] [app] Starting request: urn:request:9fed87d9-43c9-4f5e-aa8e-915b676291f5 (/health/instance) 2020-03-31 09:15:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:18,279 [550] [DEBUG] [app] Starting request: urn:request:01f4f971-7be9-497f-a6eb-9be7a6863bd9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:18,279 [550] [DEBUG] [app] Starting request: urn:request:01f4f971-7be9-497f-a6eb-9be7a6863bd9 (/v1/_internal_ping) 2020-03-31 09:15:18,280 [550] [DEBUG] [app] Ending request: urn:request:01f4f971-7be9-497f-a6eb-9be7a6863bd9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:18,280 [550] [DEBUG] [app] Ending request: urn:request:01f4f971-7be9-497f-a6eb-9be7a6863bd9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:18,281 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:15:18,281 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:18,283 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:15:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:15:18,283 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:18,284 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:18,284 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:18,285 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:18,285 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:18,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:18,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:18,309 [518] [DEBUG] [app] Starting request: urn:request:f9160f98-f652-4f16-a272-ca1a69f28781 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:18,309 [518] [DEBUG] [app] Starting request: urn:request:f9160f98-f652-4f16-a272-ca1a69f28781 (/_internal_ping) 2020-03-31 09:15:18,310 [518] [DEBUG] [app] Ending request: urn:request:f9160f98-f652-4f16-a272-ca1a69f28781 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:18,310 [518] [DEBUG] [app] Ending request: urn:request:f9160f98-f652-4f16-a272-ca1a69f28781 (/_internal_ping) 2020-03-31 09:15:18,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:15:18,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:15:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:15:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:15:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:18,323 [541] [DEBUG] [app] Starting request: urn:request:036a30d0-9b5b-4fe3-81d8-c4daf0cd71c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:18,323 [541] [DEBUG] [app] Starting request: urn:request:036a30d0-9b5b-4fe3-81d8-c4daf0cd71c7 (/c1/_internal_ping) 2020-03-31 09:15:18,324 [541] [DEBUG] [app] Ending request: urn:request:036a30d0-9b5b-4fe3-81d8-c4daf0cd71c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:18,324 [541] [DEBUG] [app] Ending request: urn:request:036a30d0-9b5b-4fe3-81d8-c4daf0cd71c7 (/c1/_internal_ping) 2020-03-31 09:15:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:18,325 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:18,326 [518] [DEBUG] [app] Ending request: urn:request:9fed87d9-43c9-4f5e-aa8e-915b676291f5 (/health/instance) 2020-03-31 09:15:18,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:15:18,325 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:18,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:15:18,326 [518] [DEBUG] [app] Ending request: urn:request:9fed87d9-43c9-4f5e-aa8e-915b676291f5 (/health/instance) 2020-03-31 09:15:18,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:18,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:15:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:15:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:31.440707+00:00 (in 9.996069 seconds) 2020-03-31 09:15:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:31 UTC)" (scheduled at 2020-03-31 09:15:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:15:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:31.440707+00:00 (in 9.996069 seconds) 2020-03-31 09:15:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:31 UTC)" (scheduled at 2020-03-31 09:15:21.440707+00:00) 2020-03-31 09:15:21,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:15:21,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:15:21,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:15:21,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:15:21,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:15:21,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:31 UTC)" executed successfully 2020-03-31 09:15:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:15:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:52.113221+00:00 (in 29.998986 seconds) 2020-03-31 09:15:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:52 UTC)" (scheduled at 2020-03-31 09:15:22.113221+00:00) securityworker stdout | 2020-03-31 09:15:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:52.113221+00:00 (in 29.998986 seconds) 2020-03-31 09:15:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:52 UTC)" (scheduled at 2020-03-31 09:15:22.113221+00:00) 2020-03-31 09:15:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:15:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:15:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:15:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:15:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:15:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:15:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:15:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:15:52 UTC)" executed successfully 2020-03-31 09:15:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:15:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:34.323352+00:00 (in 9.999051 seconds) 2020-03-31 09:15:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:34 UTC)" (scheduled at 2020-03-31 09:15:24.323352+00:00) 2020-03-31 09:15:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:15:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:34.323352+00:00 (in 9.999051 seconds) 2020-03-31 09:15:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:34 UTC)" (scheduled at 2020-03-31 09:15:24.323352+00:00) 2020-03-31 09:15:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 24, 325052), True, datetime.datetime(2020, 3, 31, 9, 15, 24, 325052), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:15:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 24, 325052), True, datetime.datetime(2020, 3, 31, 9, 15, 24, 325052), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:15:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:15:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:15:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:34 UTC)" executed successfully 2020-03-31 09:15:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:15:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:33.330112+00:00 (in 8.918250 seconds) 2020-03-31 09:15:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:24 UTC)" (scheduled at 2020-03-31 09:15:24.410927+00:00) 2020-03-31 09:15:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:15:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:15:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:33.330112+00:00 (in 8.918250 seconds) 2020-03-31 09:15:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:24 UTC)" (scheduled at 2020-03-31 09:15:24.410927+00:00) 2020-03-31 09:15:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:15:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:24 UTC)" executed successfully 2020-03-31 09:15:24,446 [525] [DEBUG] [app] Starting request: urn:request:13aa3dae-090e-439d-8d08-1a2a5b17cd74 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:24,446 [525] [DEBUG] [app] Starting request: urn:request:13aa3dae-090e-439d-8d08-1a2a5b17cd74 (/health/instance) 2020-03-31 09:15:24,448 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:24,448 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:24,454 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:24,454 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:24,456 [556] [DEBUG] [app] Starting request: urn:request:cbf21583-ae33-439f-858c-a566bcdb7eeb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:24,456 [556] [DEBUG] [app] Starting request: urn:request:cbf21583-ae33-439f-858c-a566bcdb7eeb (/v1/_internal_ping) 2020-03-31 09:15:24,457 [556] [DEBUG] [app] Ending request: urn:request:cbf21583-ae33-439f-858c-a566bcdb7eeb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:24,457 [556] [DEBUG] [app] Ending request: urn:request:cbf21583-ae33-439f-858c-a566bcdb7eeb (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:15:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:24,461 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:24,459 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:24,460 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:15:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:24,461 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:15:24,459 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:24,460 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:24,463 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:24,464 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:24,463 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:24,464 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:24,472 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:24,472 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:24,494 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:24,494 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:24,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:24,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:24,500 [520] [DEBUG] [app] Starting request: urn:request:851ce225-9f31-4d6d-b1fc-4d4cb9ff5fa8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:24,500 [520] [DEBUG] [app] Starting request: urn:request:851ce225-9f31-4d6d-b1fc-4d4cb9ff5fa8 (/_internal_ping) 2020-03-31 09:15:24,502 [520] [DEBUG] [app] Ending request: urn:request:851ce225-9f31-4d6d-b1fc-4d4cb9ff5fa8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:24,502 [520] [DEBUG] [app] Ending request: urn:request:851ce225-9f31-4d6d-b1fc-4d4cb9ff5fa8 (/_internal_ping) 2020-03-31 09:15:24,502 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:15:24,503 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:15:24,502 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:15:24,503 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:24,507 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:24,507 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:24,508 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:15:24,508 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:24,512 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:24,512 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:24,513 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:15:24,513 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:15:24,517 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:24,517 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:24,519 [541] [DEBUG] [app] Starting request: urn:request:ad518b5c-ccbc-4e1b-8b67-d857293e34ee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:24,519 [541] [DEBUG] [app] Starting request: urn:request:ad518b5c-ccbc-4e1b-8b67-d857293e34ee (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:24,520 [541] [DEBUG] [app] Ending request: urn:request:ad518b5c-ccbc-4e1b-8b67-d857293e34ee (/c1/_internal_ping) 2020-03-31 09:15:24,521 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:24,521 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:15:24,520 [541] [DEBUG] [app] Ending request: urn:request:ad518b5c-ccbc-4e1b-8b67-d857293e34ee (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:24,521 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:24,522 [525] [DEBUG] [app] Ending request: urn:request:13aa3dae-090e-439d-8d08-1a2a5b17cd74 (/health/instance) 2020-03-31 09:15:24,523 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:15:24,521 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:24,523 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.078 49 0.078) gunicorn-web stdout | 2020-03-31 09:15:24,522 [525] [DEBUG] [app] Ending request: urn:request:13aa3dae-090e-439d-8d08-1a2a5b17cd74 (/health/instance) 2020-03-31 09:15:24,523 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:24,523 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.078 49 0.078) 2020-03-31 09:15:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:15:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:36.924497+00:00 (in 9.999323 seconds) 2020-03-31 09:15:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:36 UTC)" (scheduled at 2020-03-31 09:15:26.924497+00:00) 2020-03-31 09:15:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:15:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:36.924497+00:00 (in 9.999323 seconds) 2020-03-31 09:15:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:36 UTC)" (scheduled at 2020-03-31 09:15:26.924497+00:00) 2020-03-31 09:15:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 26, 925944), True, datetime.datetime(2020, 3, 31, 9, 15, 26, 925944), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:15:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 26, 925944), True, datetime.datetime(2020, 3, 31, 9, 15, 26, 925944), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:15:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:15:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:15:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:36 UTC)" executed successfully 2020-03-31 09:15:28,273 [518] [DEBUG] [app] Starting request: urn:request:7b7b5d79-3250-4172-bcc9-882de92ecd8c (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:28,273 [518] [DEBUG] [app] Starting request: urn:request:7b7b5d79-3250-4172-bcc9-882de92ecd8c (/health/instance) 2020-03-31 09:15:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:28,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:28,282 [538] [DEBUG] [app] Starting request: urn:request:69d32dd1-d74d-4ff9-a9a4-4f2f4116b00e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:28,282 [538] [DEBUG] [app] Starting request: urn:request:69d32dd1-d74d-4ff9-a9a4-4f2f4116b00e (/v1/_internal_ping) 2020-03-31 09:15:28,283 [538] [DEBUG] [app] Ending request: urn:request:69d32dd1-d74d-4ff9-a9a4-4f2f4116b00e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:28,283 [538] [DEBUG] [app] Ending request: urn:request:69d32dd1-d74d-4ff9-a9a4-4f2f4116b00e (/v1/_internal_ping) 2020-03-31 09:15:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:15:28,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:15:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:28,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:15:28,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:28,287 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:28,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:28,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:28,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:15:28,287 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:15:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:28,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:28,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:28,319 [518] [DEBUG] [app] Starting request: urn:request:c681594b-2cbc-4d27-b578-37028b26cf3e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:28,319 [518] [DEBUG] [app] Starting request: urn:request:c681594b-2cbc-4d27-b578-37028b26cf3e (/_internal_ping) 2020-03-31 09:15:28,320 [518] [DEBUG] [app] Ending request: urn:request:c681594b-2cbc-4d27-b578-37028b26cf3e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:28,320 [518] [DEBUG] [app] Ending request: urn:request:c681594b-2cbc-4d27-b578-37028b26cf3e (/_internal_ping) 2020-03-31 09:15:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:15:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:15:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:15:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:15:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:15:28,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:28,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:28,338 [541] [DEBUG] [app] Starting request: urn:request:7fa8b764-16b3-4a86-80f3-d8041571a969 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:28,338 [541] [DEBUG] [app] Starting request: urn:request:7fa8b764-16b3-4a86-80f3-d8041571a969 (/c1/_internal_ping) 2020-03-31 09:15:28,339 [541] [DEBUG] [app] Ending request: urn:request:7fa8b764-16b3-4a86-80f3-d8041571a969 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:28,339 [541] [DEBUG] [app] Ending request: urn:request:7fa8b764-16b3-4a86-80f3-d8041571a969 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:28,340 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:28,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:15:28,340 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:28,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:28,342 [518] [DEBUG] [app] Ending request: urn:request:7b7b5d79-3250-4172-bcc9-882de92ecd8c (/health/instance) 2020-03-31 09:15:28,342 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:28,342 [518] [DEBUG] [app] Ending request: urn:request:7b7b5d79-3250-4172-bcc9-882de92ecd8c (/health/instance) 2020-03-31 09:15:28,342 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:28,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 09:15:28,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:15:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:15:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:00 UTC)" (scheduled at 2020-03-31 09:15:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:15:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:00 UTC)" (scheduled at 2020-03-31 09:15:30.337699+00:00) 2020-03-31 09:15:30,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:00.337699+00:00 (in 29.999177 seconds) buildlogsarchiver stdout | 2020-03-31 09:15:30,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:00.337699+00:00 (in 29.999177 seconds) 2020-03-31 09:15:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 15, 30, 339175), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:15:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 15, 30, 339175), False, 50, 1, 0]) 2020-03-31 09:15:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:15:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:15:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:15:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:00 UTC)" executed successfully 2020-03-31 09:15:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:15:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:41.440707+00:00 (in 9.999080 seconds) 2020-03-31 09:15:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:41 UTC)" (scheduled at 2020-03-31 09:15:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:15:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:41.440707+00:00 (in 9.999080 seconds) 2020-03-31 09:15:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:41 UTC)" (scheduled at 2020-03-31 09:15:31.440707+00:00) 2020-03-31 09:15:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:15:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:15:31,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:15:31,464 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:15:31,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:15:31,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:31,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:15:31,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:41 UTC)" executed successfully 2020-03-31 09:15:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:15:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:03.330112+00:00 (in 29.999156 seconds) 2020-03-31 09:15:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:03 UTC)" (scheduled at 2020-03-31 09:15:33.330112+00:00) 2020-03-31 09:15:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:15:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:03.330112+00:00 (in 29.999156 seconds) 2020-03-31 09:15:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:03 UTC)" (scheduled at 2020-03-31 09:15:33.330112+00:00) 2020-03-31 09:15:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 33, 331831), True, datetime.datetime(2020, 3, 31, 9, 15, 33, 331831), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:15:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 33, 331831), True, datetime.datetime(2020, 3, 31, 9, 15, 33, 331831), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:15:33,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:15:33,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:15:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:03 UTC)" executed successfully 2020-03-31 09:15:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:15:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:33 UTC)" (scheduled at 2020-03-31 09:15:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:15:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:33 UTC)" (scheduled at 2020-03-31 09:15:33.366791+00:00) 2020-03-31 09:15:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:43.539999+00:00 (in 10.172419 seconds) 2020-03-31 09:15:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:15:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:43.539999+00:00 (in 10.172419 seconds) 2020-03-31 09:15:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 33, 368437), True, datetime.datetime(2020, 3, 31, 9, 15, 33, 368437), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:15:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 33, 368437), True, datetime.datetime(2020, 3, 31, 9, 15, 33, 368437), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:15:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:33,383 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:15:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:15:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:33 UTC)" executed successfully 2020-03-31 09:15:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:15:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:44.323352+00:00 (in 9.999270 seconds) 2020-03-31 09:15:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:44 UTC)" (scheduled at 2020-03-31 09:15:34.323352+00:00) 2020-03-31 09:15:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:15:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:44.323352+00:00 (in 9.999270 seconds) 2020-03-31 09:15:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:44 UTC)" (scheduled at 2020-03-31 09:15:34.323352+00:00) 2020-03-31 09:15:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 34, 324990), True, datetime.datetime(2020, 3, 31, 9, 15, 34, 324990), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:15:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 34, 324990), True, datetime.datetime(2020, 3, 31, 9, 15, 34, 324990), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:15:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:15:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:15:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:44 UTC)" executed successfully 2020-03-31 09:15:34,446 [527] [DEBUG] [app] Starting request: urn:request:cd47bf82-7ba4-4327-9856-a538a8b335c0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:34,446 [527] [DEBUG] [app] Starting request: urn:request:cd47bf82-7ba4-4327-9856-a538a8b335c0 (/health/instance) 2020-03-31 09:15:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:34,455 [545] [DEBUG] [app] Starting request: urn:request:16fb497a-a53b-45e4-8fea-b8ae55bf6200 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:34,455 [545] [DEBUG] [app] Starting request: urn:request:16fb497a-a53b-45e4-8fea-b8ae55bf6200 (/v1/_internal_ping) 2020-03-31 09:15:34,456 [545] [DEBUG] [app] Ending request: urn:request:16fb497a-a53b-45e4-8fea-b8ae55bf6200 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:34,456 [545] [DEBUG] [app] Ending request: urn:request:16fb497a-a53b-45e4-8fea-b8ae55bf6200 (/v1/_internal_ping) 2020-03-31 09:15:34,456 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:15:34,456 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:34,458 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:15:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:15:34,458 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:34,460 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:34,460 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:34,461 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:34,461 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:34,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:34,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:34,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:34,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:34,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:34,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:34,489 [518] [DEBUG] [app] Starting request: urn:request:267a60af-71c8-4a12-b04f-ec1f8e838ec0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:34,489 [518] [DEBUG] [app] Starting request: urn:request:267a60af-71c8-4a12-b04f-ec1f8e838ec0 (/_internal_ping) 2020-03-31 09:15:34,490 [518] [DEBUG] [app] Ending request: urn:request:267a60af-71c8-4a12-b04f-ec1f8e838ec0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:34,490 [518] [DEBUG] [app] Ending request: urn:request:267a60af-71c8-4a12-b04f-ec1f8e838ec0 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:15:34,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:34,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:15:34,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:34,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:34,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:34,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:15:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:15:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:34,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:34,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:34,504 [541] [DEBUG] [app] Starting request: urn:request:cc435ff6-8769-44c9-aa1a-dd64e0b63233 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:34,504 [541] [DEBUG] [app] Starting request: urn:request:cc435ff6-8769-44c9-aa1a-dd64e0b63233 (/c1/_internal_ping) 2020-03-31 09:15:34,505 [541] [DEBUG] [app] Ending request: urn:request:cc435ff6-8769-44c9-aa1a-dd64e0b63233 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:34,505 [541] [DEBUG] [app] Ending request: urn:request:cc435ff6-8769-44c9-aa1a-dd64e0b63233 (/c1/_internal_ping) 2020-03-31 09:15:34,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:34,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:34,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:15:34,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:34,507 [527] [DEBUG] [app] Ending request: urn:request:cd47bf82-7ba4-4327-9856-a538a8b335c0 (/health/instance) 2020-03-31 09:15:34,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:34,507 [527] [DEBUG] [app] Ending request: urn:request:cd47bf82-7ba4-4327-9856-a538a8b335c0 (/health/instance) 2020-03-31 09:15:34,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:34,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:15:34,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:15:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:15:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:38.010934+00:00 (in 1.085597 seconds) storagereplication stdout | 2020-03-31 09:15:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:38.010934+00:00 (in 1.085597 seconds) 2020-03-31 09:15:36,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:46 UTC)" (scheduled at 2020-03-31 09:15:36.924497+00:00) storagereplication stdout | 2020-03-31 09:15:36,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:46 UTC)" (scheduled at 2020-03-31 09:15:36.924497+00:00) 2020-03-31 09:15:36,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:15:36,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:36,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 36, 928007), True, datetime.datetime(2020, 3, 31, 9, 15, 36, 928007), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:15:36,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 36, 928007), True, datetime.datetime(2020, 3, 31, 9, 15, 36, 928007), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:15:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:15:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:15:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:46 UTC)" executed successfully 2020-03-31 09:15:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:15:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:46.924497+00:00 (in 8.912785 seconds) 2020-03-31 09:15:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:38 UTC)" (scheduled at 2020-03-31 09:15:38.010934+00:00) 2020-03-31 09:15:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:15:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:46.924497+00:00 (in 8.912785 seconds) 2020-03-31 09:15:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:38 UTC)" (scheduled at 2020-03-31 09:15:38.010934+00:00) 2020-03-31 09:15:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:15:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:15:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:38 UTC)" executed successfully 2020-03-31 09:15:38,273 [526] [DEBUG] [app] Starting request: urn:request:ba7dd518-9831-451a-8aea-2e3816016cd5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:38,273 [526] [DEBUG] [app] Starting request: urn:request:ba7dd518-9831-451a-8aea-2e3816016cd5 (/health/instance) 2020-03-31 09:15:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:38,278 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:38,280 [546] [DEBUG] [app] Starting request: urn:request:ecd53ae8-a366-42aa-bfc9-824f6e977518 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:38,280 [546] [DEBUG] [app] Starting request: urn:request:ecd53ae8-a366-42aa-bfc9-824f6e977518 (/v1/_internal_ping) 2020-03-31 09:15:38,281 [546] [DEBUG] [app] Ending request: urn:request:ecd53ae8-a366-42aa-bfc9-824f6e977518 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:38,281 [546] [DEBUG] [app] Ending request: urn:request:ecd53ae8-a366-42aa-bfc9-824f6e977518 (/v1/_internal_ping) 2020-03-31 09:15:38,281 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:15:38,281 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:38,281 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:38,282 [526] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:15:38,281 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:38,282 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:38,283 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:38,283 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:38,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:38,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:38,285 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:38,285 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:38,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:38,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:38,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:38,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:38,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:38,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:38,312 [526] [DEBUG] [app] Starting request: urn:request:2656a807-ecc3-4a74-9f78-fb031d3ccafb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:38,312 [526] [DEBUG] [app] Starting request: urn:request:2656a807-ecc3-4a74-9f78-fb031d3ccafb (/_internal_ping) 2020-03-31 09:15:38,313 [526] [DEBUG] [app] Ending request: urn:request:2656a807-ecc3-4a74-9f78-fb031d3ccafb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:38,313 [526] [DEBUG] [app] Ending request: urn:request:2656a807-ecc3-4a74-9f78-fb031d3ccafb (/_internal_ping) 2020-03-31 09:15:38,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:15:38,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:38,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:15:38,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:38,317 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:38,317 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:15:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:38,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:15:38,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:15:38,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:38,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:38,327 [541] [DEBUG] [app] Starting request: urn:request:b9a39107-f31b-4851-aaca-2212355bc965 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:38,327 [541] [DEBUG] [app] Starting request: urn:request:b9a39107-f31b-4851-aaca-2212355bc965 (/c1/_internal_ping) 2020-03-31 09:15:38,328 [541] [DEBUG] [app] Ending request: urn:request:b9a39107-f31b-4851-aaca-2212355bc965 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:38,328 [541] [DEBUG] [app] Ending request: urn:request:b9a39107-f31b-4851-aaca-2212355bc965 (/c1/_internal_ping) 2020-03-31 09:15:38,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:38,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:38,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:38,330 [526] [DEBUG] [app] Ending request: urn:request:ba7dd518-9831-451a-8aea-2e3816016cd5 (/health/instance) 2020-03-31 09:15:38,331 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:15:38,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:38,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:15:38,330 [526] [DEBUG] [app] Ending request: urn:request:ba7dd518-9831-451a-8aea-2e3816016cd5 (/health/instance) 2020-03-31 09:15:38,331 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:38,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:15:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:15:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:51.440707+00:00 (in 9.998976 seconds) 2020-03-31 09:15:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:51 UTC)" (scheduled at 2020-03-31 09:15:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:15:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:51.440707+00:00 (in 9.998976 seconds) 2020-03-31 09:15:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:51 UTC)" (scheduled at 2020-03-31 09:15:41.440707+00:00) 2020-03-31 09:15:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:15:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:15:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:15:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:15:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:15:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:51 UTC)" executed successfully 2020-03-31 09:15:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:15:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:33.366791+00:00 (in 49.825623 seconds) 2020-03-31 09:15:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:43 UTC)" (scheduled at 2020-03-31 09:15:43.539999+00:00) 2020-03-31 09:15:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:15:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:33.366791+00:00 (in 49.825623 seconds) 2020-03-31 09:15:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:43 UTC)" (scheduled at 2020-03-31 09:15:43.539999+00:00) 2020-03-31 09:15:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:15:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:15:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:43 UTC)" executed successfully 2020-03-31 09:15:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:15:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:49.498966+00:00 (in 5.174500 seconds) 2020-03-31 09:15:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:54 UTC)" (scheduled at 2020-03-31 09:15:44.323352+00:00) 2020-03-31 09:15:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:15:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:49.498966+00:00 (in 5.174500 seconds) 2020-03-31 09:15:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:54 UTC)" (scheduled at 2020-03-31 09:15:44.323352+00:00) 2020-03-31 09:15:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 44, 325398), True, datetime.datetime(2020, 3, 31, 9, 15, 44, 325398), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:15:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 44, 325398), True, datetime.datetime(2020, 3, 31, 9, 15, 44, 325398), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:15:44,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:15:44,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:15:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:54 UTC)" executed successfully 2020-03-31 09:15:44,446 [518] [DEBUG] [app] Starting request: urn:request:7ce4c52d-1cb6-4640-ab43-14616c78bbdc (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:44,446 [518] [DEBUG] [app] Starting request: urn:request:7ce4c52d-1cb6-4640-ab43-14616c78bbdc (/health/instance) 2020-03-31 09:15:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:44,454 [550] [DEBUG] [app] Starting request: urn:request:4a896817-38ee-47ca-b9d2-95604dbd9acf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:44,454 [550] [DEBUG] [app] Starting request: urn:request:4a896817-38ee-47ca-b9d2-95604dbd9acf (/v1/_internal_ping) 2020-03-31 09:15:44,455 [550] [DEBUG] [app] Ending request: urn:request:4a896817-38ee-47ca-b9d2-95604dbd9acf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:44,455 [550] [DEBUG] [app] Ending request: urn:request:4a896817-38ee-47ca-b9d2-95604dbd9acf (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:44,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:15:44,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:15:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:44,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:44,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:44,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:44,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:44,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:44,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:44,487 [525] [DEBUG] [app] Starting request: urn:request:7bab6b3e-bad6-485c-9ccf-1921d5c065f9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:44,487 [525] [DEBUG] [app] Starting request: urn:request:7bab6b3e-bad6-485c-9ccf-1921d5c065f9 (/_internal_ping) 2020-03-31 09:15:44,487 [525] [DEBUG] [app] Ending request: urn:request:7bab6b3e-bad6-485c-9ccf-1921d5c065f9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:44,487 [525] [DEBUG] [app] Ending request: urn:request:7bab6b3e-bad6-485c-9ccf-1921d5c065f9 (/_internal_ping) 2020-03-31 09:15:44,488 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:15:44,488 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:15:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:15:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:15:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:15:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:44,503 [541] [DEBUG] [app] Starting request: urn:request:b6ddd686-a8ba-4dc3-a83e-7d030fd3fde3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:44,503 [541] [DEBUG] [app] Starting request: urn:request:b6ddd686-a8ba-4dc3-a83e-7d030fd3fde3 (/c1/_internal_ping) 2020-03-31 09:15:44,504 [541] [DEBUG] [app] Ending request: urn:request:b6ddd686-a8ba-4dc3-a83e-7d030fd3fde3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:44,504 [541] [DEBUG] [app] Ending request: urn:request:b6ddd686-a8ba-4dc3-a83e-7d030fd3fde3 (/c1/_internal_ping) 2020-03-31 09:15:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:44,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:15:44,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:44,507 [518] [DEBUG] [app] Ending request: urn:request:7ce4c52d-1cb6-4640-ab43-14616c78bbdc (/health/instance) 2020-03-31 09:15:44,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:44,507 [518] [DEBUG] [app] Ending request: urn:request:7ce4c52d-1cb6-4640-ab43-14616c78bbdc (/health/instance) 2020-03-31 09:15:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:44,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:15:44,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:15:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:15:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:15.064514+00:00 (in 29.999119 seconds) 2020-03-31 09:15:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:15 UTC)" (scheduled at 2020-03-31 09:15:45.064514+00:00) gcworker stdout | 2020-03-31 09:15:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:15.064514+00:00 (in 29.999119 seconds) 2020-03-31 09:15:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:15 UTC)" (scheduled at 2020-03-31 09:15:45.064514+00:00) 2020-03-31 09:15:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436545, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:15:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436545, 1209600, 500, 1, 0]) 2020-03-31 09:15:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:15:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:15:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:15:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:15 UTC)" executed successfully 2020-03-31 09:15:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:15:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:56.924497+00:00 (in 9.998963 seconds) 2020-03-31 09:15:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:56 UTC)" (scheduled at 2020-03-31 09:15:46.924497+00:00) 2020-03-31 09:15:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:15:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:56.924497+00:00 (in 9.998963 seconds) 2020-03-31 09:15:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:56 UTC)" (scheduled at 2020-03-31 09:15:46.924497+00:00) 2020-03-31 09:15:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 46, 926173), True, datetime.datetime(2020, 3, 31, 9, 15, 46, 926173), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:15:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 46, 926173), True, datetime.datetime(2020, 3, 31, 9, 15, 46, 926173), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:15:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:15:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:15:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:15:56 UTC)" executed successfully 2020-03-31 09:15:48,273 [527] [DEBUG] [app] Starting request: urn:request:0a853314-9474-4648-9336-ca918e4e1c59 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:48,273 [527] [DEBUG] [app] Starting request: urn:request:0a853314-9474-4648-9336-ca918e4e1c59 (/health/instance) 2020-03-31 09:15:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:48,282 [550] [DEBUG] [app] Starting request: urn:request:36bf0aaa-ab5a-4781-ab67-875c73e91029 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:48,282 [550] [DEBUG] [app] Starting request: urn:request:36bf0aaa-ab5a-4781-ab67-875c73e91029 (/v1/_internal_ping) 2020-03-31 09:15:48,283 [550] [DEBUG] [app] Ending request: urn:request:36bf0aaa-ab5a-4781-ab67-875c73e91029 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:48,283 [550] [DEBUG] [app] Ending request: urn:request:36bf0aaa-ab5a-4781-ab67-875c73e91029 (/v1/_internal_ping) 2020-03-31 09:15:48,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:15:48,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:48,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:48,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:48,286 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:15:48,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:48,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:15:48,286 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:48,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:48,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:48,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:48,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:48,303 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:48,303 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:48,320 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:48,320 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:48,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:48,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:48,325 [527] [DEBUG] [app] Starting request: urn:request:83580ca8-b479-4f76-8be8-6d1d008c4817 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:48,325 [527] [DEBUG] [app] Starting request: urn:request:83580ca8-b479-4f76-8be8-6d1d008c4817 (/_internal_ping) 2020-03-31 09:15:48,326 [527] [DEBUG] [app] Ending request: urn:request:83580ca8-b479-4f76-8be8-6d1d008c4817 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:48,326 [527] [DEBUG] [app] Ending request: urn:request:83580ca8-b479-4f76-8be8-6d1d008c4817 (/_internal_ping) 2020-03-31 09:15:48,326 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:15:48,326 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:15:48,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:15:48,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:48,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:15:48,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:48,338 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:15:48,339 [535] [DEBUG] [app] Starting request: urn:request:8872eec6-d5c5-47f1-8b5d-1d19a6be6ff7 (/c1/_internal_ping) 2020-03-31 09:15:48,340 [535] [DEBUG] [app] Ending request: urn:request:8872eec6-d5c5-47f1-8b5d-1d19a6be6ff7 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:15:48,341 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:48,340 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:48,338 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:15:48,339 [535] [DEBUG] [app] Starting request: urn:request:8872eec6-d5c5-47f1-8b5d-1d19a6be6ff7 (/c1/_internal_ping) 2020-03-31 09:15:48,340 [535] [DEBUG] [app] Ending request: urn:request:8872eec6-d5c5-47f1-8b5d-1d19a6be6ff7 (/c1/_internal_ping) 2020-03-31 09:15:48,342 [527] [DEBUG] [app] Ending request: urn:request:0a853314-9474-4648-9336-ca918e4e1c59 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:48,341 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:48,342 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:48,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:15:48,340 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:48,342 [527] [DEBUG] [app] Ending request: urn:request:0a853314-9474-4648-9336-ca918e4e1c59 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:48,342 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:48,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:15:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:15:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:54.323352+00:00 (in 4.823285 seconds) 2020-03-31 09:15:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:49 UTC)" (scheduled at 2020-03-31 09:15:49.498966+00:00) 2020-03-31 09:15:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:15:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:15:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:54.323352+00:00 (in 4.823285 seconds) 2020-03-31 09:15:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:49 UTC)" (scheduled at 2020-03-31 09:15:49.498966+00:00) 2020-03-31 09:15:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:15:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:49 UTC)" executed successfully 2020-03-31 09:15:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:15:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:59.560326+00:00 (in 8.447252 seconds) 2020-03-31 09:15:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:51 UTC)" (scheduled at 2020-03-31 09:15:51.110610+00:00) 2020-03-31 09:15:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:15:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:15:59.560326+00:00 (in 8.447252 seconds) 2020-03-31 09:15:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:51 UTC)" (scheduled at 2020-03-31 09:15:51.110610+00:00) 2020-03-31 09:15:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 51, 114039), True, datetime.datetime(2020, 3, 31, 9, 15, 51, 114039), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:15:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 51, 114039), True, datetime.datetime(2020, 3, 31, 9, 15, 51, 114039), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:15:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:15:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:15:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:51 UTC)" executed successfully 2020-03-31 09:15:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:15:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:01.440707+00:00 (in 9.999320 seconds) 2020-03-31 09:15:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:01 UTC)" (scheduled at 2020-03-31 09:15:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:15:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:01.440707+00:00 (in 9.999320 seconds) 2020-03-31 09:15:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:01 UTC)" (scheduled at 2020-03-31 09:15:51.440707+00:00) 2020-03-31 09:15:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:15:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:15:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:15:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:15:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:15:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:01 UTC)" executed successfully 2020-03-31 09:15:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:15:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:22.113221+00:00 (in 29.999143 seconds) 2020-03-31 09:15:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:22 UTC)" (scheduled at 2020-03-31 09:15:52.113221+00:00) securityworker stdout | 2020-03-31 09:15:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:22.113221+00:00 (in 29.999143 seconds) 2020-03-31 09:15:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:22 UTC)" (scheduled at 2020-03-31 09:15:52.113221+00:00) 2020-03-31 09:15:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:15:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:15:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:15:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:15:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:15:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:15:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:15:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:22 UTC)" executed successfully 2020-03-31 09:15:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:15:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:04.323352+00:00 (in 9.999119 seconds) 2020-03-31 09:15:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:04 UTC)" (scheduled at 2020-03-31 09:15:54.323352+00:00) 2020-03-31 09:15:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:15:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:04.323352+00:00 (in 9.999119 seconds) 2020-03-31 09:15:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:04 UTC)" (scheduled at 2020-03-31 09:15:54.323352+00:00) 2020-03-31 09:15:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 54, 325100), True, datetime.datetime(2020, 3, 31, 9, 15, 54, 325100), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:15:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 54, 325100), True, datetime.datetime(2020, 3, 31, 9, 15, 54, 325100), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:15:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:15:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:15:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:04 UTC)" executed successfully 2020-03-31 09:15:54,446 [518] [DEBUG] [app] Starting request: urn:request:a3435750-8562-4c19-a14f-e437eb756e98 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:54,446 [518] [DEBUG] [app] Starting request: urn:request:a3435750-8562-4c19-a14f-e437eb756e98 (/health/instance) 2020-03-31 09:15:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:54,455 [538] [DEBUG] [app] Starting request: urn:request:35202a2b-66b4-4a20-a91d-b8fe20f24d1e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:54,455 [538] [DEBUG] [app] Starting request: urn:request:35202a2b-66b4-4a20-a91d-b8fe20f24d1e (/v1/_internal_ping) 2020-03-31 09:15:54,456 [538] [DEBUG] [app] Ending request: urn:request:35202a2b-66b4-4a20-a91d-b8fe20f24d1e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:54,456 [538] [DEBUG] [app] Ending request: urn:request:35202a2b-66b4-4a20-a91d-b8fe20f24d1e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:54,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:54,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:15:54,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:54,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:54,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:15:54,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:54,462 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:54,462 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:15:54,464 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:54,464 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:15:54,465 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:54,465 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:54,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:54,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:54,491 [527] [DEBUG] [app] Starting request: urn:request:bf0179c4-479e-415f-9ef4-79406948426e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:54,491 [527] [DEBUG] [app] Starting request: urn:request:bf0179c4-479e-415f-9ef4-79406948426e (/_internal_ping) 2020-03-31 09:15:54,493 [527] [DEBUG] [app] Ending request: urn:request:bf0179c4-479e-415f-9ef4-79406948426e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:54,493 [527] [DEBUG] [app] Ending request: urn:request:bf0179c4-479e-415f-9ef4-79406948426e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:15:54,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:15:54,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:15:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:15:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:54,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:54,508 [541] [DEBUG] [app] Starting request: urn:request:9e381c2b-c0a1-4737-8893-6b9d3bf94bf7 (/c1/_internal_ping) 2020-03-31 09:15:54,509 [541] [DEBUG] [app] Ending request: urn:request:9e381c2b-c0a1-4737-8893-6b9d3bf94bf7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:54,510 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:54,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:54,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:15:54,508 [541] [DEBUG] [app] Starting request: urn:request:9e381c2b-c0a1-4737-8893-6b9d3bf94bf7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:54,509 [541] [DEBUG] [app] Ending request: urn:request:9e381c2b-c0a1-4737-8893-6b9d3bf94bf7 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:54,511 [518] [DEBUG] [app] Ending request: urn:request:a3435750-8562-4c19-a14f-e437eb756e98 (/health/instance) 2020-03-31 09:15:54,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:15:54,510 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:54,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:15:54,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 10.131.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:15:54,511 [518] [DEBUG] [app] Ending request: urn:request:a3435750-8562-4c19-a14f-e437eb756e98 (/health/instance) 2020-03-31 09:15:54,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:54,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:15:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:06.924497+00:00 (in 9.999204 seconds) 2020-03-31 09:15:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:06 UTC)" (scheduled at 2020-03-31 09:15:56.924497+00:00) 2020-03-31 09:15:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:15:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:15:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:06.924497+00:00 (in 9.999204 seconds) 2020-03-31 09:15:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:06 UTC)" (scheduled at 2020-03-31 09:15:56.924497+00:00) 2020-03-31 09:15:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:15:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 56, 926026), True, datetime.datetime(2020, 3, 31, 9, 15, 56, 926026), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:15:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 15, 56, 926026), True, datetime.datetime(2020, 3, 31, 9, 15, 56, 926026), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:15:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:15:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:15:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:15:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:06 UTC)" executed successfully 2020-03-31 09:15:58,272 [527] [DEBUG] [app] Starting request: urn:request:5b87cabd-f632-4965-ae3d-8c9de13bf4f4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:58,272 [527] [DEBUG] [app] Starting request: urn:request:5b87cabd-f632-4965-ae3d-8c9de13bf4f4 (/health/instance) 2020-03-31 09:15:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:58,282 [555] [DEBUG] [app] Starting request: urn:request:5c0ef529-12f9-4e9b-a3aa-3884c2fccbb7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:15:58,282 [555] [DEBUG] [app] Starting request: urn:request:5c0ef529-12f9-4e9b-a3aa-3884c2fccbb7 (/v1/_internal_ping) 2020-03-31 09:15:58,283 [555] [DEBUG] [app] Ending request: urn:request:5c0ef529-12f9-4e9b-a3aa-3884c2fccbb7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:15:58,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:58,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:15:58,283 [555] [DEBUG] [app] Ending request: urn:request:5c0ef529-12f9-4e9b-a3aa-3884c2fccbb7 (/v1/_internal_ping) 2020-03-31 09:15:58,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:15:58,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:58,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:15:58,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:58,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:15:58,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:15:58,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:15:58,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:15:58,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:15:58,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:15:58,300 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:15:58,300 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:15:58,314 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:15:58,314 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:15:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:58,320 [527] [DEBUG] [app] Starting request: urn:request:122737c4-5405-4917-8c27-1f6c07d630f2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:58,320 [527] [DEBUG] [app] Starting request: urn:request:122737c4-5405-4917-8c27-1f6c07d630f2 (/_internal_ping) 2020-03-31 09:15:58,321 [527] [DEBUG] [app] Ending request: urn:request:122737c4-5405-4917-8c27-1f6c07d630f2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:15:58,321 [527] [DEBUG] [app] Ending request: urn:request:122737c4-5405-4917-8c27-1f6c07d630f2 (/_internal_ping) 2020-03-31 09:15:58,322 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:15:58,322 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:15:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:15:58,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:15:58,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:15:58,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:58,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:58,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:15:58,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:15:58,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:15:58,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:15:58,337 [541] [DEBUG] [app] Starting request: urn:request:2ea7d443-0ede-4be0-b703-5b48da2c7600 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:58,337 [541] [DEBUG] [app] Starting request: urn:request:2ea7d443-0ede-4be0-b703-5b48da2c7600 (/c1/_internal_ping) 2020-03-31 09:15:58,339 [541] [DEBUG] [app] Ending request: urn:request:2ea7d443-0ede-4be0-b703-5b48da2c7600 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:15:58,339 [541] [DEBUG] [app] Ending request: urn:request:2ea7d443-0ede-4be0-b703-5b48da2c7600 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:15:58,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:15:58,339 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:15:58,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:15:58,339 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:15:58,342 [527] [DEBUG] [app] Ending request: urn:request:5b87cabd-f632-4965-ae3d-8c9de13bf4f4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:15:58,342 [527] [DEBUG] [app] Ending request: urn:request:5b87cabd-f632-4965-ae3d-8c9de13bf4f4 (/health/instance) 2020-03-31 09:15:58,342 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:15:58,342 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:15:58,343 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:15:58,343 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:15:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:15:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:15:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:15:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:15:59,563 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:59 UTC)" (scheduled at 2020-03-31 09:15:59.560326+00:00) 2020-03-31 09:15:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:51.110610+00:00 (in 51.547882 seconds) 2020-03-31 09:15:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:15:59,563 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:59 UTC)" (scheduled at 2020-03-31 09:15:59.560326+00:00) 2020-03-31 09:15:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:51.110610+00:00 (in 51.547882 seconds) 2020-03-31 09:15:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:15:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:15:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:16:59 UTC)" executed successfully 2020-03-31 09:16:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:16:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:30.337699+00:00 (in 29.997817 seconds) 2020-03-31 09:16:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:30 UTC)" (scheduled at 2020-03-31 09:16:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:16:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:30.337699+00:00 (in 29.997817 seconds) 2020-03-31 09:16:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:30 UTC)" (scheduled at 2020-03-31 09:16:00.337699+00:00) 2020-03-31 09:16:00,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 16, 0, 340739), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:16:00,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 16, 0, 340739), False, 50, 1, 0]) 2020-03-31 09:16:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:16:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:16:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:16:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:30 UTC)" executed successfully 2020-03-31 09:16:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:16:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:11.440707+00:00 (in 9.999267 seconds) 2020-03-31 09:16:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:11 UTC)" (scheduled at 2020-03-31 09:16:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:16:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:11.440707+00:00 (in 9.999267 seconds) 2020-03-31 09:16:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:11 UTC)" (scheduled at 2020-03-31 09:16:01.440707+00:00) 2020-03-31 09:16:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:16:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:16:01,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:16:01,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:16:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:16:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:11 UTC)" executed successfully 2020-03-31 09:16:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:16:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:24.410927+00:00 (in 21.080118 seconds) 2020-03-31 09:16:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:33 UTC)" (scheduled at 2020-03-31 09:16:03.330112+00:00) 2020-03-31 09:16:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:16:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:24.410927+00:00 (in 21.080118 seconds) 2020-03-31 09:16:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:33 UTC)" (scheduled at 2020-03-31 09:16:03.330112+00:00) 2020-03-31 09:16:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 3, 331825), True, datetime.datetime(2020, 3, 31, 9, 16, 3, 331825), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:16:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 3, 331825), True, datetime.datetime(2020, 3, 31, 9, 16, 3, 331825), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:16:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:16:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:16:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:33 UTC)" executed successfully 2020-03-31 09:16:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:16:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:14.323352+00:00 (in 9.999302 seconds) 2020-03-31 09:16:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:14 UTC)" (scheduled at 2020-03-31 09:16:04.323352+00:00) 2020-03-31 09:16:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:16:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:14.323352+00:00 (in 9.999302 seconds) 2020-03-31 09:16:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:14 UTC)" (scheduled at 2020-03-31 09:16:04.323352+00:00) 2020-03-31 09:16:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 4, 324963), True, datetime.datetime(2020, 3, 31, 9, 16, 4, 324963), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:16:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 4, 324963), True, datetime.datetime(2020, 3, 31, 9, 16, 4, 324963), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:16:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:16:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:16:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:14 UTC)" executed successfully 2020-03-31 09:16:04,446 [526] [DEBUG] [app] Starting request: urn:request:b31f01fd-17c1-4c91-b6b0-c266a111233f (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:04,446 [526] [DEBUG] [app] Starting request: urn:request:b31f01fd-17c1-4c91-b6b0-c266a111233f (/health/instance) 2020-03-31 09:16:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:04,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:04,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:04,455 [555] [DEBUG] [app] Starting request: urn:request:63be2359-8d23-40df-b91a-b6f5d1a2d8de (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:04,455 [555] [DEBUG] [app] Starting request: urn:request:63be2359-8d23-40df-b91a-b6f5d1a2d8de (/v1/_internal_ping) 2020-03-31 09:16:04,456 [555] [DEBUG] [app] Ending request: urn:request:63be2359-8d23-40df-b91a-b6f5d1a2d8de (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:04,456 [555] [DEBUG] [app] Ending request: urn:request:63be2359-8d23-40df-b91a-b6f5d1a2d8de (/v1/_internal_ping) 2020-03-31 09:16:04,457 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:04,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:16:04,457 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:04,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:04,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:04,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:04,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:16:04,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:04,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:04,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:04,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:04,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:04,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:04,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:04,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:04,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:04,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:04,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:04,491 [526] [DEBUG] [app] Starting request: urn:request:9d5f7caf-b6b0-4945-92df-bfdab97b2d34 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:04,491 [526] [DEBUG] [app] Starting request: urn:request:9d5f7caf-b6b0-4945-92df-bfdab97b2d34 (/_internal_ping) 2020-03-31 09:16:04,492 [526] [DEBUG] [app] Ending request: urn:request:9d5f7caf-b6b0-4945-92df-bfdab97b2d34 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:04,492 [526] [DEBUG] [app] Ending request: urn:request:9d5f7caf-b6b0-4945-92df-bfdab97b2d34 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:16:04,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:04,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:16:04,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:04,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:04,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:04,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:04,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:16:04,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:16:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:04,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:04,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:04,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:04,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:04,505 [541] [DEBUG] [app] Starting request: urn:request:7bfea980-575a-4636-ba6f-903dad1b5bdc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:04,505 [541] [DEBUG] [app] Starting request: urn:request:7bfea980-575a-4636-ba6f-903dad1b5bdc (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:04,506 [541] [DEBUG] [app] Ending request: urn:request:7bfea980-575a-4636-ba6f-903dad1b5bdc (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:04,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:04,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:16:04,506 [541] [DEBUG] [app] Ending request: urn:request:7bfea980-575a-4636-ba6f-903dad1b5bdc (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:04,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:04,508 [526] [DEBUG] [app] Ending request: urn:request:b31f01fd-17c1-4c91-b6b0-c266a111233f (/health/instance) 2020-03-31 09:16:04,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:16:04,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:04,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:16:04,508 [526] [DEBUG] [app] Ending request: urn:request:b31f01fd-17c1-4c91-b6b0-c266a111233f (/health/instance) 2020-03-31 09:16:04,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:04,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:16:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:16:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:16.924497+00:00 (in 9.999182 seconds) 2020-03-31 09:16:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:16 UTC)" (scheduled at 2020-03-31 09:16:06.924497+00:00) 2020-03-31 09:16:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 6, 926236), True, datetime.datetime(2020, 3, 31, 9, 16, 6, 926236), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:16:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:16.924497+00:00 (in 9.999182 seconds) 2020-03-31 09:16:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:16 UTC)" (scheduled at 2020-03-31 09:16:06.924497+00:00) 2020-03-31 09:16:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:16:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 6, 926236), True, datetime.datetime(2020, 3, 31, 9, 16, 6, 926236), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:16:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:16:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:16:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:16 UTC)" executed successfully 2020-03-31 09:16:08,273 [518] [DEBUG] [app] Starting request: urn:request:ec6fcde1-ffd6-4f72-8349-4bcb5ef8c073 (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:08,273 [518] [DEBUG] [app] Starting request: urn:request:ec6fcde1-ffd6-4f72-8349-4bcb5ef8c073 (/health/instance) 2020-03-31 09:16:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:08,280 [556] [DEBUG] [app] Starting request: urn:request:da9193f2-558a-425b-97a0-b262170b526e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:08,280 [556] [DEBUG] [app] Starting request: urn:request:da9193f2-558a-425b-97a0-b262170b526e (/v1/_internal_ping) 2020-03-31 09:16:08,282 [556] [DEBUG] [app] Ending request: urn:request:da9193f2-558a-425b-97a0-b262170b526e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:08,282 [556] [DEBUG] [app] Ending request: urn:request:da9193f2-558a-425b-97a0-b262170b526e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:16:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:08,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:16:08,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:08,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:08,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:08,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:08,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:08,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:08,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:08,316 [518] [DEBUG] [app] Starting request: urn:request:aa9efcff-5d2e-49bb-a2d7-040d7ae1725e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:08,316 [518] [DEBUG] [app] Starting request: urn:request:aa9efcff-5d2e-49bb-a2d7-040d7ae1725e (/_internal_ping) 2020-03-31 09:16:08,317 [518] [DEBUG] [app] Ending request: urn:request:aa9efcff-5d2e-49bb-a2d7-040d7ae1725e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:08,317 [518] [DEBUG] [app] Ending request: urn:request:aa9efcff-5d2e-49bb-a2d7-040d7ae1725e (/_internal_ping) 2020-03-31 09:16:08,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:08,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:08,330 [543] [DEBUG] [app] Starting request: urn:request:fcd34107-cce1-4556-8e47-6e02ce012788 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:08,330 [543] [DEBUG] [app] Starting request: urn:request:fcd34107-cce1-4556-8e47-6e02ce012788 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:08,331 [543] [DEBUG] [app] Ending request: urn:request:fcd34107-cce1-4556-8e47-6e02ce012788 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:08,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:16:08,331 [543] [DEBUG] [app] Ending request: urn:request:fcd34107-cce1-4556-8e47-6e02ce012788 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:08,333 [518] [DEBUG] [app] Ending request: urn:request:ec6fcde1-ffd6-4f72-8349-4bcb5ef8c073 (/health/instance) 2020-03-31 09:16:08,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:16:08,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:16:08,333 [518] [DEBUG] [app] Ending request: urn:request:ec6fcde1-ffd6-4f72-8349-4bcb5ef8c073 (/health/instance) 2020-03-31 09:16:08,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:16:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:16:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:21.440707+00:00 (in 9.999186 seconds) 2020-03-31 09:16:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:21 UTC)" (scheduled at 2020-03-31 09:16:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:16:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:21.440707+00:00 (in 9.999186 seconds) 2020-03-31 09:16:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:21 UTC)" (scheduled at 2020-03-31 09:16:11.440707+00:00) 2020-03-31 09:16:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:16:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:16:11,456 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:16:11,456 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:16:11,456 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:11,456 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:16:11,456 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:11,456 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:21 UTC)" executed successfully 2020-03-31 09:16:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:16:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:24.323352+00:00 (in 9.999059 seconds) 2020-03-31 09:16:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:24 UTC)" (scheduled at 2020-03-31 09:16:14.323352+00:00) 2020-03-31 09:16:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:16:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:24.323352+00:00 (in 9.999059 seconds) 2020-03-31 09:16:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:24 UTC)" (scheduled at 2020-03-31 09:16:14.323352+00:00) 2020-03-31 09:16:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 14, 325374), True, datetime.datetime(2020, 3, 31, 9, 16, 14, 325374), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:16:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 14, 325374), True, datetime.datetime(2020, 3, 31, 9, 16, 14, 325374), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:16:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:16:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:16:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:24 UTC)" executed successfully 2020-03-31 09:16:14,445 [518] [DEBUG] [app] Starting request: urn:request:a62752a6-2626-467a-bcea-3fb1de4cf0e9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:14,445 [518] [DEBUG] [app] Starting request: urn:request:a62752a6-2626-467a-bcea-3fb1de4cf0e9 (/health/instance) 2020-03-31 09:16:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:14,454 [538] [DEBUG] [app] Starting request: urn:request:e73cefd5-9829-46bd-8ad5-1695cdd1570b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:14,454 [538] [DEBUG] [app] Starting request: urn:request:e73cefd5-9829-46bd-8ad5-1695cdd1570b (/v1/_internal_ping) 2020-03-31 09:16:14,455 [538] [DEBUG] [app] Ending request: urn:request:e73cefd5-9829-46bd-8ad5-1695cdd1570b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:14,455 [538] [DEBUG] [app] Ending request: urn:request:e73cefd5-9829-46bd-8ad5-1695cdd1570b (/v1/_internal_ping) 2020-03-31 09:16:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:16:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:16:14,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:16:14,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:14,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:14,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:14,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:14,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:14,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:14,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:14,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:14,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:14,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:14,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:14,492 [518] [DEBUG] [app] Starting request: urn:request:3a85d790-d045-421b-ba5b-6f08a43380c8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:14,492 [518] [DEBUG] [app] Starting request: urn:request:3a85d790-d045-421b-ba5b-6f08a43380c8 (/_internal_ping) 2020-03-31 09:16:14,493 [518] [DEBUG] [app] Ending request: urn:request:3a85d790-d045-421b-ba5b-6f08a43380c8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:14,493 [518] [DEBUG] [app] Ending request: urn:request:3a85d790-d045-421b-ba5b-6f08a43380c8 (/_internal_ping) 2020-03-31 09:16:14,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:14,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:16:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:16:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:14,506 [541] [DEBUG] [app] Starting request: urn:request:149b9b71-ce5e-4e8c-926b-17db80f3d2d8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:14,506 [541] [DEBUG] [app] Starting request: urn:request:149b9b71-ce5e-4e8c-926b-17db80f3d2d8 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:14,507 [541] [DEBUG] [app] Ending request: urn:request:149b9b71-ce5e-4e8c-926b-17db80f3d2d8 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:14,507 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:16:14,507 [541] [DEBUG] [app] Ending request: urn:request:149b9b71-ce5e-4e8c-926b-17db80f3d2d8 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:14,509 [518] [DEBUG] [app] Ending request: urn:request:a62752a6-2626-467a-bcea-3fb1de4cf0e9 (/health/instance) 2020-03-31 09:16:14,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:16:14,507 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:14,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:16:14,509 [518] [DEBUG] [app] Ending request: urn:request:a62752a6-2626-467a-bcea-3fb1de4cf0e9 (/health/instance) 2020-03-31 09:16:14,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:14,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:16:15,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:16:15,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:45.064514+00:00 (in 29.997860 seconds) 2020-03-31 09:16:15,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:45 UTC)" (scheduled at 2020-03-31 09:16:15.064514+00:00) gcworker stdout | 2020-03-31 09:16:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:45.064514+00:00 (in 29.997860 seconds) 2020-03-31 09:16:15,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:45 UTC)" (scheduled at 2020-03-31 09:16:15.064514+00:00) 2020-03-31 09:16:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436575, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:16:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436575, 1209600, 500, 1, 0]) 2020-03-31 09:16:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:16:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:16:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:16:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:45 UTC)" executed successfully 2020-03-31 09:16:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:16:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:26.924497+00:00 (in 9.999104 seconds) 2020-03-31 09:16:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:26 UTC)" (scheduled at 2020-03-31 09:16:16.924497+00:00) 2020-03-31 09:16:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:16:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:26.924497+00:00 (in 9.999104 seconds) 2020-03-31 09:16:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:26 UTC)" (scheduled at 2020-03-31 09:16:16.924497+00:00) 2020-03-31 09:16:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 16, 926117), True, datetime.datetime(2020, 3, 31, 9, 16, 16, 926117), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:16:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 16, 926117), True, datetime.datetime(2020, 3, 31, 9, 16, 16, 926117), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:16:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:16:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:16:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:26 UTC)" executed successfully 2020-03-31 09:16:18,273 [518] [DEBUG] [app] Starting request: urn:request:a3e01025-9512-44f9-bf2d-db1e5afa6f1a (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:18,273 [518] [DEBUG] [app] Starting request: urn:request:a3e01025-9512-44f9-bf2d-db1e5afa6f1a (/health/instance) 2020-03-31 09:16:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:18,277 [554] [DEBUG] [app] Starting request: urn:request:091396ae-c3b6-4b4b-a1cb-582f9eb2100e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:18,277 [554] [DEBUG] [app] Starting request: urn:request:091396ae-c3b6-4b4b-a1cb-582f9eb2100e (/v1/_internal_ping) 2020-03-31 09:16:18,278 [554] [DEBUG] [app] Ending request: urn:request:091396ae-c3b6-4b4b-a1cb-582f9eb2100e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:18,278 [554] [DEBUG] [app] Ending request: urn:request:091396ae-c3b6-4b4b-a1cb-582f9eb2100e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:18,279 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:16:18,279 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:16:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:16:18,282 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:16:18,282 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:18,283 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:18,283 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:18,284 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:18,284 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:18,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:18,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:18,311 [518] [DEBUG] [app] Starting request: urn:request:0f72576a-034a-41e4-818a-66fc78f5f2e4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:18,311 [518] [DEBUG] [app] Starting request: urn:request:0f72576a-034a-41e4-818a-66fc78f5f2e4 (/_internal_ping) 2020-03-31 09:16:18,312 [518] [DEBUG] [app] Ending request: urn:request:0f72576a-034a-41e4-818a-66fc78f5f2e4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:18,312 [518] [DEBUG] [app] Ending request: urn:request:0f72576a-034a-41e4-818a-66fc78f5f2e4 (/_internal_ping) 2020-03-31 09:16:18,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:16:18,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:16:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:18,327 [535] [DEBUG] [app] Starting request: urn:request:f699705a-8f94-47f3-be97-28dba40693bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:18,327 [535] [DEBUG] [app] Starting request: urn:request:f699705a-8f94-47f3-be97-28dba40693bd (/c1/_internal_ping) 2020-03-31 09:16:18,328 [535] [DEBUG] [app] Ending request: urn:request:f699705a-8f94-47f3-be97-28dba40693bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:18,328 [535] [DEBUG] [app] Ending request: urn:request:f699705a-8f94-47f3-be97-28dba40693bd (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:18,329 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:16:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:16:18,329 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:18,330 [518] [DEBUG] [app] Ending request: urn:request:a3e01025-9512-44f9-bf2d-db1e5afa6f1a (/health/instance) 2020-03-31 09:16:18,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:16:18,330 [518] [DEBUG] [app] Ending request: urn:request:a3e01025-9512-44f9-bf2d-db1e5afa6f1a (/health/instance) 2020-03-31 09:16:18,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:16:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:16:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:21,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:31 UTC)" (scheduled at 2020-03-31 09:16:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:16:21,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:31 UTC)" (scheduled at 2020-03-31 09:16:21.440707+00:00) 2020-03-31 09:16:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:16:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:16:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:31.440707+00:00 (in 9.999254 seconds) repositoryactioncounter stdout | 2020-03-31 09:16:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:31.440707+00:00 (in 9.999254 seconds) 2020-03-31 09:16:21,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:16:21,459 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:16:21,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:16:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:16:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:31 UTC)" executed successfully 2020-03-31 09:16:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:52.113221+00:00 (in 29.999129 seconds) 2020-03-31 09:16:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:52 UTC)" (scheduled at 2020-03-31 09:16:22.113221+00:00) 2020-03-31 09:16:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:16:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:16:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:52.113221+00:00 (in 29.999129 seconds) 2020-03-31 09:16:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:52 UTC)" (scheduled at 2020-03-31 09:16:22.113221+00:00) securityworker stdout | 2020-03-31 09:16:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:16:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:16:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:16:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:16:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:16:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:16:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:16:52 UTC)" executed successfully 2020-03-31 09:16:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:16:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:34.323352+00:00 (in 9.998838 seconds) 2020-03-31 09:16:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:34 UTC)" (scheduled at 2020-03-31 09:16:24.323352+00:00) 2020-03-31 09:16:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:16:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:34.323352+00:00 (in 9.998838 seconds) 2020-03-31 09:16:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:34 UTC)" (scheduled at 2020-03-31 09:16:24.323352+00:00) 2020-03-31 09:16:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 24, 325244), True, datetime.datetime(2020, 3, 31, 9, 16, 24, 325244), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:16:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 24, 325244), True, datetime.datetime(2020, 3, 31, 9, 16, 24, 325244), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:16:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:16:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:16:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:34 UTC)" executed successfully 2020-03-31 09:16:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:16:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:33.330112+00:00 (in 8.917843 seconds) 2020-03-31 09:16:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:24 UTC)" (scheduled at 2020-03-31 09:16:24.410927+00:00) 2020-03-31 09:16:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:16:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:33.330112+00:00 (in 8.917843 seconds) 2020-03-31 09:16:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:24 UTC)" (scheduled at 2020-03-31 09:16:24.410927+00:00) 2020-03-31 09:16:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:24 UTC)" executed successfully 2020-03-31 09:16:24,446 [518] [DEBUG] [app] Starting request: urn:request:a585b8d3-cc89-44f7-82fa-d201e8d761b9 (/health/instance) 2020-03-31 09:16:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:24,446 [518] [DEBUG] [app] Starting request: urn:request:a585b8d3-cc89-44f7-82fa-d201e8d761b9 (/health/instance) 2020-03-31 09:16:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:24,456 [555] [DEBUG] [app] Starting request: urn:request:002b3415-a9e3-4dd3-ae37-e73f5e3f1b10 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:24,456 [555] [DEBUG] [app] Starting request: urn:request:002b3415-a9e3-4dd3-ae37-e73f5e3f1b10 (/v1/_internal_ping) 2020-03-31 09:16:24,457 [555] [DEBUG] [app] Ending request: urn:request:002b3415-a9e3-4dd3-ae37-e73f5e3f1b10 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:24,457 [555] [DEBUG] [app] Ending request: urn:request:002b3415-a9e3-4dd3-ae37-e73f5e3f1b10 (/v1/_internal_ping) 2020-03-31 09:16:24,458 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:16:24,458 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:16:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:16:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:24,461 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:24,461 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:16:24,461 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:16:24,461 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:24,462 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:24,462 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:24,463 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:24,463 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:24,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:24,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:24,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:24,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:24,493 [518] [DEBUG] [app] Starting request: urn:request:d0441a6a-7361-4efb-bbce-f582e6f157c0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:24,493 [518] [DEBUG] [app] Starting request: urn:request:d0441a6a-7361-4efb-bbce-f582e6f157c0 (/_internal_ping) 2020-03-31 09:16:24,493 [518] [DEBUG] [app] Ending request: urn:request:d0441a6a-7361-4efb-bbce-f582e6f157c0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:24,493 [518] [DEBUG] [app] Ending request: urn:request:d0441a6a-7361-4efb-bbce-f582e6f157c0 (/_internal_ping) 2020-03-31 09:16:24,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:16:24,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:16:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:24,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:16:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:24,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:24,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:24,507 [535] [DEBUG] [app] Starting request: urn:request:0d017a5e-dee5-4668-9a17-aa14ef4baeb4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:24,507 [535] [DEBUG] [app] Starting request: urn:request:0d017a5e-dee5-4668-9a17-aa14ef4baeb4 (/c1/_internal_ping) 2020-03-31 09:16:24,508 [535] [DEBUG] [app] Ending request: urn:request:0d017a5e-dee5-4668-9a17-aa14ef4baeb4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:24,508 [535] [DEBUG] [app] Ending request: urn:request:0d017a5e-dee5-4668-9a17-aa14ef4baeb4 (/c1/_internal_ping) 2020-03-31 09:16:24,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:24,509 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:16:24,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:24,511 [518] [DEBUG] [app] Ending request: urn:request:a585b8d3-cc89-44f7-82fa-d201e8d761b9 (/health/instance) 2020-03-31 09:16:24,511 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:16:24,509 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:24,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:16:24,511 [518] [DEBUG] [app] Ending request: urn:request:a585b8d3-cc89-44f7-82fa-d201e8d761b9 (/health/instance) 2020-03-31 09:16:24,511 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:24,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:16:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:16:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:36.924497+00:00 (in 9.999221 seconds) 2020-03-31 09:16:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:36 UTC)" (scheduled at 2020-03-31 09:16:26.924497+00:00) 2020-03-31 09:16:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:16:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:36.924497+00:00 (in 9.999221 seconds) 2020-03-31 09:16:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:36 UTC)" (scheduled at 2020-03-31 09:16:26.924497+00:00) 2020-03-31 09:16:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 26, 925956), True, datetime.datetime(2020, 3, 31, 9, 16, 26, 925956), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:16:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 26, 925956), True, datetime.datetime(2020, 3, 31, 9, 16, 26, 925956), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:16:26,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:16:26,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:16:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:36 UTC)" executed successfully 2020-03-31 09:16:28,274 [518] [DEBUG] [app] Starting request: urn:request:ba1d0870-86b6-4fd8-8e5c-e5800260474e (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:28,274 [518] [DEBUG] [app] Starting request: urn:request:ba1d0870-86b6-4fd8-8e5c-e5800260474e (/health/instance) 2020-03-31 09:16:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:28,279 [556] [DEBUG] [app] Starting request: urn:request:29fd9c7a-c982-4403-a426-ae1f4398e5db (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:28,279 [556] [DEBUG] [app] Starting request: urn:request:29fd9c7a-c982-4403-a426-ae1f4398e5db (/v1/_internal_ping) 2020-03-31 09:16:28,281 [556] [DEBUG] [app] Ending request: urn:request:29fd9c7a-c982-4403-a426-ae1f4398e5db (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:28,281 [556] [DEBUG] [app] Ending request: urn:request:29fd9c7a-c982-4403-a426-ae1f4398e5db (/v1/_internal_ping) 2020-03-31 09:16:28,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:16:28,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:16:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:16:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:16:28,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:28,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:28,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:28,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:28,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:28,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:28,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:28,315 [518] [DEBUG] [app] Starting request: urn:request:05fa5803-a98d-479f-b8ce-9c48138b1a9a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:28,315 [518] [DEBUG] [app] Starting request: urn:request:05fa5803-a98d-479f-b8ce-9c48138b1a9a (/_internal_ping) 2020-03-31 09:16:28,316 [518] [DEBUG] [app] Ending request: urn:request:05fa5803-a98d-479f-b8ce-9c48138b1a9a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:28,316 [518] [DEBUG] [app] Ending request: urn:request:05fa5803-a98d-479f-b8ce-9c48138b1a9a (/_internal_ping) 2020-03-31 09:16:28,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:16:28,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:28,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:16:28,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:28,333 [541] [DEBUG] [app] Starting request: urn:request:9528dc8e-040b-42de-8a56-c68b7b243a86 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:28,333 [541] [DEBUG] [app] Starting request: urn:request:9528dc8e-040b-42de-8a56-c68b7b243a86 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:28,334 [541] [DEBUG] [app] Ending request: urn:request:9528dc8e-040b-42de-8a56-c68b7b243a86 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:28,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:28,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:16:28,334 [541] [DEBUG] [app] Ending request: urn:request:9528dc8e-040b-42de-8a56-c68b7b243a86 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:28,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:28,337 [518] [DEBUG] [app] Ending request: urn:request:ba1d0870-86b6-4fd8-8e5c-e5800260474e (/health/instance) 2020-03-31 09:16:28,337 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:16:28,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:28,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:16:28,337 [518] [DEBUG] [app] Ending request: urn:request:ba1d0870-86b6-4fd8-8e5c-e5800260474e (/health/instance) 2020-03-31 09:16:28,337 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:28,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:16:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:16:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:00.337699+00:00 (in 29.999243 seconds) 2020-03-31 09:16:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:00 UTC)" (scheduled at 2020-03-31 09:16:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:16:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:00.337699+00:00 (in 29.999243 seconds) 2020-03-31 09:16:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:00 UTC)" (scheduled at 2020-03-31 09:16:30.337699+00:00) 2020-03-31 09:16:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 16, 30, 339130), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:16:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 16, 30, 339130), False, 50, 1, 0]) 2020-03-31 09:16:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:16:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:16:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:16:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:00 UTC)" executed successfully 2020-03-31 09:16:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:16:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:41.440707+00:00 (in 9.996067 seconds) 2020-03-31 09:16:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:41 UTC)" (scheduled at 2020-03-31 09:16:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:16:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:41.440707+00:00 (in 9.996067 seconds) 2020-03-31 09:16:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:41 UTC)" (scheduled at 2020-03-31 09:16:31.440707+00:00) 2020-03-31 09:16:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:16:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:16:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:16:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:16:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:16:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:41 UTC)" executed successfully 2020-03-31 09:16:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:16:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:03.330112+00:00 (in 29.998990 seconds) 2020-03-31 09:16:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:03 UTC)" (scheduled at 2020-03-31 09:16:33.330112+00:00) 2020-03-31 09:16:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:16:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:03.330112+00:00 (in 29.998990 seconds) 2020-03-31 09:16:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:03 UTC)" (scheduled at 2020-03-31 09:16:33.330112+00:00) 2020-03-31 09:16:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 33, 332113), True, datetime.datetime(2020, 3, 31, 9, 16, 33, 332113), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:16:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 33, 332113), True, datetime.datetime(2020, 3, 31, 9, 16, 33, 332113), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:16:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:16:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:16:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:03 UTC)" executed successfully 2020-03-31 09:16:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:16:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:43.539999+00:00 (in 10.172111 seconds) 2020-03-31 09:16:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:33 UTC)" (scheduled at 2020-03-31 09:16:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:16:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:43.539999+00:00 (in 10.172111 seconds) 2020-03-31 09:16:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:33 UTC)" (scheduled at 2020-03-31 09:16:33.366791+00:00) 2020-03-31 09:16:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:16:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 33, 368968), True, datetime.datetime(2020, 3, 31, 9, 16, 33, 368968), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:16:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 33, 368968), True, datetime.datetime(2020, 3, 31, 9, 16, 33, 368968), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:16:33,386 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:16:33,386 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:16:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:33 UTC)" executed successfully 2020-03-31 09:16:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:16:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:44.323352+00:00 (in 9.999280 seconds) 2020-03-31 09:16:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:44 UTC)" (scheduled at 2020-03-31 09:16:34.323352+00:00) 2020-03-31 09:16:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:16:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:44.323352+00:00 (in 9.999280 seconds) 2020-03-31 09:16:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:44 UTC)" (scheduled at 2020-03-31 09:16:34.323352+00:00) 2020-03-31 09:16:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 34, 325134), True, datetime.datetime(2020, 3, 31, 9, 16, 34, 325134), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:16:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 34, 325134), True, datetime.datetime(2020, 3, 31, 9, 16, 34, 325134), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:16:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:16:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:16:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:44 UTC)" executed successfully 2020-03-31 09:16:34,446 [526] [DEBUG] [app] Starting request: urn:request:97c51837-5202-4bd2-bc5e-b2461c38bc53 (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:34,446 [526] [DEBUG] [app] Starting request: urn:request:97c51837-5202-4bd2-bc5e-b2461c38bc53 (/health/instance) 2020-03-31 09:16:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:34,453 [556] [DEBUG] [app] Starting request: urn:request:c0cc7243-a529-41fd-8599-806395c14009 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:34,453 [556] [DEBUG] [app] Starting request: urn:request:c0cc7243-a529-41fd-8599-806395c14009 (/v1/_internal_ping) 2020-03-31 09:16:34,454 [556] [DEBUG] [app] Ending request: urn:request:c0cc7243-a529-41fd-8599-806395c14009 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:34,454 [556] [DEBUG] [app] Ending request: urn:request:c0cc7243-a529-41fd-8599-806395c14009 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:34,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:16:34,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:34,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:16:34,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:34,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:34,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:34,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:34,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:34,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:34,469 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:34,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:34,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:34,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:34,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:34,487 [521] [DEBUG] [app] Starting request: urn:request:c270781c-b264-40a4-bd06-b7c33d563072 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:34,487 [521] [DEBUG] [app] Starting request: urn:request:c270781c-b264-40a4-bd06-b7c33d563072 (/_internal_ping) 2020-03-31 09:16:34,488 [521] [DEBUG] [app] Ending request: urn:request:c270781c-b264-40a4-bd06-b7c33d563072 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:34,488 [521] [DEBUG] [app] Ending request: urn:request:c270781c-b264-40a4-bd06-b7c33d563072 (/_internal_ping) 2020-03-31 09:16:34,488 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:34,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:34,488 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:34,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:16:34,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:34,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:34,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:34,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:16:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:34,502 [541] [DEBUG] [app] Starting request: urn:request:b7ae705a-24a7-42e7-af15-91a2be3c5126 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:34,502 [541] [DEBUG] [app] Starting request: urn:request:b7ae705a-24a7-42e7-af15-91a2be3c5126 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:34,503 [541] [DEBUG] [app] Ending request: urn:request:b7ae705a-24a7-42e7-af15-91a2be3c5126 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:34,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:34,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:16:34,503 [541] [DEBUG] [app] Ending request: urn:request:b7ae705a-24a7-42e7-af15-91a2be3c5126 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:34,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:16:34,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:34,505 [526] [DEBUG] [app] Ending request: urn:request:97c51837-5202-4bd2-bc5e-b2461c38bc53 (/health/instance) 2020-03-31 09:16:34,506 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:34,505 [526] [DEBUG] [app] Ending request: urn:request:97c51837-5202-4bd2-bc5e-b2461c38bc53 (/health/instance) 2020-03-31 09:16:34,506 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:34,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:16:34,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:16:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:38.010934+00:00 (in 1.085676 seconds) 2020-03-31 09:16:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:46 UTC)" (scheduled at 2020-03-31 09:16:36.924497+00:00) 2020-03-31 09:16:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:16:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:16:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:38.010934+00:00 (in 1.085676 seconds) 2020-03-31 09:16:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:46 UTC)" (scheduled at 2020-03-31 09:16:36.924497+00:00) 2020-03-31 09:16:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:36,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 36, 926148), True, datetime.datetime(2020, 3, 31, 9, 16, 36, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:16:36,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 36, 926148), True, datetime.datetime(2020, 3, 31, 9, 16, 36, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:16:36,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:16:36,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:36,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:16:36,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:46 UTC)" executed successfully 2020-03-31 09:16:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:16:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:46.924497+00:00 (in 8.912808 seconds) 2020-03-31 09:16:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:38 UTC)" (scheduled at 2020-03-31 09:16:38.010934+00:00) 2020-03-31 09:16:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:16:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:46.924497+00:00 (in 8.912808 seconds) 2020-03-31 09:16:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:38 UTC)" (scheduled at 2020-03-31 09:16:38.010934+00:00) 2020-03-31 09:16:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:38 UTC)" executed successfully 2020-03-31 09:16:38,273 [526] [DEBUG] [app] Starting request: urn:request:fedc1cbe-b9a3-482c-b1e7-eec8b342dade (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:38,273 [526] [DEBUG] [app] Starting request: urn:request:fedc1cbe-b9a3-482c-b1e7-eec8b342dade (/health/instance) 2020-03-31 09:16:38,276 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:38,276 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:38,278 [556] [DEBUG] [app] Starting request: urn:request:16f235d5-0db1-4141-b61e-0855c101bf6b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:38,278 [556] [DEBUG] [app] Starting request: urn:request:16f235d5-0db1-4141-b61e-0855c101bf6b (/v1/_internal_ping) 2020-03-31 09:16:38,279 [556] [DEBUG] [app] Ending request: urn:request:16f235d5-0db1-4141-b61e-0855c101bf6b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:38,279 [556] [DEBUG] [app] Ending request: urn:request:16f235d5-0db1-4141-b61e-0855c101bf6b (/v1/_internal_ping) 2020-03-31 09:16:38,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:16:38,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:38,279 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:38,280 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:38,279 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:38,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:38,280 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:16:38,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:38,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:38,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:38,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:38,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:38,305 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:38,305 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:38,309 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:38,309 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:38,310 [526] [DEBUG] [app] Starting request: urn:request:70353cde-0de8-4345-8fbb-0144a11056c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:38,310 [526] [DEBUG] [app] Starting request: urn:request:70353cde-0de8-4345-8fbb-0144a11056c2 (/_internal_ping) 2020-03-31 09:16:38,311 [526] [DEBUG] [app] Ending request: urn:request:70353cde-0de8-4345-8fbb-0144a11056c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:38,311 [526] [DEBUG] [app] Ending request: urn:request:70353cde-0de8-4345-8fbb-0144a11056c2 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:16:38,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:38,312 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:16:38,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:38,312 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:38,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:38,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:16:38,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:16:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:38,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:38,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:16:38,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:38,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:38,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:38,325 [541] [DEBUG] [app] Starting request: urn:request:f120c351-8c14-44a2-ad2a-2682b83316a3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:38,325 [541] [DEBUG] [app] Starting request: urn:request:f120c351-8c14-44a2-ad2a-2682b83316a3 (/c1/_internal_ping) 2020-03-31 09:16:38,326 [541] [DEBUG] [app] Ending request: urn:request:f120c351-8c14-44a2-ad2a-2682b83316a3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:38,326 [541] [DEBUG] [app] Ending request: urn:request:f120c351-8c14-44a2-ad2a-2682b83316a3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:38,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:38,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:16:38,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:16:38,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:38,328 [526] [DEBUG] [app] Ending request: urn:request:fedc1cbe-b9a3-482c-b1e7-eec8b342dade (/health/instance) 2020-03-31 09:16:38,328 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:38,328 [526] [DEBUG] [app] Ending request: urn:request:fedc1cbe-b9a3-482c-b1e7-eec8b342dade (/health/instance) 2020-03-31 09:16:38,328 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:38,329 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:16:38,329 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:16:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:16:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:51.440707+00:00 (in 9.999298 seconds) 2020-03-31 09:16:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:51 UTC)" (scheduled at 2020-03-31 09:16:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:16:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:51.440707+00:00 (in 9.999298 seconds) 2020-03-31 09:16:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:51 UTC)" (scheduled at 2020-03-31 09:16:41.440707+00:00) 2020-03-31 09:16:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:16:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:16:41,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:16:41,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:16:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:41,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:16:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:41,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:51 UTC)" executed successfully 2020-03-31 09:16:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:16:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:33.366791+00:00 (in 49.825688 seconds) 2020-03-31 09:16:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:43 UTC)" (scheduled at 2020-03-31 09:16:43.539999+00:00) 2020-03-31 09:16:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:16:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:33.366791+00:00 (in 49.825688 seconds) 2020-03-31 09:16:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:43 UTC)" (scheduled at 2020-03-31 09:16:43.539999+00:00) 2020-03-31 09:16:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:43 UTC)" executed successfully 2020-03-31 09:16:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:16:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:49.498966+00:00 (in 5.174409 seconds) 2020-03-31 09:16:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:54 UTC)" (scheduled at 2020-03-31 09:16:44.323352+00:00) 2020-03-31 09:16:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:16:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:49.498966+00:00 (in 5.174409 seconds) 2020-03-31 09:16:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:54 UTC)" (scheduled at 2020-03-31 09:16:44.323352+00:00) 2020-03-31 09:16:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 44, 325286), True, datetime.datetime(2020, 3, 31, 9, 16, 44, 325286), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:16:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 44, 325286), True, datetime.datetime(2020, 3, 31, 9, 16, 44, 325286), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:16:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:16:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:16:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:54 UTC)" executed successfully 2020-03-31 09:16:44,445 [518] [DEBUG] [app] Starting request: urn:request:8888fe95-8148-40b1-bc0e-c02cc67d533a (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:44,445 [518] [DEBUG] [app] Starting request: urn:request:8888fe95-8148-40b1-bc0e-c02cc67d533a (/health/instance) 2020-03-31 09:16:44,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:44,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:44,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:44,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:44,454 [538] [DEBUG] [app] Starting request: urn:request:470fa287-0da9-41b3-a0c4-1fdc33a8c30a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:44,454 [538] [DEBUG] [app] Starting request: urn:request:470fa287-0da9-41b3-a0c4-1fdc33a8c30a (/v1/_internal_ping) 2020-03-31 09:16:44,455 [538] [DEBUG] [app] Ending request: urn:request:470fa287-0da9-41b3-a0c4-1fdc33a8c30a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:44,455 [538] [DEBUG] [app] Ending request: urn:request:470fa287-0da9-41b3-a0c4-1fdc33a8c30a (/v1/_internal_ping) 2020-03-31 09:16:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:16:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:16:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:16:44,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:16:44,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:44,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:44,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:44,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:44,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:44,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:44,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:44,493 [518] [DEBUG] [app] Starting request: urn:request:668c7eda-b5a4-4fbc-affe-a9e82a720caf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:44,493 [518] [DEBUG] [app] Starting request: urn:request:668c7eda-b5a4-4fbc-affe-a9e82a720caf (/_internal_ping) 2020-03-31 09:16:44,494 [518] [DEBUG] [app] Ending request: urn:request:668c7eda-b5a4-4fbc-affe-a9e82a720caf (/_internal_ping) 2020-03-31 09:16:44,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:16:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:16:44,494 [518] [DEBUG] [app] Ending request: urn:request:668c7eda-b5a4-4fbc-affe-a9e82a720caf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:44,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:16:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:16:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:16:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:16:44,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:44,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:44,509 [536] [DEBUG] [app] Starting request: urn:request:b8dd7855-22f5-4baf-acc5-00a446ffe6f8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:44,509 [536] [DEBUG] [app] Starting request: urn:request:b8dd7855-22f5-4baf-acc5-00a446ffe6f8 (/c1/_internal_ping) 2020-03-31 09:16:44,510 [536] [DEBUG] [app] Ending request: urn:request:b8dd7855-22f5-4baf-acc5-00a446ffe6f8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:44,510 [536] [DEBUG] [app] Ending request: urn:request:b8dd7855-22f5-4baf-acc5-00a446ffe6f8 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:44,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:44,510 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:16:44,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:16:44,510 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:44,512 [518] [DEBUG] [app] Ending request: urn:request:8888fe95-8148-40b1-bc0e-c02cc67d533a (/health/instance) 2020-03-31 09:16:44,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:44,512 [518] [DEBUG] [app] Ending request: urn:request:8888fe95-8148-40b1-bc0e-c02cc67d533a (/health/instance) 2020-03-31 09:16:44,512 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:44,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:16:44,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:16:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:16:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:15.064514+00:00 (in 29.995914 seconds) 2020-03-31 09:16:45,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:15 UTC)" (scheduled at 2020-03-31 09:16:45.064514+00:00) gcworker stdout | 2020-03-31 09:16:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:15.064514+00:00 (in 29.995914 seconds) 2020-03-31 09:16:45,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:15 UTC)" (scheduled at 2020-03-31 09:16:45.064514+00:00) 2020-03-31 09:16:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436605, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:16:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436605, 1209600, 500, 1, 0]) 2020-03-31 09:16:45,088 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:16:45,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:16:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:16:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:15 UTC)" executed successfully 2020-03-31 09:16:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:16:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:56.924497+00:00 (in 9.999281 seconds) 2020-03-31 09:16:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:56 UTC)" (scheduled at 2020-03-31 09:16:46.924497+00:00) 2020-03-31 09:16:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:16:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:56.924497+00:00 (in 9.999281 seconds) 2020-03-31 09:16:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:56 UTC)" (scheduled at 2020-03-31 09:16:46.924497+00:00) 2020-03-31 09:16:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 46, 925873), True, datetime.datetime(2020, 3, 31, 9, 16, 46, 925873), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:16:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 46, 925873), True, datetime.datetime(2020, 3, 31, 9, 16, 46, 925873), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:16:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:16:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:16:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:16:56 UTC)" executed successfully 2020-03-31 09:16:48,272 [518] [DEBUG] [app] Starting request: urn:request:7252b3f5-bb09-4a69-9afb-161ee7075385 (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:48,272 [518] [DEBUG] [app] Starting request: urn:request:7252b3f5-bb09-4a69-9afb-161ee7075385 (/health/instance) 2020-03-31 09:16:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:48,278 [555] [DEBUG] [app] Starting request: urn:request:199a8600-c010-4457-aa8d-fcb886703c34 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:48,278 [555] [DEBUG] [app] Starting request: urn:request:199a8600-c010-4457-aa8d-fcb886703c34 (/v1/_internal_ping) 2020-03-31 09:16:48,279 [555] [DEBUG] [app] Ending request: urn:request:199a8600-c010-4457-aa8d-fcb886703c34 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:48,279 [555] [DEBUG] [app] Ending request: urn:request:199a8600-c010-4457-aa8d-fcb886703c34 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:48,279 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:48,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:16:48,279 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:48,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:16:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:16:48,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:16:48,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:48,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:48,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:48,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:48,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:48,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:48,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:48,311 [518] [DEBUG] [app] Starting request: urn:request:f6947cff-9510-40fc-912c-eaa3be707534 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:48,311 [518] [DEBUG] [app] Starting request: urn:request:f6947cff-9510-40fc-912c-eaa3be707534 (/_internal_ping) 2020-03-31 09:16:48,311 [518] [DEBUG] [app] Ending request: urn:request:f6947cff-9510-40fc-912c-eaa3be707534 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:48,311 [518] [DEBUG] [app] Ending request: urn:request:f6947cff-9510-40fc-912c-eaa3be707534 (/_internal_ping) 2020-03-31 09:16:48,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:48,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:16:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:16:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:48,325 [543] [DEBUG] [app] Starting request: urn:request:df1a3f3d-fb7d-459c-b828-ae21587b3a86 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:48,325 [543] [DEBUG] [app] Starting request: urn:request:df1a3f3d-fb7d-459c-b828-ae21587b3a86 (/c1/_internal_ping) 2020-03-31 09:16:48,326 [543] [DEBUG] [app] Ending request: urn:request:df1a3f3d-fb7d-459c-b828-ae21587b3a86 (/c1/_internal_ping) 2020-03-31 09:16:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:48,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:16:48,326 [543] [DEBUG] [app] Ending request: urn:request:df1a3f3d-fb7d-459c-b828-ae21587b3a86 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:48,328 [518] [DEBUG] [app] Ending request: urn:request:7252b3f5-bb09-4a69-9afb-161ee7075385 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:16:48,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:48,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:48,328 [518] [DEBUG] [app] Ending request: urn:request:7252b3f5-bb09-4a69-9afb-161ee7075385 (/health/instance) 2020-03-31 09:16:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:16:48,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:16:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:16:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:54.323352+00:00 (in 4.823568 seconds) 2020-03-31 09:16:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:49 UTC)" (scheduled at 2020-03-31 09:16:49.498966+00:00) 2020-03-31 09:16:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:16:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:54.323352+00:00 (in 4.823568 seconds) 2020-03-31 09:16:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:49 UTC)" (scheduled at 2020-03-31 09:16:49.498966+00:00) 2020-03-31 09:16:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:49 UTC)" executed successfully 2020-03-31 09:16:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:16:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:59.560326+00:00 (in 8.448336 seconds) 2020-03-31 09:16:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:51 UTC)" (scheduled at 2020-03-31 09:16:51.110610+00:00) 2020-03-31 09:16:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:16:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:16:59.560326+00:00 (in 8.448336 seconds) 2020-03-31 09:16:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:51 UTC)" (scheduled at 2020-03-31 09:16:51.110610+00:00) 2020-03-31 09:16:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 51, 112936), True, datetime.datetime(2020, 3, 31, 9, 16, 51, 112936), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:16:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 51, 112936), True, datetime.datetime(2020, 3, 31, 9, 16, 51, 112936), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:16:51,133 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:16:51,133 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:51,133 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:16:51,133 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:51 UTC)" executed successfully 2020-03-31 09:16:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:16:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:01.440707+00:00 (in 9.999201 seconds) 2020-03-31 09:16:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:01 UTC)" (scheduled at 2020-03-31 09:16:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:16:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:01.440707+00:00 (in 9.999201 seconds) 2020-03-31 09:16:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:01 UTC)" (scheduled at 2020-03-31 09:16:51.440707+00:00) 2020-03-31 09:16:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:16:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:16:51,467 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:16:51,467 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:16:51,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:51,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:16:51,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:51,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:01 UTC)" executed successfully 2020-03-31 09:16:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:16:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:22.113221+00:00 (in 29.999245 seconds) 2020-03-31 09:16:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:22 UTC)" (scheduled at 2020-03-31 09:16:52.113221+00:00) securityworker stdout | 2020-03-31 09:16:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:22.113221+00:00 (in 29.999245 seconds) 2020-03-31 09:16:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:22 UTC)" (scheduled at 2020-03-31 09:16:52.113221+00:00) 2020-03-31 09:16:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:16:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:16:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:16:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:16:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:16:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:16:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:16:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:22 UTC)" executed successfully 2020-03-31 09:16:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:16:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:04.323352+00:00 (in 9.998887 seconds) 2020-03-31 09:16:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:04 UTC)" (scheduled at 2020-03-31 09:16:54.323352+00:00) 2020-03-31 09:16:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:16:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:04.323352+00:00 (in 9.998887 seconds) 2020-03-31 09:16:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:04 UTC)" (scheduled at 2020-03-31 09:16:54.323352+00:00) 2020-03-31 09:16:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 54, 325300), True, datetime.datetime(2020, 3, 31, 9, 16, 54, 325300), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:16:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 54, 325300), True, datetime.datetime(2020, 3, 31, 9, 16, 54, 325300), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:16:54,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:16:54,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:54,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:54,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:16:54,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:54,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:04 UTC)" executed successfully 2020-03-31 09:16:54,445 [518] [DEBUG] [app] Starting request: urn:request:84ad3621-ebcb-486d-9d01-43ac2967f8e2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:54,445 [518] [DEBUG] [app] Starting request: urn:request:84ad3621-ebcb-486d-9d01-43ac2967f8e2 (/health/instance) 2020-03-31 09:16:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:54,454 [552] [DEBUG] [app] Starting request: urn:request:15934373-f228-4738-b178-c407eab71bfc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:54,454 [552] [DEBUG] [app] Starting request: urn:request:15934373-f228-4738-b178-c407eab71bfc (/v1/_internal_ping) 2020-03-31 09:16:54,455 [552] [DEBUG] [app] Ending request: urn:request:15934373-f228-4738-b178-c407eab71bfc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:54,455 [552] [DEBUG] [app] Ending request: urn:request:15934373-f228-4738-b178-c407eab71bfc (/v1/_internal_ping) 2020-03-31 09:16:54,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-registry stdout | 2020-03-31 09:16:54,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:16:54,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:16:54,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:54,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:54,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:54,487 [518] [DEBUG] [app] Starting request: urn:request:e21e7f30-50f8-4bc2-85e2-cfe1e0ae1f44 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:54,487 [518] [DEBUG] [app] Starting request: urn:request:e21e7f30-50f8-4bc2-85e2-cfe1e0ae1f44 (/_internal_ping) 2020-03-31 09:16:54,488 [518] [DEBUG] [app] Ending request: urn:request:e21e7f30-50f8-4bc2-85e2-cfe1e0ae1f44 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:54,488 [518] [DEBUG] [app] Ending request: urn:request:e21e7f30-50f8-4bc2-85e2-cfe1e0ae1f44 (/_internal_ping) 2020-03-31 09:16:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:16:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:16:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:16:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:16:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:16:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:54,501 [541] [DEBUG] [app] Starting request: urn:request:90df3889-0b0d-4c1e-9d95-e25b618f3b95 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:54,501 [541] [DEBUG] [app] Starting request: urn:request:90df3889-0b0d-4c1e-9d95-e25b618f3b95 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:54,502 [541] [DEBUG] [app] Ending request: urn:request:90df3889-0b0d-4c1e-9d95-e25b618f3b95 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:54,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:16:54,502 [541] [DEBUG] [app] Ending request: urn:request:90df3889-0b0d-4c1e-9d95-e25b618f3b95 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:54,504 [518] [DEBUG] [app] Ending request: urn:request:84ad3621-ebcb-486d-9d01-43ac2967f8e2 (/health/instance) 2020-03-31 09:16:54,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:16:54,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:16:54,504 [518] [DEBUG] [app] Ending request: urn:request:84ad3621-ebcb-486d-9d01-43ac2967f8e2 (/health/instance) 2020-03-31 09:16:54,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:16:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:16:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:03.010318+00:00 (in 6.085147 seconds) 2020-03-31 09:16:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:06 UTC)" (scheduled at 2020-03-31 09:16:56.924497+00:00) 2020-03-31 09:16:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:16:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:03.010318+00:00 (in 6.085147 seconds) 2020-03-31 09:16:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:06 UTC)" (scheduled at 2020-03-31 09:16:56.924497+00:00) 2020-03-31 09:16:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:16:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 56, 925904), True, datetime.datetime(2020, 3, 31, 9, 16, 56, 925904), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:16:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 16, 56, 925904), True, datetime.datetime(2020, 3, 31, 9, 16, 56, 925904), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:16:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:16:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:16:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:16:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:16:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:06 UTC)" executed successfully 2020-03-31 09:16:58,273 [527] [DEBUG] [app] Starting request: urn:request:a0575bf3-cacb-4f8c-9cf5-9a08379a72bd (/health/instance) gunicorn-web stdout | 2020-03-31 09:16:58,273 [527] [DEBUG] [app] Starting request: urn:request:a0575bf3-cacb-4f8c-9cf5-9a08379a72bd (/health/instance) 2020-03-31 09:16:58,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:58,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:58,281 [551] [DEBUG] [app] Starting request: urn:request:93724ff8-066c-413b-a6e2-7127287c5925 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:58,281 [551] [DEBUG] [app] Starting request: urn:request:93724ff8-066c-413b-a6e2-7127287c5925 (/v1/_internal_ping) 2020-03-31 09:16:58,282 [551] [DEBUG] [app] Ending request: urn:request:93724ff8-066c-413b-a6e2-7127287c5925 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:16:58,282 [551] [DEBUG] [app] Ending request: urn:request:93724ff8-066c-413b-a6e2-7127287c5925 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:16:58,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:16:58,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:16:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:16:58,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:16:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:16:58,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:16:58,287 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:16:58,287 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:16:58,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:16:58,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:16:58,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:16:58,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:16:58,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:16:58,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:16:58,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:58,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:58,315 [523] [DEBUG] [app] Starting request: urn:request:fd82afbe-7193-4f82-b1b3-7274a48e1e3d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:58,315 [523] [DEBUG] [app] Starting request: urn:request:fd82afbe-7193-4f82-b1b3-7274a48e1e3d (/_internal_ping) 2020-03-31 09:16:58,316 [523] [DEBUG] [app] Ending request: urn:request:fd82afbe-7193-4f82-b1b3-7274a48e1e3d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:16:58,316 [523] [DEBUG] [app] Ending request: urn:request:fd82afbe-7193-4f82-b1b3-7274a48e1e3d (/_internal_ping) 2020-03-31 09:16:58,317 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:58,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:16:58,317 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:58,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:16:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:16:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:16:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:16:58,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:16:58,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:16:58,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:16:58,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:16:58,332 [543] [DEBUG] [app] Starting request: urn:request:74a4bb39-dcff-461a-b6e9-444a56f9de4c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:58,332 [543] [DEBUG] [app] Starting request: urn:request:74a4bb39-dcff-461a-b6e9-444a56f9de4c (/c1/_internal_ping) 2020-03-31 09:16:58,333 [543] [DEBUG] [app] Ending request: urn:request:74a4bb39-dcff-461a-b6e9-444a56f9de4c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:16:58,333 [543] [DEBUG] [app] Ending request: urn:request:74a4bb39-dcff-461a-b6e9-444a56f9de4c (/c1/_internal_ping) 2020-03-31 09:16:58,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:58,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:16:58,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:16:58,335 [527] [DEBUG] [app] Ending request: urn:request:a0575bf3-cacb-4f8c-9cf5-9a08379a72bd (/health/instance) 2020-03-31 09:16:58,336 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:16:58,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:16:58,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:16:58,335 [527] [DEBUG] [app] Ending request: urn:request:a0575bf3-cacb-4f8c-9cf5-9a08379a72bd (/health/instance) 2020-03-31 09:16:58,336 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:16:58,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:16:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:16:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:16:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:16:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:16:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:51.110610+00:00 (in 51.549282 seconds) 2020-03-31 09:16:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:59 UTC)" (scheduled at 2020-03-31 09:16:59.560326+00:00) 2020-03-31 09:16:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:16:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:51.110610+00:00 (in 51.549282 seconds) 2020-03-31 09:16:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:59 UTC)" (scheduled at 2020-03-31 09:16:59.560326+00:00) 2020-03-31 09:16:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:16:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:17:59 UTC)" executed successfully 2020-03-31 09:17:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:17:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:30.337699+00:00 (in 29.999195 seconds) 2020-03-31 09:17:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:30 UTC)" (scheduled at 2020-03-31 09:17:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:17:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:30.337699+00:00 (in 29.999195 seconds) 2020-03-31 09:17:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:30 UTC)" (scheduled at 2020-03-31 09:17:00.337699+00:00) 2020-03-31 09:17:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 17, 0, 339271), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:17:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 17, 0, 339271), False, 50, 1, 0]) 2020-03-31 09:17:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:17:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:17:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:17:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:30 UTC)" executed successfully 2020-03-31 09:17:01,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:17:01,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:11.440707+00:00 (in 9.996805 seconds) 2020-03-31 09:17:01,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:11 UTC)" (scheduled at 2020-03-31 09:17:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:17:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:11.440707+00:00 (in 9.996805 seconds) 2020-03-31 09:17:01,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:11 UTC)" (scheduled at 2020-03-31 09:17:01.440707+00:00) 2020-03-31 09:17:01,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:17:01,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:17:01,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:17:01,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:17:01,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:01,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:17:01,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:01,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:11 UTC)" executed successfully 2020-03-31 09:17:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:17:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:06.924497+00:00 (in 3.913391 seconds) 2020-03-31 09:17:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:03 UTC)" (scheduled at 2020-03-31 09:17:03.010318+00:00) storagereplication stdout | 2020-03-31 09:17:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:06.924497+00:00 (in 3.913391 seconds) 2020-03-31 09:17:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:03 UTC)" (scheduled at 2020-03-31 09:17:03.010318+00:00) 2020-03-31 09:17:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:17:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), u'imagestoragereplication/%']) 2020-03-31 09:17:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), True, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:17:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), True, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), 0, u'imagestoragereplication/%']) 2020-03-31 09:17:03,033 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), True, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:17:03,033 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), True, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 17, 3, 12014), u'imagestoragereplication/%']) 2020-03-31 09:17:03,036 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:17:03,036 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:03,037 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:03,037 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:03 UTC)" executed successfully 2020-03-31 09:17:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:17:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:17:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 09:17:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:17:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:17:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:24.410927+00:00 (in 21.079692 seconds) 2020-03-31 09:17:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:33 UTC)" (scheduled at 2020-03-31 09:17:03.330112+00:00) 2020-03-31 09:17:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:17:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:24.410927+00:00 (in 21.079692 seconds) 2020-03-31 09:17:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:33 UTC)" (scheduled at 2020-03-31 09:17:03.330112+00:00) 2020-03-31 09:17:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 3, 332180), True, datetime.datetime(2020, 3, 31, 9, 17, 3, 332180), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:17:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 3, 332180), True, datetime.datetime(2020, 3, 31, 9, 17, 3, 332180), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:17:03,355 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:17:03,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:17:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:33 UTC)" executed successfully 2020-03-31 09:17:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:14.323352+00:00 (in 9.999141 seconds) 2020-03-31 09:17:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:14 UTC)" (scheduled at 2020-03-31 09:17:04.323352+00:00) 2020-03-31 09:17:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:17:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:14.323352+00:00 (in 9.999141 seconds) 2020-03-31 09:17:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:14 UTC)" (scheduled at 2020-03-31 09:17:04.323352+00:00) 2020-03-31 09:17:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 4, 325391), True, datetime.datetime(2020, 3, 31, 9, 17, 4, 325391), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:17:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 4, 325391), True, datetime.datetime(2020, 3, 31, 9, 17, 4, 325391), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:17:04,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:17:04,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:14 UTC)" executed successfully 2020-03-31 09:17:04,446 [527] [DEBUG] [app] Starting request: urn:request:acffd008-2a17-4dec-9e28-a525074b7dc1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:04,446 [527] [DEBUG] [app] Starting request: urn:request:acffd008-2a17-4dec-9e28-a525074b7dc1 (/health/instance) 2020-03-31 09:17:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:04,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:04,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:04,453 [549] [DEBUG] [app] Starting request: urn:request:e92a2bdb-9363-4bf1-810a-cf4c55e9518c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:04,453 [549] [DEBUG] [app] Starting request: urn:request:e92a2bdb-9363-4bf1-810a-cf4c55e9518c (/v1/_internal_ping) 2020-03-31 09:17:04,454 [549] [DEBUG] [app] Ending request: urn:request:e92a2bdb-9363-4bf1-810a-cf4c55e9518c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:04,454 [549] [DEBUG] [app] Ending request: urn:request:e92a2bdb-9363-4bf1-810a-cf4c55e9518c (/v1/_internal_ping) 2020-03-31 09:17:04,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:17:04,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:04,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:17:04,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:04,461 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:17:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:04,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:04,461 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:04,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:04,476 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:04,476 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:04,491 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:04,491 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:04,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:04,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:04,497 [518] [DEBUG] [app] Starting request: urn:request:bd1744e3-7b31-422b-a704-64dfba2cd321 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:04,497 [518] [DEBUG] [app] Starting request: urn:request:bd1744e3-7b31-422b-a704-64dfba2cd321 (/_internal_ping) 2020-03-31 09:17:04,498 [518] [DEBUG] [app] Ending request: urn:request:bd1744e3-7b31-422b-a704-64dfba2cd321 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:04,498 [518] [DEBUG] [app] Ending request: urn:request:bd1744e3-7b31-422b-a704-64dfba2cd321 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:17:04,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:04,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:04,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:04,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:04,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:04,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:04,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:04,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:17:04,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:04,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:04,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:04,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:04,512 [536] [DEBUG] [app] Starting request: urn:request:be80f6fa-4715-4792-9367-242c7fe923e9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:04,512 [536] [DEBUG] [app] Starting request: urn:request:be80f6fa-4715-4792-9367-242c7fe923e9 (/c1/_internal_ping) 2020-03-31 09:17:04,513 [536] [DEBUG] [app] Ending request: urn:request:be80f6fa-4715-4792-9367-242c7fe923e9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:04,513 [536] [DEBUG] [app] Ending request: urn:request:be80f6fa-4715-4792-9367-242c7fe923e9 (/c1/_internal_ping) 2020-03-31 09:17:04,514 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:04,514 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:04,514 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:04,515 [527] [DEBUG] [app] Ending request: urn:request:acffd008-2a17-4dec-9e28-a525074b7dc1 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:17:04,514 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:04,515 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:04,515 [527] [DEBUG] [app] Ending request: urn:request:acffd008-2a17-4dec-9e28-a525074b7dc1 (/health/instance) 2020-03-31 09:17:04,516 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:17:04,515 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:04,516 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:17:06,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:17:06,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:16.924497+00:00 (in 9.998173 seconds) 2020-03-31 09:17:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:16 UTC)" (scheduled at 2020-03-31 09:17:06.924497+00:00) 2020-03-31 09:17:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:17:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:16.924497+00:00 (in 9.998173 seconds) 2020-03-31 09:17:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:16 UTC)" (scheduled at 2020-03-31 09:17:06.924497+00:00) 2020-03-31 09:17:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 6, 927342), True, datetime.datetime(2020, 3, 31, 9, 17, 6, 927342), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:17:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 6, 927342), True, datetime.datetime(2020, 3, 31, 9, 17, 6, 927342), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:17:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:17:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:16 UTC)" executed successfully 2020-03-31 09:17:08,273 [518] [DEBUG] [app] Starting request: urn:request:6cfa56fc-f678-4baa-8d37-ad8a7a95f549 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:08,273 [518] [DEBUG] [app] Starting request: urn:request:6cfa56fc-f678-4baa-8d37-ad8a7a95f549 (/health/instance) 2020-03-31 09:17:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:08,280 [542] [DEBUG] [app] Starting request: urn:request:40b75503-63cb-443e-aad3-28ff3514cca3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:08,280 [542] [DEBUG] [app] Starting request: urn:request:40b75503-63cb-443e-aad3-28ff3514cca3 (/v1/_internal_ping) 2020-03-31 09:17:08,281 [542] [DEBUG] [app] Ending request: urn:request:40b75503-63cb-443e-aad3-28ff3514cca3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:08,281 [542] [DEBUG] [app] Ending request: urn:request:40b75503-63cb-443e-aad3-28ff3514cca3 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:08,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:17:08,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:08,285 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:08,285 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:08,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:08,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:08,287 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:08,287 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:08,313 [527] [DEBUG] [app] Starting request: urn:request:d1420d42-7746-4b22-b699-5d2d4183aae3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:08,313 [527] [DEBUG] [app] Starting request: urn:request:d1420d42-7746-4b22-b699-5d2d4183aae3 (/_internal_ping) 2020-03-31 09:17:08,314 [527] [DEBUG] [app] Ending request: urn:request:d1420d42-7746-4b22-b699-5d2d4183aae3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:08,314 [527] [DEBUG] [app] Ending request: urn:request:d1420d42-7746-4b22-b699-5d2d4183aae3 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:17:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:17:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:08,326 [543] [DEBUG] [app] Starting request: urn:request:73ab63c3-dcae-43a6-8558-df80b1d76ca6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:08,326 [543] [DEBUG] [app] Starting request: urn:request:73ab63c3-dcae-43a6-8558-df80b1d76ca6 (/c1/_internal_ping) 2020-03-31 09:17:08,328 [543] [DEBUG] [app] Ending request: urn:request:73ab63c3-dcae-43a6-8558-df80b1d76ca6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:08,328 [543] [DEBUG] [app] Ending request: urn:request:73ab63c3-dcae-43a6-8558-df80b1d76ca6 (/c1/_internal_ping) 2020-03-31 09:17:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:08,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:08,330 [518] [DEBUG] [app] Ending request: urn:request:6cfa56fc-f678-4baa-8d37-ad8a7a95f549 (/health/instance) 2020-03-31 09:17:08,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:17:08,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:17:08,330 [518] [DEBUG] [app] Ending request: urn:request:6cfa56fc-f678-4baa-8d37-ad8a7a95f549 (/health/instance) 2020-03-31 09:17:08,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:17:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:17:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:17:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:21.440707+00:00 (in 9.999161 seconds) 2020-03-31 09:17:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:21 UTC)" (scheduled at 2020-03-31 09:17:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:17:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:21.440707+00:00 (in 9.999161 seconds) 2020-03-31 09:17:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:21 UTC)" (scheduled at 2020-03-31 09:17:11.440707+00:00) 2020-03-31 09:17:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:17:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:17:11,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:17:11,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:17:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:17:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:21 UTC)" executed successfully 2020-03-31 09:17:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:24.323352+00:00 (in 9.998632 seconds) 2020-03-31 09:17:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:24 UTC)" (scheduled at 2020-03-31 09:17:14.323352+00:00) 2020-03-31 09:17:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:17:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:24.323352+00:00 (in 9.998632 seconds) 2020-03-31 09:17:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:24 UTC)" (scheduled at 2020-03-31 09:17:14.323352+00:00) 2020-03-31 09:17:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 14, 326203), True, datetime.datetime(2020, 3, 31, 9, 17, 14, 326203), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:17:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 14, 326203), True, datetime.datetime(2020, 3, 31, 9, 17, 14, 326203), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:17:14,352 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:17:14,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:14,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:14,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:14,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:14,352 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:24 UTC)" executed successfully 2020-03-31 09:17:14,446 [526] [DEBUG] [app] Starting request: urn:request:d141ef4e-c299-44d6-a8c4-9a1df847d4ac (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:14,446 [526] [DEBUG] [app] Starting request: urn:request:d141ef4e-c299-44d6-a8c4-9a1df847d4ac (/health/instance) 2020-03-31 09:17:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:14,454 [552] [DEBUG] [app] Starting request: urn:request:1e38ccc0-272a-4f99-9a5b-4c3bf05deb54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:14,454 [552] [DEBUG] [app] Starting request: urn:request:1e38ccc0-272a-4f99-9a5b-4c3bf05deb54 (/v1/_internal_ping) 2020-03-31 09:17:14,455 [552] [DEBUG] [app] Ending request: urn:request:1e38ccc0-272a-4f99-9a5b-4c3bf05deb54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:14,455 [552] [DEBUG] [app] Ending request: urn:request:1e38ccc0-272a-4f99-9a5b-4c3bf05deb54 (/v1/_internal_ping) 2020-03-31 09:17:14,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:17:14,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:14,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:14,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:17:14,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:14,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:14,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:14,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:14,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:14,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:14,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:14,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:14,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:14,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:14,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:14,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:14,486 [526] [DEBUG] [app] Starting request: urn:request:ef203393-03ce-4fe9-9e4e-f79cd104ed1e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:14,486 [526] [DEBUG] [app] Starting request: urn:request:ef203393-03ce-4fe9-9e4e-f79cd104ed1e (/_internal_ping) 2020-03-31 09:17:14,486 [526] [DEBUG] [app] Ending request: urn:request:ef203393-03ce-4fe9-9e4e-f79cd104ed1e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:14,486 [526] [DEBUG] [app] Ending request: urn:request:ef203393-03ce-4fe9-9e4e-f79cd104ed1e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:17:14,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:14,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:14,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:14,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:14,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:14,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:14,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:14,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:17:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:14,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:14,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:14,501 [536] [DEBUG] [app] Starting request: urn:request:66df9608-2498-4c83-b7fb-308e116e9869 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:14,501 [536] [DEBUG] [app] Starting request: urn:request:66df9608-2498-4c83-b7fb-308e116e9869 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:14,502 [536] [DEBUG] [app] Ending request: urn:request:66df9608-2498-4c83-b7fb-308e116e9869 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:14,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:17:14,502 [536] [DEBUG] [app] Ending request: urn:request:66df9608-2498-4c83-b7fb-308e116e9869 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:14,504 [526] [DEBUG] [app] Ending request: urn:request:d141ef4e-c299-44d6-a8c4-9a1df847d4ac (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:17:14,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:14,504 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:14,504 [526] [DEBUG] [app] Ending request: urn:request:d141ef4e-c299-44d6-a8c4-9a1df847d4ac (/health/instance) 2020-03-31 09:17:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:17:14,504 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:17:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:17:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:45 UTC)" (scheduled at 2020-03-31 09:17:15.064514+00:00) gcworker stdout | 2020-03-31 09:17:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:45 UTC)" (scheduled at 2020-03-31 09:17:15.064514+00:00) 2020-03-31 09:17:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436635, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:17:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436635, 1209600, 500, 1, 0]) 2020-03-31 09:17:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:45.064514+00:00 (in 29.999007 seconds) gcworker stdout | 2020-03-31 09:17:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:45.064514+00:00 (in 29.999007 seconds) 2020-03-31 09:17:15,086 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:17:15,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:17:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:17:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:45 UTC)" executed successfully 2020-03-31 09:17:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:17:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:26.924497+00:00 (in 9.999050 seconds) storagereplication stdout | 2020-03-31 09:17:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:26.924497+00:00 (in 9.999050 seconds) 2020-03-31 09:17:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:26 UTC)" (scheduled at 2020-03-31 09:17:16.924497+00:00) 2020-03-31 09:17:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:17:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:26 UTC)" (scheduled at 2020-03-31 09:17:16.924497+00:00) 2020-03-31 09:17:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 16, 926729), True, datetime.datetime(2020, 3, 31, 9, 17, 16, 926729), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:17:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 16, 926729), True, datetime.datetime(2020, 3, 31, 9, 17, 16, 926729), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:17:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:17:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:26 UTC)" executed successfully 2020-03-31 09:17:18,273 [518] [DEBUG] [app] Starting request: urn:request:680fb487-f264-4fe0-b225-54695a36db89 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:18,273 [518] [DEBUG] [app] Starting request: urn:request:680fb487-f264-4fe0-b225-54695a36db89 (/health/instance) 2020-03-31 09:17:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:18,280 [556] [DEBUG] [app] Starting request: urn:request:34e12ac3-7300-484c-8e89-2a0581fdbaf5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:18,280 [556] [DEBUG] [app] Starting request: urn:request:34e12ac3-7300-484c-8e89-2a0581fdbaf5 (/v1/_internal_ping) 2020-03-31 09:17:18,281 [556] [DEBUG] [app] Ending request: urn:request:34e12ac3-7300-484c-8e89-2a0581fdbaf5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:18,281 [556] [DEBUG] [app] Ending request: urn:request:34e12ac3-7300-484c-8e89-2a0581fdbaf5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:18,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:17:18,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:18,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:18,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:18,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:18,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:18,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:18,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:18,314 [518] [DEBUG] [app] Starting request: urn:request:f469d667-d628-4d36-9f36-0cfb01f96495 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:18,314 [518] [DEBUG] [app] Starting request: urn:request:f469d667-d628-4d36-9f36-0cfb01f96495 (/_internal_ping) 2020-03-31 09:17:18,314 [518] [DEBUG] [app] Ending request: urn:request:f469d667-d628-4d36-9f36-0cfb01f96495 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:18,314 [518] [DEBUG] [app] Ending request: urn:request:f469d667-d628-4d36-9f36-0cfb01f96495 (/_internal_ping) 2020-03-31 09:17:18,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:17:18,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:17:18,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:17:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:17:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:17:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:18,328 [532] [DEBUG] [app] Starting request: urn:request:646ca07f-6591-41d1-9c87-b67b88e21dd1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:18,328 [532] [DEBUG] [app] Starting request: urn:request:646ca07f-6591-41d1-9c87-b67b88e21dd1 (/c1/_internal_ping) 2020-03-31 09:17:18,329 [532] [DEBUG] [app] Ending request: urn:request:646ca07f-6591-41d1-9c87-b67b88e21dd1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:18,329 [532] [DEBUG] [app] Ending request: urn:request:646ca07f-6591-41d1-9c87-b67b88e21dd1 (/c1/_internal_ping) 2020-03-31 09:17:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:18,329 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:18,330 [518] [DEBUG] [app] Ending request: urn:request:680fb487-f264-4fe0-b225-54695a36db89 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:17:18,329 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:18,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:17:18,330 [518] [DEBUG] [app] Ending request: urn:request:680fb487-f264-4fe0-b225-54695a36db89 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:18,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:17:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:17:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:31.440707+00:00 (in 9.999033 seconds) 2020-03-31 09:17:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:31 UTC)" (scheduled at 2020-03-31 09:17:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:17:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:31.440707+00:00 (in 9.999033 seconds) 2020-03-31 09:17:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:31 UTC)" (scheduled at 2020-03-31 09:17:21.440707+00:00) 2020-03-31 09:17:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:17:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:17:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:17:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:17:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:17:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:31 UTC)" executed successfully 2020-03-31 09:17:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:17:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:52.113221+00:00 (in 29.999081 seconds) 2020-03-31 09:17:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:52 UTC)" (scheduled at 2020-03-31 09:17:22.113221+00:00) securityworker stdout | 2020-03-31 09:17:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:52.113221+00:00 (in 29.999081 seconds) 2020-03-31 09:17:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:52 UTC)" (scheduled at 2020-03-31 09:17:22.113221+00:00) 2020-03-31 09:17:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:17:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:17:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:17:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:17:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:17:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:17:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:17:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:17:52 UTC)" executed successfully 2020-03-31 09:17:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:31.498397+00:00 (in 7.173665 seconds) notificationworker stdout | 2020-03-31 09:17:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:31.498397+00:00 (in 7.173665 seconds) 2020-03-31 09:17:24,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:34 UTC)" (scheduled at 2020-03-31 09:17:24.323352+00:00) 2020-03-31 09:17:24,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:24,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 24, 326723), True, datetime.datetime(2020, 3, 31, 9, 17, 24, 326723), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:17:24,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:34 UTC)" (scheduled at 2020-03-31 09:17:24.323352+00:00) 2020-03-31 09:17:24,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:24,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 24, 326723), True, datetime.datetime(2020, 3, 31, 9, 17, 24, 326723), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:17:24,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:17:24,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:24,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:24,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:24,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:24,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:34 UTC)" executed successfully 2020-03-31 09:17:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:17:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:33.330112+00:00 (in 8.918414 seconds) 2020-03-31 09:17:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:24 UTC)" (scheduled at 2020-03-31 09:17:24.410927+00:00) 2020-03-31 09:17:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:17:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:33.330112+00:00 (in 8.918414 seconds) 2020-03-31 09:17:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:24 UTC)" (scheduled at 2020-03-31 09:17:24.410927+00:00) 2020-03-31 09:17:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:17:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:24 UTC)" executed successfully 2020-03-31 09:17:24,446 [518] [DEBUG] [app] Starting request: urn:request:222326b5-9623-4796-80ff-903c0b5efef9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:24,446 [518] [DEBUG] [app] Starting request: urn:request:222326b5-9623-4796-80ff-903c0b5efef9 (/health/instance) 2020-03-31 09:17:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:24,456 [556] [DEBUG] [app] Starting request: urn:request:5814e9a8-eff6-469e-a92d-08c0b10d9d45 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:24,456 [556] [DEBUG] [app] Starting request: urn:request:5814e9a8-eff6-469e-a92d-08c0b10d9d45 (/v1/_internal_ping) 2020-03-31 09:17:24,457 [556] [DEBUG] [app] Ending request: urn:request:5814e9a8-eff6-469e-a92d-08c0b10d9d45 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:24,457 [556] [DEBUG] [app] Ending request: urn:request:5814e9a8-eff6-469e-a92d-08c0b10d9d45 (/v1/_internal_ping) 2020-03-31 09:17:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:17:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:17:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:24,463 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:24,463 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:24,465 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:24,465 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:24,466 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:24,466 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:24,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:24,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:24,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:24,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:24,495 [527] [DEBUG] [app] Starting request: urn:request:38e797ce-930c-4636-9f6e-82d22dc22fe9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:24,495 [527] [DEBUG] [app] Starting request: urn:request:38e797ce-930c-4636-9f6e-82d22dc22fe9 (/_internal_ping) 2020-03-31 09:17:24,497 [527] [DEBUG] [app] Ending request: urn:request:38e797ce-930c-4636-9f6e-82d22dc22fe9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:24,497 [527] [DEBUG] [app] Ending request: urn:request:38e797ce-930c-4636-9f6e-82d22dc22fe9 (/_internal_ping) 2020-03-31 09:17:24,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:24,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:17:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:17:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:24,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:17:24,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:17:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:24,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:24,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:24,514 [541] [DEBUG] [app] Starting request: urn:request:10c29638-2a89-475f-9290-52a516cd8183 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:24,514 [541] [DEBUG] [app] Starting request: urn:request:10c29638-2a89-475f-9290-52a516cd8183 (/c1/_internal_ping) 2020-03-31 09:17:24,516 [541] [DEBUG] [app] Ending request: urn:request:10c29638-2a89-475f-9290-52a516cd8183 (/c1/_internal_ping) 2020-03-31 09:17:24,517 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:17:24,517 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:17:24,516 [541] [DEBUG] [app] Ending request: urn:request:10c29638-2a89-475f-9290-52a516cd8183 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:24,517 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:17:24,519 [518] [DEBUG] [app] Ending request: urn:request:222326b5-9623-4796-80ff-903c0b5efef9 (/health/instance) 2020-03-31 09:17:24,519 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:17:24,517 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:24,520 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) gunicorn-web stdout | 2020-03-31 09:17:24,519 [518] [DEBUG] [app] Ending request: urn:request:222326b5-9623-4796-80ff-903c0b5efef9 (/health/instance) 2020-03-31 09:17:24,519 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:24,520 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) 2020-03-31 09:17:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:36.924497+00:00 (in 9.999184 seconds) 2020-03-31 09:17:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:36 UTC)" (scheduled at 2020-03-31 09:17:26.924497+00:00) 2020-03-31 09:17:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 26, 926098), True, datetime.datetime(2020, 3, 31, 9, 17, 26, 926098), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:17:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:36.924497+00:00 (in 9.999184 seconds) 2020-03-31 09:17:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:36 UTC)" (scheduled at 2020-03-31 09:17:26.924497+00:00) 2020-03-31 09:17:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 26, 926098), True, datetime.datetime(2020, 3, 31, 9, 17, 26, 926098), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:17:26,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:17:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:36 UTC)" executed successfully 2020-03-31 09:17:28,272 [518] [DEBUG] [app] Starting request: urn:request:c98910c6-d07c-461f-abd3-dfe9b17ab67a (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:28,272 [518] [DEBUG] [app] Starting request: urn:request:c98910c6-d07c-461f-abd3-dfe9b17ab67a (/health/instance) 2020-03-31 09:17:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:28,277 [538] [DEBUG] [app] Starting request: urn:request:646cb5d5-dc16-4354-a20f-8f364b311a33 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:28,277 [538] [DEBUG] [app] Starting request: urn:request:646cb5d5-dc16-4354-a20f-8f364b311a33 (/v1/_internal_ping) 2020-03-31 09:17:28,278 [538] [DEBUG] [app] Ending request: urn:request:646cb5d5-dc16-4354-a20f-8f364b311a33 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:28,278 [538] [DEBUG] [app] Ending request: urn:request:646cb5d5-dc16-4354-a20f-8f364b311a33 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:17:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:28,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:28,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:28,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:28,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:28,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:28,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:28,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:28,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:28,309 [527] [DEBUG] [app] Starting request: urn:request:8bf68b29-9413-4964-9b46-91e95d8e1a26 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:28,309 [527] [DEBUG] [app] Starting request: urn:request:8bf68b29-9413-4964-9b46-91e95d8e1a26 (/_internal_ping) 2020-03-31 09:17:28,310 [527] [DEBUG] [app] Ending request: urn:request:8bf68b29-9413-4964-9b46-91e95d8e1a26 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:28,310 [527] [DEBUG] [app] Ending request: urn:request:8bf68b29-9413-4964-9b46-91e95d8e1a26 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:17:28,310 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:28,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:28,310 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:28,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:17:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:17:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:17:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:28,324 [535] [DEBUG] [app] Starting request: urn:request:86747cb0-b313-4c06-bcb0-5024a95d8756 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:28,324 [535] [DEBUG] [app] Starting request: urn:request:86747cb0-b313-4c06-bcb0-5024a95d8756 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:28,325 [535] [DEBUG] [app] Ending request: urn:request:86747cb0-b313-4c06-bcb0-5024a95d8756 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:28,326 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:17:28,325 [535] [DEBUG] [app] Ending request: urn:request:86747cb0-b313-4c06-bcb0-5024a95d8756 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:28,327 [518] [DEBUG] [app] Ending request: urn:request:c98910c6-d07c-461f-abd3-dfe9b17ab67a (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:17:28,326 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:28,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:28,327 [518] [DEBUG] [app] Ending request: urn:request:c98910c6-d07c-461f-abd3-dfe9b17ab67a (/health/instance) 2020-03-31 09:17:28,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:17:28,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:28,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:17:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:17:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:00.337699+00:00 (in 29.999138 seconds) 2020-03-31 09:17:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:00 UTC)" (scheduled at 2020-03-31 09:17:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:17:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:00.337699+00:00 (in 29.999138 seconds) 2020-03-31 09:17:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:00 UTC)" (scheduled at 2020-03-31 09:17:30.337699+00:00) 2020-03-31 09:17:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 17, 30, 339351), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:17:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 17, 30, 339351), False, 50, 1, 0]) 2020-03-31 09:17:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:17:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:17:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:17:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:00 UTC)" executed successfully 2020-03-31 09:17:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:17:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:41.440707+00:00 (in 9.999203 seconds) 2020-03-31 09:17:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:41 UTC)" (scheduled at 2020-03-31 09:17:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:17:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:41.440707+00:00 (in 9.999203 seconds) 2020-03-31 09:17:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:41 UTC)" (scheduled at 2020-03-31 09:17:31.440707+00:00) 2020-03-31 09:17:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:17:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:17:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:17:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:17:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:17:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:41 UTC)" executed successfully 2020-03-31 09:17:31,498 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:31,498 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:34.323352+00:00 (in 2.824176 seconds) 2020-03-31 09:17:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:31 UTC)" (scheduled at 2020-03-31 09:17:31.498397+00:00) notificationworker stdout | 2020-03-31 09:17:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:34.323352+00:00 (in 2.824176 seconds) 2020-03-31 09:17:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:31 UTC)" (scheduled at 2020-03-31 09:17:31.498397+00:00) 2020-03-31 09:17:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), u'notification/%']) notificationworker stdout | 2020-03-31 09:17:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), u'notification/%']) 2020-03-31 09:17:31,516 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), True, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:17:31,516 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), True, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), 0, u'notification/%']) 2020-03-31 09:17:31,521 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), True, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), u'notification/%']) notificationworker stdout | 2020-03-31 09:17:31,521 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), True, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 17, 31, 500057), u'notification/%']) 2020-03-31 09:17:31,525 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:17:31,525 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:31,528 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:31,528 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:31 UTC)" executed successfully 2020-03-31 09:17:31,528 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:17:31,528 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:31,529 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:17:31,529 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:31,530 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 09:17:31,530 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:17:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:17:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:03.330112+00:00 (in 29.999243 seconds) 2020-03-31 09:17:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:03 UTC)" (scheduled at 2020-03-31 09:17:33.330112+00:00) 2020-03-31 09:17:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:17:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:03.330112+00:00 (in 29.999243 seconds) 2020-03-31 09:17:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:03 UTC)" (scheduled at 2020-03-31 09:17:33.330112+00:00) 2020-03-31 09:17:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 33, 331732), True, datetime.datetime(2020, 3, 31, 9, 17, 33, 331732), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:17:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 33, 331732), True, datetime.datetime(2020, 3, 31, 9, 17, 33, 331732), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:17:33,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:17:33,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:17:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:03 UTC)" executed successfully 2020-03-31 09:17:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:17:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:43.539999+00:00 (in 10.172422 seconds) 2020-03-31 09:17:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:33 UTC)" (scheduled at 2020-03-31 09:17:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:17:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:43.539999+00:00 (in 10.172422 seconds) 2020-03-31 09:17:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:33 UTC)" (scheduled at 2020-03-31 09:17:33.366791+00:00) 2020-03-31 09:17:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:17:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 33, 368495), True, datetime.datetime(2020, 3, 31, 9, 17, 33, 368495), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:17:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 33, 368495), True, datetime.datetime(2020, 3, 31, 9, 17, 33, 368495), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:17:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:17:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:17:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:33 UTC)" executed successfully 2020-03-31 09:17:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:44.323352+00:00 (in 9.999121 seconds) 2020-03-31 09:17:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:44 UTC)" (scheduled at 2020-03-31 09:17:34.323352+00:00) 2020-03-31 09:17:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:17:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:44.323352+00:00 (in 9.999121 seconds) 2020-03-31 09:17:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:44 UTC)" (scheduled at 2020-03-31 09:17:34.323352+00:00) 2020-03-31 09:17:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 34, 325164), True, datetime.datetime(2020, 3, 31, 9, 17, 34, 325164), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:17:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 34, 325164), True, datetime.datetime(2020, 3, 31, 9, 17, 34, 325164), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:17:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:17:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:44 UTC)" executed successfully 2020-03-31 09:17:34,446 [518] [DEBUG] [app] Starting request: urn:request:b7dcf197-b9da-41d9-89db-9e190515993b (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:34,446 [518] [DEBUG] [app] Starting request: urn:request:b7dcf197-b9da-41d9-89db-9e190515993b (/health/instance) 2020-03-31 09:17:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:34,454 [546] [DEBUG] [app] Starting request: urn:request:285a117b-6f82-41c1-a53b-e0a29b697a8f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:34,454 [546] [DEBUG] [app] Starting request: urn:request:285a117b-6f82-41c1-a53b-e0a29b697a8f (/v1/_internal_ping) 2020-03-31 09:17:34,455 [546] [DEBUG] [app] Ending request: urn:request:285a117b-6f82-41c1-a53b-e0a29b697a8f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:34,455 [546] [DEBUG] [app] Ending request: urn:request:285a117b-6f82-41c1-a53b-e0a29b697a8f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:34,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:17:34,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:34,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:34,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:34,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:34,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:34,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:34,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:34,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:34,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:34,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:34,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:34,491 [527] [DEBUG] [app] Starting request: urn:request:ae1cf5e1-c5e7-4f06-8a9c-91ce028650f6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:34,491 [527] [DEBUG] [app] Starting request: urn:request:ae1cf5e1-c5e7-4f06-8a9c-91ce028650f6 (/_internal_ping) 2020-03-31 09:17:34,492 [527] [DEBUG] [app] Ending request: urn:request:ae1cf5e1-c5e7-4f06-8a9c-91ce028650f6 (/_internal_ping) 2020-03-31 09:17:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:34,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:17:34,492 [527] [DEBUG] [app] Ending request: urn:request:ae1cf5e1-c5e7-4f06-8a9c-91ce028650f6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:34,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:17:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:17:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:34,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:34,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:34,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:34,506 [543] [DEBUG] [app] Starting request: urn:request:a76c5b03-90e9-4ad7-9d9f-86315395b034 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:34,506 [543] [DEBUG] [app] Starting request: urn:request:a76c5b03-90e9-4ad7-9d9f-86315395b034 (/c1/_internal_ping) 2020-03-31 09:17:34,507 [543] [DEBUG] [app] Ending request: urn:request:a76c5b03-90e9-4ad7-9d9f-86315395b034 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:34,507 [543] [DEBUG] [app] Ending request: urn:request:a76c5b03-90e9-4ad7-9d9f-86315395b034 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:34,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:34,508 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:34,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:17:34,508 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:34,509 [518] [DEBUG] [app] Ending request: urn:request:b7dcf197-b9da-41d9-89db-9e190515993b (/health/instance) 2020-03-31 09:17:34,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:34,509 [518] [DEBUG] [app] Ending request: urn:request:b7dcf197-b9da-41d9-89db-9e190515993b (/health/instance) 2020-03-31 09:17:34,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:34,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:17:34,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:17:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:17:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:38.010934+00:00 (in 1.085425 seconds) 2020-03-31 09:17:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:46 UTC)" (scheduled at 2020-03-31 09:17:36.924497+00:00) 2020-03-31 09:17:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:17:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:38.010934+00:00 (in 1.085425 seconds) 2020-03-31 09:17:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:46 UTC)" (scheduled at 2020-03-31 09:17:36.924497+00:00) 2020-03-31 09:17:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 36, 926699), True, datetime.datetime(2020, 3, 31, 9, 17, 36, 926699), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:17:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 36, 926699), True, datetime.datetime(2020, 3, 31, 9, 17, 36, 926699), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:17:36,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:17:36,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:36,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:36,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:46 UTC)" executed successfully 2020-03-31 09:17:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:17:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:46.924497+00:00 (in 8.912465 seconds) 2020-03-31 09:17:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:38 UTC)" (scheduled at 2020-03-31 09:17:38.010934+00:00) 2020-03-31 09:17:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:46.924497+00:00 (in 8.912465 seconds) 2020-03-31 09:17:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:38 UTC)" (scheduled at 2020-03-31 09:17:38.010934+00:00) 2020-03-31 09:17:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:38 UTC)" executed successfully 2020-03-31 09:17:38,272 [523] [DEBUG] [app] Starting request: urn:request:a7c15e53-b513-4fa5-b6f1-40c88118347f (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:38,272 [523] [DEBUG] [app] Starting request: urn:request:a7c15e53-b513-4fa5-b6f1-40c88118347f (/health/instance) 2020-03-31 09:17:38,279 [523] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 09:17:38,279 [523] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 09:17:38,288 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:38,288 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:38,290 [551] [DEBUG] [app] Starting request: urn:request:030142a1-55a1-46a8-9748-31c93de1c7ac (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:38,290 [551] [DEBUG] [app] Starting request: urn:request:030142a1-55a1-46a8-9748-31c93de1c7ac (/v1/_internal_ping) 2020-03-31 09:17:38,291 [551] [DEBUG] [app] Ending request: urn:request:030142a1-55a1-46a8-9748-31c93de1c7ac (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:38,291 [551] [DEBUG] [app] Ending request: urn:request:030142a1-55a1-46a8-9748-31c93de1c7ac (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:38,292 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:38,292 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:17:38,292 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:38,292 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:38,294 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:38,294 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:38,295 [523] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:38,295 [523] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:38,296 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:38,296 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:38,297 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:38,297 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:38,307 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:38,307 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:38,321 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:38,321 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:38,325 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:38,325 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:38,326 [523] [DEBUG] [app] Starting request: urn:request:5ecb85f5-5d15-4c62-8a01-6d97da6a7a36 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:38,326 [523] [DEBUG] [app] Starting request: urn:request:5ecb85f5-5d15-4c62-8a01-6d97da6a7a36 (/_internal_ping) 2020-03-31 09:17:38,327 [523] [DEBUG] [app] Ending request: urn:request:5ecb85f5-5d15-4c62-8a01-6d97da6a7a36 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:38,327 [523] [DEBUG] [app] Ending request: urn:request:5ecb85f5-5d15-4c62-8a01-6d97da6a7a36 (/_internal_ping) 2020-03-31 09:17:38,327 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:17:38,327 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:38,328 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:17:38,328 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:38,331 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:38,331 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:38,331 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:38,331 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:38,334 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:38,334 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:17:38,335 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:38,335 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:38,337 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 17, 38, 336400), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 17, 38, 336442), u'quay', None, datetime.datetime(2020, 3, 30, 9, 17, 38, 336467), datetime.datetime(2020, 3, 24, 9, 17, 38, 336483), None]) gunicorn-web stdout | 2020-03-31 09:17:38,337 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 17, 38, 336400), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 17, 38, 336442), u'quay', None, datetime.datetime(2020, 3, 30, 9, 17, 38, 336467), datetime.datetime(2020, 3, 24, 9, 17, 38, 336483), None]) 2020-03-31 09:17:38,352 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:38,352 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:38,353 [532] [DEBUG] [app] Starting request: urn:request:5381fcd1-484f-4e1f-8208-8dec657fdae6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:38,353 [532] [DEBUG] [app] Starting request: urn:request:5381fcd1-484f-4e1f-8208-8dec657fdae6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:38,354 [532] [DEBUG] [app] Ending request: urn:request:5381fcd1-484f-4e1f-8208-8dec657fdae6 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:38,355 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:38,355 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:17:38,354 [532] [DEBUG] [app] Ending request: urn:request:5381fcd1-484f-4e1f-8208-8dec657fdae6 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:38,355 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:17:38,355 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:38,357 [523] [DEBUG] [app] Ending request: urn:request:a7c15e53-b513-4fa5-b6f1-40c88118347f (/health/instance) 2020-03-31 09:17:38,357 [523] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:38,357 [523] [DEBUG] [app] Ending request: urn:request:a7c15e53-b513-4fa5-b6f1-40c88118347f (/health/instance) 2020-03-31 09:17:38,357 [523] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:38,357 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.086 49 0.086) gunicorn-web stdout | 2020-03-31 09:17:38,357 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.086 49 0.086) 2020-03-31 09:17:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:17:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:41 UTC)" (scheduled at 2020-03-31 09:17:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:17:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:41 UTC)" (scheduled at 2020-03-31 09:17:41.440707+00:00) 2020-03-31 09:17:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:51.440707+00:00 (in 9.998789 seconds) repositoryactioncounter stdout | 2020-03-31 09:17:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:51.440707+00:00 (in 9.998789 seconds) 2020-03-31 09:17:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:17:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:17:41,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:17:41,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:17:41,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:17:41,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:51 UTC)" executed successfully 2020-03-31 09:17:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:17:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:33.366791+00:00 (in 49.825691 seconds) 2020-03-31 09:17:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:43 UTC)" (scheduled at 2020-03-31 09:17:43.539999+00:00) 2020-03-31 09:17:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:17:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:33.366791+00:00 (in 49.825691 seconds) 2020-03-31 09:17:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:43 UTC)" (scheduled at 2020-03-31 09:17:43.539999+00:00) 2020-03-31 09:17:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:43 UTC)" executed successfully 2020-03-31 09:17:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:49.498966+00:00 (in 5.174745 seconds) 2020-03-31 09:17:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:54 UTC)" (scheduled at 2020-03-31 09:17:44.323352+00:00) 2020-03-31 09:17:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:17:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:49.498966+00:00 (in 5.174745 seconds) 2020-03-31 09:17:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:54 UTC)" (scheduled at 2020-03-31 09:17:44.323352+00:00) 2020-03-31 09:17:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 44, 325135), True, datetime.datetime(2020, 3, 31, 9, 17, 44, 325135), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:17:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 44, 325135), True, datetime.datetime(2020, 3, 31, 9, 17, 44, 325135), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:17:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:17:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:54 UTC)" executed successfully 2020-03-31 09:17:44,446 [526] [DEBUG] [app] Starting request: urn:request:b0283645-8439-4da2-9af9-f88d7d2a6fe9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:44,446 [526] [DEBUG] [app] Starting request: urn:request:b0283645-8439-4da2-9af9-f88d7d2a6fe9 (/health/instance) 2020-03-31 09:17:44,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:44,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:44,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:44,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:44,455 [546] [DEBUG] [app] Starting request: urn:request:ae3b9e02-5ad5-40e5-bd4e-443a6877a86d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:44,455 [546] [DEBUG] [app] Starting request: urn:request:ae3b9e02-5ad5-40e5-bd4e-443a6877a86d (/v1/_internal_ping) 2020-03-31 09:17:44,456 [546] [DEBUG] [app] Ending request: urn:request:ae3b9e02-5ad5-40e5-bd4e-443a6877a86d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:44,456 [546] [DEBUG] [app] Ending request: urn:request:ae3b9e02-5ad5-40e5-bd4e-443a6877a86d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:44,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:44,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:17:44,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:44,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:44,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:17:44,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:44,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:44,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:44,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:44,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:44,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:44,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:44,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:44,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:44,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:44,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:44,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:44,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:44,491 [526] [DEBUG] [app] Starting request: urn:request:7d2dd361-6e2e-40e3-b9d0-bec3b12afe4d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:44,491 [526] [DEBUG] [app] Starting request: urn:request:7d2dd361-6e2e-40e3-b9d0-bec3b12afe4d (/_internal_ping) 2020-03-31 09:17:44,492 [526] [DEBUG] [app] Ending request: urn:request:7d2dd361-6e2e-40e3-b9d0-bec3b12afe4d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:44,492 [526] [DEBUG] [app] Ending request: urn:request:7d2dd361-6e2e-40e3-b9d0-bec3b12afe4d (/_internal_ping) 2020-03-31 09:17:44,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:17:44,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:44,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:17:44,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:44,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:44,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:44,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:44,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:44,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:44,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:17:44,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:44,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:44,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:44,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:44,506 [535] [DEBUG] [app] Starting request: urn:request:18fa22b1-5255-4ac5-b874-42dc9fa77be9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:44,506 [535] [DEBUG] [app] Starting request: urn:request:18fa22b1-5255-4ac5-b874-42dc9fa77be9 (/c1/_internal_ping) 2020-03-31 09:17:44,507 [535] [DEBUG] [app] Ending request: urn:request:18fa22b1-5255-4ac5-b874-42dc9fa77be9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:44,507 [535] [DEBUG] [app] Ending request: urn:request:18fa22b1-5255-4ac5-b874-42dc9fa77be9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:44,507 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:44,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:17:44,507 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:44,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:44,508 [526] [DEBUG] [app] Ending request: urn:request:b0283645-8439-4da2-9af9-f88d7d2a6fe9 (/health/instance) 2020-03-31 09:17:44,509 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:44,508 [526] [DEBUG] [app] Ending request: urn:request:b0283645-8439-4da2-9af9-f88d7d2a6fe9 (/health/instance) 2020-03-31 09:17:44,509 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:44,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:17:44,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:17:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:17:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:45,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:15.064514+00:00 (in 29.995799 seconds) 2020-03-31 09:17:45,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:15 UTC)" (scheduled at 2020-03-31 09:17:45.064514+00:00) gcworker stdout | 2020-03-31 09:17:45,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:15.064514+00:00 (in 29.995799 seconds) 2020-03-31 09:17:45,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:15 UTC)" (scheduled at 2020-03-31 09:17:45.064514+00:00) 2020-03-31 09:17:45,072 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436665, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:17:45,072 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436665, 1209600, 500, 1, 0]) 2020-03-31 09:17:45,089 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:17:45,089 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:17:45,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:45,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:17:45,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:45,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:15 UTC)" executed successfully 2020-03-31 09:17:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:56.924497+00:00 (in 9.999229 seconds) 2020-03-31 09:17:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:56 UTC)" (scheduled at 2020-03-31 09:17:46.924497+00:00) 2020-03-31 09:17:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:17:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:17:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:56.924497+00:00 (in 9.999229 seconds) 2020-03-31 09:17:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:56 UTC)" (scheduled at 2020-03-31 09:17:46.924497+00:00) 2020-03-31 09:17:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 46, 925932), True, datetime.datetime(2020, 3, 31, 9, 17, 46, 925932), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:17:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 46, 925932), True, datetime.datetime(2020, 3, 31, 9, 17, 46, 925932), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:17:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:17:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:17:56 UTC)" executed successfully 2020-03-31 09:17:48,273 [518] [DEBUG] [app] Starting request: urn:request:2c36045d-9f6e-4c37-891a-be870da1b688 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:48,273 [518] [DEBUG] [app] Starting request: urn:request:2c36045d-9f6e-4c37-891a-be870da1b688 (/health/instance) 2020-03-31 09:17:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:48,281 [544] [DEBUG] [app] Starting request: urn:request:cd5a44e2-2a1f-4ff3-91b9-db5ad42bc8bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:48,281 [544] [DEBUG] [app] Starting request: urn:request:cd5a44e2-2a1f-4ff3-91b9-db5ad42bc8bd (/v1/_internal_ping) 2020-03-31 09:17:48,282 [544] [DEBUG] [app] Ending request: urn:request:cd5a44e2-2a1f-4ff3-91b9-db5ad42bc8bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:48,282 [544] [DEBUG] [app] Ending request: urn:request:cd5a44e2-2a1f-4ff3-91b9-db5ad42bc8bd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:48,283 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:17:48,283 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:48,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:17:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:17:48,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:48,287 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:48,287 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:48,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:48,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:48,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:48,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:48,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:48,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:48,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:48,314 [518] [DEBUG] [app] Starting request: urn:request:5f1f1969-17a8-42f6-853c-0406512634fc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:48,314 [518] [DEBUG] [app] Starting request: urn:request:5f1f1969-17a8-42f6-853c-0406512634fc (/_internal_ping) 2020-03-31 09:17:48,315 [518] [DEBUG] [app] Ending request: urn:request:5f1f1969-17a8-42f6-853c-0406512634fc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:48,315 [518] [DEBUG] [app] Ending request: urn:request:5f1f1969-17a8-42f6-853c-0406512634fc (/_internal_ping) 2020-03-31 09:17:48,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:17:48,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:17:48,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:48,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:17:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:48,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:48,328 [534] [DEBUG] [app] Starting request: urn:request:1c3ffcd6-77d8-4ea2-8f05-a2ecabf2c792 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:48,328 [534] [DEBUG] [app] Starting request: urn:request:1c3ffcd6-77d8-4ea2-8f05-a2ecabf2c792 (/c1/_internal_ping) 2020-03-31 09:17:48,330 [534] [DEBUG] [app] Ending request: urn:request:1c3ffcd6-77d8-4ea2-8f05-a2ecabf2c792 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:48,330 [534] [DEBUG] [app] Ending request: urn:request:1c3ffcd6-77d8-4ea2-8f05-a2ecabf2c792 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:48,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:17:48,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:17:48,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:48,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:48,332 [518] [DEBUG] [app] Ending request: urn:request:2c36045d-9f6e-4c37-891a-be870da1b688 (/health/instance) 2020-03-31 09:17:48,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:48,332 [518] [DEBUG] [app] Ending request: urn:request:2c36045d-9f6e-4c37-891a-be870da1b688 (/health/instance) 2020-03-31 09:17:48,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:48,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:17:48,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:17:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:54.323352+00:00 (in 4.823545 seconds) 2020-03-31 09:17:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:49 UTC)" (scheduled at 2020-03-31 09:17:49.498966+00:00) 2020-03-31 09:17:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:54.323352+00:00 (in 4.823545 seconds) 2020-03-31 09:17:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:49 UTC)" (scheduled at 2020-03-31 09:17:49.498966+00:00) 2020-03-31 09:17:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:49 UTC)" executed successfully 2020-03-31 09:17:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:17:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:59.560326+00:00 (in 8.448666 seconds) 2020-03-31 09:17:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:51 UTC)" (scheduled at 2020-03-31 09:17:51.110610+00:00) 2020-03-31 09:17:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:17:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:17:59.560326+00:00 (in 8.448666 seconds) 2020-03-31 09:17:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:51 UTC)" (scheduled at 2020-03-31 09:17:51.110610+00:00) 2020-03-31 09:17:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 51, 112464), True, datetime.datetime(2020, 3, 31, 9, 17, 51, 112464), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:17:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 51, 112464), True, datetime.datetime(2020, 3, 31, 9, 17, 51, 112464), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:17:51,132 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:17:51,132 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:17:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:51 UTC)" executed successfully 2020-03-31 09:17:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:17:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:01.440707+00:00 (in 9.996079 seconds) 2020-03-31 09:17:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:01 UTC)" (scheduled at 2020-03-31 09:17:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:17:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:01.440707+00:00 (in 9.996079 seconds) 2020-03-31 09:17:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:01 UTC)" (scheduled at 2020-03-31 09:17:51.440707+00:00) 2020-03-31 09:17:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:17:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:17:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:17:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:17:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:17:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:01 UTC)" executed successfully 2020-03-31 09:17:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:17:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:22.113221+00:00 (in 29.997395 seconds) securityworker stdout | 2020-03-31 09:17:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:22.113221+00:00 (in 29.997395 seconds) 2020-03-31 09:17:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:22 UTC)" (scheduled at 2020-03-31 09:17:52.113221+00:00) securityworker stdout | 2020-03-31 09:17:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:22 UTC)" (scheduled at 2020-03-31 09:17:52.113221+00:00) 2020-03-31 09:17:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:17:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:17:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:17:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:17:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:17:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:17:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:17:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:22 UTC)" executed successfully 2020-03-31 09:17:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:17:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:04.323352+00:00 (in 9.998900 seconds) 2020-03-31 09:17:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:04 UTC)" (scheduled at 2020-03-31 09:17:54.323352+00:00) 2020-03-31 09:17:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:17:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:04.323352+00:00 (in 9.998900 seconds) 2020-03-31 09:17:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:04 UTC)" (scheduled at 2020-03-31 09:17:54.323352+00:00) 2020-03-31 09:17:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 54, 325461), True, datetime.datetime(2020, 3, 31, 9, 17, 54, 325461), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:17:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 54, 325461), True, datetime.datetime(2020, 3, 31, 9, 17, 54, 325461), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:17:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:17:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:17:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:04 UTC)" executed successfully 2020-03-31 09:17:54,445 [518] [DEBUG] [app] Starting request: urn:request:f6da52c6-a6a2-4935-b186-ef762e3c6d1f (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:54,445 [518] [DEBUG] [app] Starting request: urn:request:f6da52c6-a6a2-4935-b186-ef762e3c6d1f (/health/instance) 2020-03-31 09:17:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:54,452 [554] [DEBUG] [app] Starting request: urn:request:2f678835-c7dc-4a56-a7bf-a348391240e1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:54,452 [554] [DEBUG] [app] Starting request: urn:request:2f678835-c7dc-4a56-a7bf-a348391240e1 (/v1/_internal_ping) 2020-03-31 09:17:54,453 [554] [DEBUG] [app] Ending request: urn:request:2f678835-c7dc-4a56-a7bf-a348391240e1 (/v1/_internal_ping) 2020-03-31 09:17:54,454 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:54,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:17:54,453 [554] [DEBUG] [app] Ending request: urn:request:2f678835-c7dc-4a56-a7bf-a348391240e1 (/v1/_internal_ping) 2020-03-31 09:17:54,454 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:54,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:54,456 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:54,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:17:54,456 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:17:54,457 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:17:54,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:17:54,457 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:54,458 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:54,458 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:54,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:54,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:54,493 [518] [DEBUG] [app] Starting request: urn:request:b745adc4-d98a-49a8-97fa-1eb9f3198aa1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:54,493 [518] [DEBUG] [app] Starting request: urn:request:b745adc4-d98a-49a8-97fa-1eb9f3198aa1 (/_internal_ping) 2020-03-31 09:17:54,494 [518] [DEBUG] [app] Ending request: urn:request:b745adc4-d98a-49a8-97fa-1eb9f3198aa1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:54,494 [518] [DEBUG] [app] Ending request: urn:request:b745adc4-d98a-49a8-97fa-1eb9f3198aa1 (/_internal_ping) 2020-03-31 09:17:54,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:17:54,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:17:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:17:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:17:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:17:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:54,506 [535] [DEBUG] [app] Starting request: urn:request:2cfd9c9a-3f03-41d7-ba2b-0bcda4b71839 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:54,506 [535] [DEBUG] [app] Starting request: urn:request:2cfd9c9a-3f03-41d7-ba2b-0bcda4b71839 (/c1/_internal_ping) 2020-03-31 09:17:54,507 [535] [DEBUG] [app] Ending request: urn:request:2cfd9c9a-3f03-41d7-ba2b-0bcda4b71839 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:54,507 [535] [DEBUG] [app] Ending request: urn:request:2cfd9c9a-3f03-41d7-ba2b-0bcda4b71839 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:54,508 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:54,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:17:54,508 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:54,509 [518] [DEBUG] [app] Ending request: urn:request:f6da52c6-a6a2-4935-b186-ef762e3c6d1f (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:54,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:54,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:17:54,509 [518] [DEBUG] [app] Ending request: urn:request:f6da52c6-a6a2-4935-b186-ef762e3c6d1f (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:54,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:17:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:17:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:06.924497+00:00 (in 9.999242 seconds) 2020-03-31 09:17:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:06 UTC)" (scheduled at 2020-03-31 09:17:56.924497+00:00) 2020-03-31 09:17:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:17:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:06.924497+00:00 (in 9.999242 seconds) 2020-03-31 09:17:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:06 UTC)" (scheduled at 2020-03-31 09:17:56.924497+00:00) 2020-03-31 09:17:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:17:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 56, 925954), True, datetime.datetime(2020, 3, 31, 9, 17, 56, 925954), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:17:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 17, 56, 925954), True, datetime.datetime(2020, 3, 31, 9, 17, 56, 925954), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:17:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:17:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:17:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:17:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:06 UTC)" executed successfully 2020-03-31 09:17:58,273 [526] [DEBUG] [app] Starting request: urn:request:ee6bedd4-2332-4eb1-ba34-c3d54fa56cb4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:17:58,273 [526] [DEBUG] [app] Starting request: urn:request:ee6bedd4-2332-4eb1-ba34-c3d54fa56cb4 (/health/instance) 2020-03-31 09:17:58,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:58,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:58,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:58,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:58,281 [553] [DEBUG] [app] Starting request: urn:request:77a38074-75c2-4f5e-88ac-4e948f4c5558 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:58,281 [553] [DEBUG] [app] Starting request: urn:request:77a38074-75c2-4f5e-88ac-4e948f4c5558 (/v1/_internal_ping) 2020-03-31 09:17:58,282 [553] [DEBUG] [app] Ending request: urn:request:77a38074-75c2-4f5e-88ac-4e948f4c5558 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:17:58,282 [553] [DEBUG] [app] Ending request: urn:request:77a38074-75c2-4f5e-88ac-4e948f4c5558 (/v1/_internal_ping) 2020-03-31 09:17:58,283 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:58,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:17:58,283 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:58,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:17:58,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:58,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:17:58,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:17:58,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:17:58,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:17:58,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:17:58,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:17:58,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:17:58,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:17:58,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:17:58,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:17:58,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:17:58,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:58,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:58,316 [525] [DEBUG] [app] Starting request: urn:request:05c2baa2-5284-41c9-afc8-faeea87d81a8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:58,316 [525] [DEBUG] [app] Starting request: urn:request:05c2baa2-5284-41c9-afc8-faeea87d81a8 (/_internal_ping) 2020-03-31 09:17:58,317 [525] [DEBUG] [app] Ending request: urn:request:05c2baa2-5284-41c9-afc8-faeea87d81a8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:17:58,317 [525] [DEBUG] [app] Ending request: urn:request:05c2baa2-5284-41c9-afc8-faeea87d81a8 (/_internal_ping) 2020-03-31 09:17:58,317 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:58,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:17:58,317 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:58,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:17:58,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:58,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:58,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:17:58,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:17:58,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:58,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:58,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:17:58,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:17:58,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:17:58,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:17:58,332 [537] [DEBUG] [app] Starting request: urn:request:aaae63ba-1b85-4316-a3e4-08bc3701b803 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:58,332 [537] [DEBUG] [app] Starting request: urn:request:aaae63ba-1b85-4316-a3e4-08bc3701b803 (/c1/_internal_ping) 2020-03-31 09:17:58,333 [537] [DEBUG] [app] Ending request: urn:request:aaae63ba-1b85-4316-a3e4-08bc3701b803 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:17:58,333 [537] [DEBUG] [app] Ending request: urn:request:aaae63ba-1b85-4316-a3e4-08bc3701b803 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:17:58,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:17:58,333 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:17:58,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:17:58,333 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:17:58,335 [526] [DEBUG] [app] Ending request: urn:request:ee6bedd4-2332-4eb1-ba34-c3d54fa56cb4 (/health/instance) 2020-03-31 09:17:58,336 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:17:58,335 [526] [DEBUG] [app] Ending request: urn:request:ee6bedd4-2332-4eb1-ba34-c3d54fa56cb4 (/health/instance) 2020-03-31 09:17:58,336 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:17:58,336 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:17:58,336 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:17:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:17:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:17:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:17:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:17:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:51.110610+00:00 (in 51.549208 seconds) 2020-03-31 09:17:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:59 UTC)" (scheduled at 2020-03-31 09:17:59.560326+00:00) 2020-03-31 09:17:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:17:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:51.110610+00:00 (in 51.549208 seconds) 2020-03-31 09:17:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:59 UTC)" (scheduled at 2020-03-31 09:17:59.560326+00:00) 2020-03-31 09:17:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:17:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:59 UTC)" executed successfully 2020-03-31 09:18:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:18:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:30.337699+00:00 (in 29.998931 seconds) 2020-03-31 09:18:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:30 UTC)" (scheduled at 2020-03-31 09:18:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:18:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:30.337699+00:00 (in 29.998931 seconds) 2020-03-31 09:18:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:30 UTC)" (scheduled at 2020-03-31 09:18:00.337699+00:00) 2020-03-31 09:18:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 18, 0, 339652), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:18:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 18, 0, 339652), False, 50, 1, 0]) 2020-03-31 09:18:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:18:00,356 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:18:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:18:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:18:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:30 UTC)" executed successfully 2020-03-31 09:18:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:18:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:11.440707+00:00 (in 9.998974 seconds) 2020-03-31 09:18:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:11 UTC)" (scheduled at 2020-03-31 09:18:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:18:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:11.440707+00:00 (in 9.998974 seconds) 2020-03-31 09:18:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:11 UTC)" (scheduled at 2020-03-31 09:18:01.440707+00:00) 2020-03-31 09:18:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:18:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:18:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:18:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:18:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:18:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:11 UTC)" executed successfully 2020-03-31 09:18:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:18:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:24.410927+00:00 (in 21.079911 seconds) 2020-03-31 09:18:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:33 UTC)" (scheduled at 2020-03-31 09:18:03.330112+00:00) 2020-03-31 09:18:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:18:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:24.410927+00:00 (in 21.079911 seconds) 2020-03-31 09:18:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:33 UTC)" (scheduled at 2020-03-31 09:18:03.330112+00:00) 2020-03-31 09:18:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 3, 331968), True, datetime.datetime(2020, 3, 31, 9, 18, 3, 331968), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:18:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 3, 331968), True, datetime.datetime(2020, 3, 31, 9, 18, 3, 331968), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:18:03,358 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:18:03,358 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:03,358 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:03,358 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:18:03,358 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:03,358 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:33 UTC)" executed successfully 2020-03-31 09:18:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:14.323352+00:00 (in 9.999191 seconds) notificationworker stdout | 2020-03-31 09:18:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:14.323352+00:00 (in 9.999191 seconds) 2020-03-31 09:18:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:14 UTC)" (scheduled at 2020-03-31 09:18:04.323352+00:00) 2020-03-31 09:18:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:18:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:14 UTC)" (scheduled at 2020-03-31 09:18:04.323352+00:00) 2020-03-31 09:18:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 4, 325277), True, datetime.datetime(2020, 3, 31, 9, 18, 4, 325277), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:18:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 4, 325277), True, datetime.datetime(2020, 3, 31, 9, 18, 4, 325277), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:18:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:18:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:18:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:14 UTC)" executed successfully 2020-03-31 09:18:04,446 [518] [DEBUG] [app] Starting request: urn:request:abdbe653-1cc9-48c2-8f9b-a6ce64bbb81e (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:04,446 [518] [DEBUG] [app] Starting request: urn:request:abdbe653-1cc9-48c2-8f9b-a6ce64bbb81e (/health/instance) 2020-03-31 09:18:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:04,454 [550] [DEBUG] [app] Starting request: urn:request:eb14a367-fa1c-4575-8131-1897f18b982b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:04,454 [550] [DEBUG] [app] Starting request: urn:request:eb14a367-fa1c-4575-8131-1897f18b982b (/v1/_internal_ping) 2020-03-31 09:18:04,455 [550] [DEBUG] [app] Ending request: urn:request:eb14a367-fa1c-4575-8131-1897f18b982b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:04,455 [550] [DEBUG] [app] Ending request: urn:request:eb14a367-fa1c-4575-8131-1897f18b982b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:18:04,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:18:04,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:18:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:04,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:04,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:04,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:04,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:04,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:04,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:04,491 [518] [DEBUG] [app] Starting request: urn:request:52f43990-4011-4387-80a8-3ebdf90edb9c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:04,491 [518] [DEBUG] [app] Starting request: urn:request:52f43990-4011-4387-80a8-3ebdf90edb9c (/_internal_ping) 2020-03-31 09:18:04,491 [518] [DEBUG] [app] Ending request: urn:request:52f43990-4011-4387-80a8-3ebdf90edb9c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:04,491 [518] [DEBUG] [app] Ending request: urn:request:52f43990-4011-4387-80a8-3ebdf90edb9c (/_internal_ping) 2020-03-31 09:18:04,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:18:04,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:18:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:18:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:18:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:04,504 [537] [DEBUG] [app] Starting request: urn:request:59fc2f4c-ee9c-4ebe-8ed4-f72e00a548a5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:04,504 [537] [DEBUG] [app] Starting request: urn:request:59fc2f4c-ee9c-4ebe-8ed4-f72e00a548a5 (/c1/_internal_ping) 2020-03-31 09:18:04,505 [537] [DEBUG] [app] Ending request: urn:request:59fc2f4c-ee9c-4ebe-8ed4-f72e00a548a5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:04,505 [537] [DEBUG] [app] Ending request: urn:request:59fc2f4c-ee9c-4ebe-8ed4-f72e00a548a5 (/c1/_internal_ping) 2020-03-31 09:18:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:04,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:18:04,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:04,507 [518] [DEBUG] [app] Ending request: urn:request:abdbe653-1cc9-48c2-8f9b-a6ce64bbb81e (/health/instance) 2020-03-31 09:18:04,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:04,507 [518] [DEBUG] [app] Ending request: urn:request:abdbe653-1cc9-48c2-8f9b-a6ce64bbb81e (/health/instance) 2020-03-31 09:18:04,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:04,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:18:04,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:18:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:18:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:16.924497+00:00 (in 9.999315 seconds) 2020-03-31 09:18:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:16 UTC)" (scheduled at 2020-03-31 09:18:06.924497+00:00) 2020-03-31 09:18:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:18:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:16.924497+00:00 (in 9.999315 seconds) 2020-03-31 09:18:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:16 UTC)" (scheduled at 2020-03-31 09:18:06.924497+00:00) 2020-03-31 09:18:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 6, 925952), True, datetime.datetime(2020, 3, 31, 9, 18, 6, 925952), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:18:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 6, 925952), True, datetime.datetime(2020, 3, 31, 9, 18, 6, 925952), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:18:06,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:18:06,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:18:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:06,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:16 UTC)" executed successfully 2020-03-31 09:18:08,273 [518] [DEBUG] [app] Starting request: urn:request:c73afb60-70c6-4fd9-9e0a-657364cb77f7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:08,273 [518] [DEBUG] [app] Starting request: urn:request:c73afb60-70c6-4fd9-9e0a-657364cb77f7 (/health/instance) 2020-03-31 09:18:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:08,279 [552] [DEBUG] [app] Starting request: urn:request:3da14292-2259-4b2b-8f7a-839795b36db0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:08,279 [552] [DEBUG] [app] Starting request: urn:request:3da14292-2259-4b2b-8f7a-839795b36db0 (/v1/_internal_ping) 2020-03-31 09:18:08,281 [552] [DEBUG] [app] Ending request: urn:request:3da14292-2259-4b2b-8f7a-839795b36db0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:08,281 [552] [DEBUG] [app] Ending request: urn:request:3da14292-2259-4b2b-8f7a-839795b36db0 (/v1/_internal_ping) 2020-03-31 09:18:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:18:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:08,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:18:08,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:08,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:08,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:08,287 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:08,287 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:08,288 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:08,288 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:08,315 [527] [DEBUG] [app] Starting request: urn:request:f5845155-ff1f-4356-9003-ae20c24fef36 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:08,315 [527] [DEBUG] [app] Starting request: urn:request:f5845155-ff1f-4356-9003-ae20c24fef36 (/_internal_ping) 2020-03-31 09:18:08,316 [527] [DEBUG] [app] Ending request: urn:request:f5845155-ff1f-4356-9003-ae20c24fef36 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:08,316 [527] [DEBUG] [app] Ending request: urn:request:f5845155-ff1f-4356-9003-ae20c24fef36 (/_internal_ping) 2020-03-31 09:18:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:08,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:18:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:18:08,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:08,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:08,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:08,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:08,331 [543] [DEBUG] [app] Starting request: urn:request:9358c030-c3b7-4a36-9ee0-765c8c24c97b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:08,331 [543] [DEBUG] [app] Starting request: urn:request:9358c030-c3b7-4a36-9ee0-765c8c24c97b (/c1/_internal_ping) 2020-03-31 09:18:08,333 [543] [DEBUG] [app] Ending request: urn:request:9358c030-c3b7-4a36-9ee0-765c8c24c97b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:08,333 [543] [DEBUG] [app] Ending request: urn:request:9358c030-c3b7-4a36-9ee0-765c8c24c97b (/c1/_internal_ping) 2020-03-31 09:18:08,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:08,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:08,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:18:08,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:08,335 [518] [DEBUG] [app] Ending request: urn:request:c73afb60-70c6-4fd9-9e0a-657364cb77f7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:08,335 [518] [DEBUG] [app] Ending request: urn:request:c73afb60-70c6-4fd9-9e0a-657364cb77f7 (/health/instance) 2020-03-31 09:18:08,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:08,335 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:18:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:18:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:18:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:21.440707+00:00 (in 9.998696 seconds) 2020-03-31 09:18:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:21 UTC)" (scheduled at 2020-03-31 09:18:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:18:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:21.440707+00:00 (in 9.998696 seconds) 2020-03-31 09:18:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:21 UTC)" (scheduled at 2020-03-31 09:18:11.440707+00:00) 2020-03-31 09:18:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:18:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:18:11,469 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:18:11,469 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:18:11,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:11,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:18:11,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:11,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:21 UTC)" executed successfully 2020-03-31 09:18:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:18:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:24.323352+00:00 (in 9.999275 seconds) 2020-03-31 09:18:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:24 UTC)" (scheduled at 2020-03-31 09:18:14.323352+00:00) 2020-03-31 09:18:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:18:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:24.323352+00:00 (in 9.999275 seconds) 2020-03-31 09:18:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:24 UTC)" (scheduled at 2020-03-31 09:18:14.323352+00:00) 2020-03-31 09:18:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 14, 324961), True, datetime.datetime(2020, 3, 31, 9, 18, 14, 324961), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:18:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 14, 324961), True, datetime.datetime(2020, 3, 31, 9, 18, 14, 324961), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:18:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:14,345 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:18:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:18:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:24 UTC)" executed successfully 2020-03-31 09:18:14,446 [518] [DEBUG] [app] Starting request: urn:request:bc228b4b-ff33-4473-8cd1-1c226e931e9e (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:14,446 [518] [DEBUG] [app] Starting request: urn:request:bc228b4b-ff33-4473-8cd1-1c226e931e9e (/health/instance) 2020-03-31 09:18:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:14,455 [547] [DEBUG] [app] Starting request: urn:request:f1850302-44c8-4de0-b3b4-a541d2ed5d18 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:14,455 [547] [DEBUG] [app] Starting request: urn:request:f1850302-44c8-4de0-b3b4-a541d2ed5d18 (/v1/_internal_ping) 2020-03-31 09:18:14,456 [547] [DEBUG] [app] Ending request: urn:request:f1850302-44c8-4de0-b3b4-a541d2ed5d18 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:14,456 [547] [DEBUG] [app] Ending request: urn:request:f1850302-44c8-4de0-b3b4-a541d2ed5d18 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:14,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:18:14,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:18:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:18:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:14,460 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:14,460 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:14,461 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:14,461 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:14,462 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:14,462 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:14,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:14,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:14,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:14,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:14,494 [518] [DEBUG] [app] Starting request: urn:request:3a6dcdc7-8bb2-46d4-bad2-62ebdea1fec1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:14,494 [518] [DEBUG] [app] Starting request: urn:request:3a6dcdc7-8bb2-46d4-bad2-62ebdea1fec1 (/_internal_ping) 2020-03-31 09:18:14,494 [518] [DEBUG] [app] Ending request: urn:request:3a6dcdc7-8bb2-46d4-bad2-62ebdea1fec1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:14,494 [518] [DEBUG] [app] Ending request: urn:request:3a6dcdc7-8bb2-46d4-bad2-62ebdea1fec1 (/_internal_ping) 2020-03-31 09:18:14,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:14,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:18:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:18:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:18:14,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:14,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:14,508 [543] [DEBUG] [app] Starting request: urn:request:a5685b05-99de-45fb-b961-741678026917 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:14,508 [543] [DEBUG] [app] Starting request: urn:request:a5685b05-99de-45fb-b961-741678026917 (/c1/_internal_ping) 2020-03-31 09:18:14,508 [543] [DEBUG] [app] Ending request: urn:request:a5685b05-99de-45fb-b961-741678026917 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:14,508 [543] [DEBUG] [app] Ending request: urn:request:a5685b05-99de-45fb-b961-741678026917 (/c1/_internal_ping) 2020-03-31 09:18:14,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:14,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:14,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:14,510 [518] [DEBUG] [app] Ending request: urn:request:bc228b4b-ff33-4473-8cd1-1c226e931e9e (/health/instance) 2020-03-31 09:18:14,510 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:18:14,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:14,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:18:14,510 [518] [DEBUG] [app] Ending request: urn:request:bc228b4b-ff33-4473-8cd1-1c226e931e9e (/health/instance) 2020-03-31 09:18:14,510 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:14,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:18:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:18:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:45.064514+00:00 (in 29.999225 seconds) 2020-03-31 09:18:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:45 UTC)" (scheduled at 2020-03-31 09:18:15.064514+00:00) gcworker stdout | 2020-03-31 09:18:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:45.064514+00:00 (in 29.999225 seconds) 2020-03-31 09:18:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:45 UTC)" (scheduled at 2020-03-31 09:18:15.064514+00:00) 2020-03-31 09:18:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436695, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:18:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436695, 1209600, 500, 1, 0]) 2020-03-31 09:18:15,089 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:18:15,089 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:18:15,089 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:18:15,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:15,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:18:15,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:45 UTC)" executed successfully 2020-03-31 09:18:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:18:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:26.924497+00:00 (in 9.999299 seconds) 2020-03-31 09:18:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:26 UTC)" (scheduled at 2020-03-31 09:18:16.924497+00:00) 2020-03-31 09:18:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:18:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:26.924497+00:00 (in 9.999299 seconds) 2020-03-31 09:18:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:26 UTC)" (scheduled at 2020-03-31 09:18:16.924497+00:00) 2020-03-31 09:18:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 16, 925813), True, datetime.datetime(2020, 3, 31, 9, 18, 16, 925813), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:18:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 16, 925813), True, datetime.datetime(2020, 3, 31, 9, 18, 16, 925813), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:18:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:18:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:18:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:26 UTC)" executed successfully 2020-03-31 09:18:18,273 [526] [DEBUG] [app] Starting request: urn:request:5cf63122-317f-4938-8acd-d74cf6da4956 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:18,273 [526] [DEBUG] [app] Starting request: urn:request:5cf63122-317f-4938-8acd-d74cf6da4956 (/health/instance) 2020-03-31 09:18:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:18,281 [553] [DEBUG] [app] Starting request: urn:request:89013333-8bbe-49e0-a132-8f7b09459057 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:18,281 [553] [DEBUG] [app] Starting request: urn:request:89013333-8bbe-49e0-a132-8f7b09459057 (/v1/_internal_ping) 2020-03-31 09:18:18,283 [553] [DEBUG] [app] Ending request: urn:request:89013333-8bbe-49e0-a132-8f7b09459057 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:18,283 [553] [DEBUG] [app] Ending request: urn:request:89013333-8bbe-49e0-a132-8f7b09459057 (/v1/_internal_ping) 2020-03-31 09:18:18,284 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:18,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-registry stdout | 2020-03-31 09:18:18,284 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:18,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:18:18,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-web stdout | 2020-03-31 09:18:18,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:18,287 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:18,287 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:18,288 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:18,288 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:18,289 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:18,289 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:18,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:18,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:18,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:18,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:18,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:18,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:18,320 [526] [DEBUG] [app] Starting request: urn:request:5fb25839-28e5-4fff-a042-c4597395e940 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:18,320 [526] [DEBUG] [app] Starting request: urn:request:5fb25839-28e5-4fff-a042-c4597395e940 (/_internal_ping) 2020-03-31 09:18:18,320 [526] [DEBUG] [app] Ending request: urn:request:5fb25839-28e5-4fff-a042-c4597395e940 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:18,320 [526] [DEBUG] [app] Ending request: urn:request:5fb25839-28e5-4fff-a042-c4597395e940 (/_internal_ping) 2020-03-31 09:18:18,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:18,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:18,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:18,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:18,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:18,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:18,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:18:18,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:18,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:18,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:18:18,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:18,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:18,331 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:18,331 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:18,333 [537] [DEBUG] [app] Starting request: urn:request:2334b198-19b2-4eb7-aff8-71a6fc28b526 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:18,333 [537] [DEBUG] [app] Starting request: urn:request:2334b198-19b2-4eb7-aff8-71a6fc28b526 (/c1/_internal_ping) 2020-03-31 09:18:18,334 [537] [DEBUG] [app] Ending request: urn:request:2334b198-19b2-4eb7-aff8-71a6fc28b526 (/c1/_internal_ping) 2020-03-31 09:18:18,335 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:18:18,334 [537] [DEBUG] [app] Ending request: urn:request:2334b198-19b2-4eb7-aff8-71a6fc28b526 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:18,335 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:18,337 [526] [DEBUG] [app] Ending request: urn:request:5cf63122-317f-4938-8acd-d74cf6da4956 (/health/instance) 2020-03-31 09:18:18,337 [526] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:18,338 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:18:18,337 [526] [DEBUG] [app] Ending request: urn:request:5cf63122-317f-4938-8acd-d74cf6da4956 (/health/instance) 2020-03-31 09:18:18,337 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:18,338 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:18,338 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-verbs stdout | 2020-03-31 09:18:18,338 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:18:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:31.440707+00:00 (in 9.999043 seconds) 2020-03-31 09:18:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:31 UTC)" (scheduled at 2020-03-31 09:18:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:18:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:31.440707+00:00 (in 9.999043 seconds) 2020-03-31 09:18:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:31 UTC)" (scheduled at 2020-03-31 09:18:21.440707+00:00) 2020-03-31 09:18:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:18:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:18:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:18:21,460 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:18:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:18:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:18:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:31 UTC)" executed successfully 2020-03-31 09:18:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:18:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:52.113221+00:00 (in 29.998779 seconds) 2020-03-31 09:18:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:52 UTC)" (scheduled at 2020-03-31 09:18:22.113221+00:00) securityworker stdout | 2020-03-31 09:18:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:52.113221+00:00 (in 29.998779 seconds) 2020-03-31 09:18:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:52 UTC)" (scheduled at 2020-03-31 09:18:22.113221+00:00) 2020-03-31 09:18:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:18:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:18:22,136 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:18:22,136 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:18:22,139 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:18:22,139 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:18:22,139 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:22,139 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:18:22,139 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:22,139 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:52 UTC)" executed successfully 2020-03-31 09:18:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:34.323352+00:00 (in 9.999147 seconds) 2020-03-31 09:18:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:34 UTC)" (scheduled at 2020-03-31 09:18:24.323352+00:00) 2020-03-31 09:18:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:18:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:18:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:34.323352+00:00 (in 9.999147 seconds) 2020-03-31 09:18:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:34 UTC)" (scheduled at 2020-03-31 09:18:24.323352+00:00) 2020-03-31 09:18:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 24, 325158), True, datetime.datetime(2020, 3, 31, 9, 18, 24, 325158), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:18:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 24, 325158), True, datetime.datetime(2020, 3, 31, 9, 18, 24, 325158), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:18:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:18:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:18:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:34 UTC)" executed successfully 2020-03-31 09:18:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:18:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:24 UTC)" (scheduled at 2020-03-31 09:18:24.410927+00:00) 2020-03-31 09:18:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:33.330112+00:00 (in 8.918381 seconds) security_notification_worker stdout | 2020-03-31 09:18:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:18:24 UTC)" (scheduled at 2020-03-31 09:18:24.410927+00:00) 2020-03-31 09:18:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:33.330112+00:00 (in 8.918381 seconds) 2020-03-31 09:18:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:18:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:24 UTC)" executed successfully 2020-03-31 09:18:24,445 [527] [DEBUG] [app] Starting request: urn:request:4d4d302a-66bf-4a93-b379-5a8c63dfb88c (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:24,445 [527] [DEBUG] [app] Starting request: urn:request:4d4d302a-66bf-4a93-b379-5a8c63dfb88c (/health/instance) 2020-03-31 09:18:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:24,453 [555] [DEBUG] [app] Starting request: urn:request:bcb248b4-6a18-4e9d-a445-529aaad64fd6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:24,453 [555] [DEBUG] [app] Starting request: urn:request:bcb248b4-6a18-4e9d-a445-529aaad64fd6 (/v1/_internal_ping) 2020-03-31 09:18:24,454 [555] [DEBUG] [app] Ending request: urn:request:bcb248b4-6a18-4e9d-a445-529aaad64fd6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:24,454 [555] [DEBUG] [app] Ending request: urn:request:bcb248b4-6a18-4e9d-a445-529aaad64fd6 (/v1/_internal_ping) 2020-03-31 09:18:24,454 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:18:24,454 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:24,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:24,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:18:24,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:18:24,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:24,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:24,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:24,458 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:24,458 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:24,459 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:24,459 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:24,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:24,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:24,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:24,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:24,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:24,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:24,488 [525] [DEBUG] [app] Starting request: urn:request:d895f1ef-25f4-4234-a61e-506a5e7c3560 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:24,488 [525] [DEBUG] [app] Starting request: urn:request:d895f1ef-25f4-4234-a61e-506a5e7c3560 (/_internal_ping) 2020-03-31 09:18:24,489 [525] [DEBUG] [app] Ending request: urn:request:d895f1ef-25f4-4234-a61e-506a5e7c3560 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:24,489 [525] [DEBUG] [app] Ending request: urn:request:d895f1ef-25f4-4234-a61e-506a5e7c3560 (/_internal_ping) 2020-03-31 09:18:24,490 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:24,490 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:24,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:24,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:18:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:18:24,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:24,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:24,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:18:24,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:18:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:24,503 [537] [DEBUG] [app] Starting request: urn:request:86219394-28b5-414f-ae93-3b3c0d60dbfa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:24,503 [537] [DEBUG] [app] Starting request: urn:request:86219394-28b5-414f-ae93-3b3c0d60dbfa (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:24,504 [537] [DEBUG] [app] Ending request: urn:request:86219394-28b5-414f-ae93-3b3c0d60dbfa (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:24,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:24,504 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:18:24,504 [537] [DEBUG] [app] Ending request: urn:request:86219394-28b5-414f-ae93-3b3c0d60dbfa (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:24,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:24,506 [527] [DEBUG] [app] Ending request: urn:request:4d4d302a-66bf-4a93-b379-5a8c63dfb88c (/health/instance) 2020-03-31 09:18:24,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:18:24,504 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:24,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:18:24,506 [527] [DEBUG] [app] Ending request: urn:request:4d4d302a-66bf-4a93-b379-5a8c63dfb88c (/health/instance) 2020-03-31 09:18:24,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:24,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:18:26,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:18:26,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:36.924497+00:00 (in 9.999350 seconds) 2020-03-31 09:18:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:36 UTC)" (scheduled at 2020-03-31 09:18:26.924497+00:00) 2020-03-31 09:18:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:18:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:36.924497+00:00 (in 9.999350 seconds) 2020-03-31 09:18:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:36 UTC)" (scheduled at 2020-03-31 09:18:26.924497+00:00) 2020-03-31 09:18:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 26, 925878), True, datetime.datetime(2020, 3, 31, 9, 18, 26, 925878), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:18:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 26, 925878), True, datetime.datetime(2020, 3, 31, 9, 18, 26, 925878), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:18:26,955 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:18:26,955 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:26,955 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:26,955 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:18:26,955 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:26,955 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:36 UTC)" executed successfully 2020-03-31 09:18:28,273 [518] [DEBUG] [app] Starting request: urn:request:cdf60b13-aec9-46e4-b788-2defcfe16e33 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:28,273 [518] [DEBUG] [app] Starting request: urn:request:cdf60b13-aec9-46e4-b788-2defcfe16e33 (/health/instance) 2020-03-31 09:18:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:28,280 [553] [DEBUG] [app] Starting request: urn:request:552a95e8-68a6-454b-8c29-1195a2f3fb73 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:28,280 [553] [DEBUG] [app] Starting request: urn:request:552a95e8-68a6-454b-8c29-1195a2f3fb73 (/v1/_internal_ping) 2020-03-31 09:18:28,281 [553] [DEBUG] [app] Ending request: urn:request:552a95e8-68a6-454b-8c29-1195a2f3fb73 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:28,281 [553] [DEBUG] [app] Ending request: urn:request:552a95e8-68a6-454b-8c29-1195a2f3fb73 (/v1/_internal_ping) 2020-03-31 09:18:28,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:18:28,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:18:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:28,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:28,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:28,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:28,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:28,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:28,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:28,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:28,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:28,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:28,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:28,309 [526] [DEBUG] [app] Starting request: urn:request:e52218d9-529b-4f1f-b7f3-d1725b0a703d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:28,309 [526] [DEBUG] [app] Starting request: urn:request:e52218d9-529b-4f1f-b7f3-d1725b0a703d (/_internal_ping) 2020-03-31 09:18:28,310 [526] [DEBUG] [app] Ending request: urn:request:e52218d9-529b-4f1f-b7f3-d1725b0a703d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:28,310 [526] [DEBUG] [app] Ending request: urn:request:e52218d9-529b-4f1f-b7f3-d1725b0a703d (/_internal_ping) 2020-03-31 09:18:28,310 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:28,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:28,310 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:28,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:28,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:18:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:18:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:28,324 [543] [DEBUG] [app] Starting request: urn:request:0a326600-8472-4314-a8db-1f4416f25581 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:28,324 [543] [DEBUG] [app] Starting request: urn:request:0a326600-8472-4314-a8db-1f4416f25581 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:28,326 [543] [DEBUG] [app] Ending request: urn:request:0a326600-8472-4314-a8db-1f4416f25581 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:28,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:18:28,326 [543] [DEBUG] [app] Ending request: urn:request:0a326600-8472-4314-a8db-1f4416f25581 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:18:28,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:28,328 [518] [DEBUG] [app] Ending request: urn:request:cdf60b13-aec9-46e4-b788-2defcfe16e33 (/health/instance) 2020-03-31 09:18:28,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:28,328 [518] [DEBUG] [app] Ending request: urn:request:cdf60b13-aec9-46e4-b788-2defcfe16e33 (/health/instance) 2020-03-31 09:18:28,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:28,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:18:28,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:18:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:18:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:00 UTC)" (scheduled at 2020-03-31 09:18:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:18:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:00 UTC)" (scheduled at 2020-03-31 09:18:30.337699+00:00) 2020-03-31 09:18:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 18, 30, 339458), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:18:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 18, 30, 339458), False, 50, 1, 0]) 2020-03-31 09:18:30,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:00.337699+00:00 (in 29.998996 seconds) buildlogsarchiver stdout | 2020-03-31 09:18:30,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:00.337699+00:00 (in 29.998996 seconds) 2020-03-31 09:18:30,359 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:18:30,359 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:18:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:18:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:00 UTC)" executed successfully 2020-03-31 09:18:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:18:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:41.440707+00:00 (in 9.998956 seconds) 2020-03-31 09:18:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:41 UTC)" (scheduled at 2020-03-31 09:18:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:18:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:41.440707+00:00 (in 9.998956 seconds) 2020-03-31 09:18:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:41 UTC)" (scheduled at 2020-03-31 09:18:31.440707+00:00) 2020-03-31 09:18:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:18:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:18:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:18:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:18:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:18:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:41 UTC)" executed successfully 2020-03-31 09:18:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:18:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:33 UTC)" (scheduled at 2020-03-31 09:18:33.330112+00:00) 2020-03-31 09:18:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:18:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:18:33 UTC)" (scheduled at 2020-03-31 09:18:33.330112+00:00) 2020-03-31 09:18:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:03.330112+00:00 (in 29.998455 seconds) security_notification_worker stdout | 2020-03-31 09:18:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:03.330112+00:00 (in 29.998455 seconds) 2020-03-31 09:18:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 33, 332546), True, datetime.datetime(2020, 3, 31, 9, 18, 33, 332546), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:18:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 33, 332546), True, datetime.datetime(2020, 3, 31, 9, 18, 33, 332546), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:18:33,354 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:18:33,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:33,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:33,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:18:33,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:33,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:03 UTC)" executed successfully 2020-03-31 09:18:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:18:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:43.539999+00:00 (in 10.172418 seconds) 2020-03-31 09:18:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:33 UTC)" (scheduled at 2020-03-31 09:18:33.366791+00:00) 2020-03-31 09:18:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:18:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:43.539999+00:00 (in 10.172418 seconds) 2020-03-31 09:18:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:33 UTC)" (scheduled at 2020-03-31 09:18:33.366791+00:00) 2020-03-31 09:18:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 33, 368493), True, datetime.datetime(2020, 3, 31, 9, 18, 33, 368493), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:18:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 33, 368493), True, datetime.datetime(2020, 3, 31, 9, 18, 33, 368493), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:18:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:18:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:18:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:33 UTC)" executed successfully 2020-03-31 09:18:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:18:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:44.323352+00:00 (in 9.998909 seconds) 2020-03-31 09:18:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:44 UTC)" (scheduled at 2020-03-31 09:18:34.323352+00:00) 2020-03-31 09:18:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:18:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:44.323352+00:00 (in 9.998909 seconds) 2020-03-31 09:18:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:44 UTC)" (scheduled at 2020-03-31 09:18:34.323352+00:00) 2020-03-31 09:18:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 34, 325404), True, datetime.datetime(2020, 3, 31, 9, 18, 34, 325404), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:18:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 34, 325404), True, datetime.datetime(2020, 3, 31, 9, 18, 34, 325404), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:18:34,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:18:34,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:18:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:44 UTC)" executed successfully 2020-03-31 09:18:34,446 [518] [DEBUG] [app] Starting request: urn:request:3c603039-8be1-4362-b301-5262b4d1fed8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:34,446 [518] [DEBUG] [app] Starting request: urn:request:3c603039-8be1-4362-b301-5262b4d1fed8 (/health/instance) 2020-03-31 09:18:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:34,453 [545] [DEBUG] [app] Starting request: urn:request:de38311a-5206-4f52-ad87-75cc48aacdfa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:34,453 [545] [DEBUG] [app] Starting request: urn:request:de38311a-5206-4f52-ad87-75cc48aacdfa (/v1/_internal_ping) 2020-03-31 09:18:34,454 [545] [DEBUG] [app] Ending request: urn:request:de38311a-5206-4f52-ad87-75cc48aacdfa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:34,454 [545] [DEBUG] [app] Ending request: urn:request:de38311a-5206-4f52-ad87-75cc48aacdfa (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:34,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:18:34,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:18:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:34,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:18:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:18:34,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:34,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:34,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:34,460 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:34,460 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:34,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:34,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:34,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:34,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:34,489 [521] [DEBUG] [app] Starting request: urn:request:d4c1d97e-fddf-4f1c-a9b3-b364bd05d9c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:34,489 [521] [DEBUG] [app] Starting request: urn:request:d4c1d97e-fddf-4f1c-a9b3-b364bd05d9c2 (/_internal_ping) 2020-03-31 09:18:34,490 [521] [DEBUG] [app] Ending request: urn:request:d4c1d97e-fddf-4f1c-a9b3-b364bd05d9c2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:34,490 [521] [DEBUG] [app] Ending request: urn:request:d4c1d97e-fddf-4f1c-a9b3-b364bd05d9c2 (/_internal_ping) 2020-03-31 09:18:34,491 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:34,491 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:18:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:18:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:34,504 [539] [DEBUG] [app] Starting request: urn:request:53f9d248-9d9c-46ca-95c8-e8c99e1d13d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:34,504 [539] [DEBUG] [app] Starting request: urn:request:53f9d248-9d9c-46ca-95c8-e8c99e1d13d6 (/c1/_internal_ping) 2020-03-31 09:18:34,505 [539] [DEBUG] [app] Ending request: urn:request:53f9d248-9d9c-46ca-95c8-e8c99e1d13d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:34,505 [539] [DEBUG] [app] Ending request: urn:request:53f9d248-9d9c-46ca-95c8-e8c99e1d13d6 (/c1/_internal_ping) 2020-03-31 09:18:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:34,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:18:34,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:34,507 [518] [DEBUG] [app] Ending request: urn:request:3c603039-8be1-4362-b301-5262b4d1fed8 (/health/instance) 2020-03-31 09:18:34,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:18:34,507 [518] [DEBUG] [app] Ending request: urn:request:3c603039-8be1-4362-b301-5262b4d1fed8 (/health/instance) 2020-03-31 09:18:34,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:18:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:18:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:38.010934+00:00 (in 1.085647 seconds) 2020-03-31 09:18:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:46 UTC)" (scheduled at 2020-03-31 09:18:36.924497+00:00) 2020-03-31 09:18:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:18:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:38.010934+00:00 (in 1.085647 seconds) 2020-03-31 09:18:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:46 UTC)" (scheduled at 2020-03-31 09:18:36.924497+00:00) 2020-03-31 09:18:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 36, 925995), True, datetime.datetime(2020, 3, 31, 9, 18, 36, 925995), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:18:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 36, 925995), True, datetime.datetime(2020, 3, 31, 9, 18, 36, 925995), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:18:36,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:18:36,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:18:36,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:46 UTC)" executed successfully 2020-03-31 09:18:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:18:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:46.924497+00:00 (in 8.912845 seconds) 2020-03-31 09:18:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:38 UTC)" (scheduled at 2020-03-31 09:18:38.010934+00:00) 2020-03-31 09:18:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:18:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:46.924497+00:00 (in 8.912845 seconds) 2020-03-31 09:18:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:38 UTC)" (scheduled at 2020-03-31 09:18:38.010934+00:00) 2020-03-31 09:18:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:18:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:38 UTC)" executed successfully 2020-03-31 09:18:38,272 [527] [DEBUG] [app] Starting request: urn:request:d10b0fc4-639f-457d-8ad1-e3585bfbeb74 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:38,272 [527] [DEBUG] [app] Starting request: urn:request:d10b0fc4-639f-457d-8ad1-e3585bfbeb74 (/health/instance) 2020-03-31 09:18:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:38,280 [542] [DEBUG] [app] Starting request: urn:request:66933f90-f09a-47a2-b60d-c33580a9861d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:38,280 [542] [DEBUG] [app] Starting request: urn:request:66933f90-f09a-47a2-b60d-c33580a9861d (/v1/_internal_ping) 2020-03-31 09:18:38,281 [542] [DEBUG] [app] Ending request: urn:request:66933f90-f09a-47a2-b60d-c33580a9861d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:38,281 [542] [DEBUG] [app] Ending request: urn:request:66933f90-f09a-47a2-b60d-c33580a9861d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:38,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:18:38,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:18:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:38,284 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:18:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:18:38,284 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:38,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:38,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:38,286 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:38,286 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:38,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:38,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:38,312 [527] [DEBUG] [app] Starting request: urn:request:c069dfbb-a08f-417d-8524-f3ad5eabacae (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:38,312 [527] [DEBUG] [app] Starting request: urn:request:c069dfbb-a08f-417d-8524-f3ad5eabacae (/_internal_ping) 2020-03-31 09:18:38,312 [527] [DEBUG] [app] Ending request: urn:request:c069dfbb-a08f-417d-8524-f3ad5eabacae (/_internal_ping) 2020-03-31 09:18:38,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:38,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:38,312 [527] [DEBUG] [app] Ending request: urn:request:c069dfbb-a08f-417d-8524-f3ad5eabacae (/_internal_ping) 2020-03-31 09:18:38,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:38,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:38,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:38,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:18:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:18:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:18:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:38,325 [541] [DEBUG] [app] Starting request: urn:request:2c6b0946-ccf4-4a7b-9f34-ca479d27973a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:38,325 [541] [DEBUG] [app] Starting request: urn:request:2c6b0946-ccf4-4a7b-9f34-ca479d27973a (/c1/_internal_ping) 2020-03-31 09:18:38,326 [541] [DEBUG] [app] Ending request: urn:request:2c6b0946-ccf4-4a7b-9f34-ca479d27973a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:38,326 [541] [DEBUG] [app] Ending request: urn:request:2c6b0946-ccf4-4a7b-9f34-ca479d27973a (/c1/_internal_ping) 2020-03-31 09:18:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:38,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:38,328 [527] [DEBUG] [app] Ending request: urn:request:d10b0fc4-639f-457d-8ad1-e3585bfbeb74 (/health/instance) 2020-03-31 09:18:38,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:18:38,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:38,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:18:38,328 [527] [DEBUG] [app] Ending request: urn:request:d10b0fc4-639f-457d-8ad1-e3585bfbeb74 (/health/instance) 2020-03-31 09:18:38,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:38,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:18:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:18:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:51.440707+00:00 (in 9.999115 seconds) 2020-03-31 09:18:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:51 UTC)" (scheduled at 2020-03-31 09:18:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:18:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:51.440707+00:00 (in 9.999115 seconds) 2020-03-31 09:18:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:51 UTC)" (scheduled at 2020-03-31 09:18:41.440707+00:00) 2020-03-31 09:18:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:18:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:18:41,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:18:41,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:18:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:18:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:51 UTC)" executed successfully 2020-03-31 09:18:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:18:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:33.366791+00:00 (in 49.825753 seconds) 2020-03-31 09:18:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:43 UTC)" (scheduled at 2020-03-31 09:18:43.539999+00:00) exportactionlogsworker stdout | 2020-03-31 09:18:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:33.366791+00:00 (in 49.825753 seconds) 2020-03-31 09:18:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:43 UTC)" (scheduled at 2020-03-31 09:18:43.539999+00:00) 2020-03-31 09:18:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:18:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:43 UTC)" executed successfully 2020-03-31 09:18:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:18:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:54 UTC)" (scheduled at 2020-03-31 09:18:44.323352+00:00) 2020-03-31 09:18:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:49.498966+00:00 (in 5.174514 seconds) 2020-03-31 09:18:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:18:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:54 UTC)" (scheduled at 2020-03-31 09:18:44.323352+00:00) 2020-03-31 09:18:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:49.498966+00:00 (in 5.174514 seconds) 2020-03-31 09:18:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 44, 325859), True, datetime.datetime(2020, 3, 31, 9, 18, 44, 325859), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:18:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 44, 325859), True, datetime.datetime(2020, 3, 31, 9, 18, 44, 325859), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:18:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:18:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:18:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:54 UTC)" executed successfully 2020-03-31 09:18:44,446 [518] [DEBUG] [app] Starting request: urn:request:497b39d9-5af9-4ea4-a878-f11e69bd927b (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:44,446 [518] [DEBUG] [app] Starting request: urn:request:497b39d9-5af9-4ea4-a878-f11e69bd927b (/health/instance) 2020-03-31 09:18:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:44,453 [538] [DEBUG] [app] Starting request: urn:request:7aec320b-bee9-4565-af1c-790805ed77f6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:44,453 [538] [DEBUG] [app] Starting request: urn:request:7aec320b-bee9-4565-af1c-790805ed77f6 (/v1/_internal_ping) 2020-03-31 09:18:44,454 [538] [DEBUG] [app] Ending request: urn:request:7aec320b-bee9-4565-af1c-790805ed77f6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:44,454 [538] [DEBUG] [app] Ending request: urn:request:7aec320b-bee9-4565-af1c-790805ed77f6 (/v1/_internal_ping) 2020-03-31 09:18:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:18:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:18:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:44,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:44,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:44,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:44,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:44,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:44,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:44,488 [518] [DEBUG] [app] Starting request: urn:request:f82f8542-f703-4817-8100-63c82762c914 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:44,488 [518] [DEBUG] [app] Starting request: urn:request:f82f8542-f703-4817-8100-63c82762c914 (/_internal_ping) 2020-03-31 09:18:44,489 [518] [DEBUG] [app] Ending request: urn:request:f82f8542-f703-4817-8100-63c82762c914 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:44,489 [518] [DEBUG] [app] Ending request: urn:request:f82f8542-f703-4817-8100-63c82762c914 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:18:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:18:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:18:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:18:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:18:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:44,502 [541] [DEBUG] [app] Starting request: urn:request:81f9bdff-8275-43c4-b155-653f95e2ccad (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:44,502 [541] [DEBUG] [app] Starting request: urn:request:81f9bdff-8275-43c4-b155-653f95e2ccad (/c1/_internal_ping) 2020-03-31 09:18:44,503 [541] [DEBUG] [app] Ending request: urn:request:81f9bdff-8275-43c4-b155-653f95e2ccad (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:44,503 [541] [DEBUG] [app] Ending request: urn:request:81f9bdff-8275-43c4-b155-653f95e2ccad (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:44,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:18:44,504 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:44,505 [518] [DEBUG] [app] Ending request: urn:request:497b39d9-5af9-4ea4-a878-f11e69bd927b (/health/instance) 2020-03-31 09:18:44,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:44,505 [518] [DEBUG] [app] Ending request: urn:request:497b39d9-5af9-4ea4-a878-f11e69bd927b (/health/instance) 2020-03-31 09:18:44,505 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:18:44,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:18:44,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:18:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:18:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:15.064514+00:00 (in 29.998714 seconds) 2020-03-31 09:18:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:15 UTC)" (scheduled at 2020-03-31 09:18:45.064514+00:00) gcworker stdout | 2020-03-31 09:18:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:15.064514+00:00 (in 29.998714 seconds) 2020-03-31 09:18:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:15 UTC)" (scheduled at 2020-03-31 09:18:45.064514+00:00) 2020-03-31 09:18:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436725, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:18:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436725, 1209600, 500, 1, 0]) 2020-03-31 09:18:45,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:18:45,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:18:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:18:45,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:15 UTC)" executed successfully 2020-03-31 09:18:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:46 UTC)" (scheduled at 2020-03-31 09:18:46.924497+00:00) 2020-03-31 09:18:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 46, 926518), True, datetime.datetime(2020, 3, 31, 9, 18, 46, 926518), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:18:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:18:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:46 UTC)" (scheduled at 2020-03-31 09:18:46.924497+00:00) 2020-03-31 09:18:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:46,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:56.924497+00:00 (in 9.998853 seconds) storagereplication stdout | 2020-03-31 09:18:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 46, 926518), True, datetime.datetime(2020, 3, 31, 9, 18, 46, 926518), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:18:46,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:56.924497+00:00 (in 9.998853 seconds) 2020-03-31 09:18:46,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:18:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:18:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:18:56 UTC)" executed successfully 2020-03-31 09:18:48,272 [527] [DEBUG] [app] Starting request: urn:request:ed880bbc-bb72-4582-8695-c7982a34b76f (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:48,272 [527] [DEBUG] [app] Starting request: urn:request:ed880bbc-bb72-4582-8695-c7982a34b76f (/health/instance) 2020-03-31 09:18:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:48,281 [546] [DEBUG] [app] Starting request: urn:request:0904b7f6-9f2f-460c-99f9-b8e805b95361 (/v1/_internal_ping) 2020-03-31 09:18:48,282 [546] [DEBUG] [app] Ending request: urn:request:0904b7f6-9f2f-460c-99f9-b8e805b95361 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:18:48,281 [546] [DEBUG] [app] Starting request: urn:request:0904b7f6-9f2f-460c-99f9-b8e805b95361 (/v1/_internal_ping) 2020-03-31 09:18:48,282 [546] [DEBUG] [app] Ending request: urn:request:0904b7f6-9f2f-460c-99f9-b8e805b95361 (/v1/_internal_ping) 2020-03-31 09:18:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:18:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:48,285 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:18:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:18:48,287 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:48,285 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:18:48,287 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:48,288 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:48,288 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:48,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:48,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:48,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:48,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:48,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:48,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:48,316 [518] [DEBUG] [app] Starting request: urn:request:5e79a85c-cd5a-4d7e-a849-e6fa5d7e42ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:48,316 [518] [DEBUG] [app] Starting request: urn:request:5e79a85c-cd5a-4d7e-a849-e6fa5d7e42ac (/_internal_ping) 2020-03-31 09:18:48,317 [518] [DEBUG] [app] Ending request: urn:request:5e79a85c-cd5a-4d7e-a849-e6fa5d7e42ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:48,317 [518] [DEBUG] [app] Ending request: urn:request:5e79a85c-cd5a-4d7e-a849-e6fa5d7e42ac (/_internal_ping) 2020-03-31 09:18:48,318 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:18:48,318 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:18:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:18:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:18:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:18:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:18:48,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:48,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:48,333 [541] [DEBUG] [app] Starting request: urn:request:70cebcfe-85e7-4422-8e99-178623ed512f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:48,333 [541] [DEBUG] [app] Starting request: urn:request:70cebcfe-85e7-4422-8e99-178623ed512f (/c1/_internal_ping) 2020-03-31 09:18:48,334 [541] [DEBUG] [app] Ending request: urn:request:70cebcfe-85e7-4422-8e99-178623ed512f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:48,334 [541] [DEBUG] [app] Ending request: urn:request:70cebcfe-85e7-4422-8e99-178623ed512f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:48,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:18:48,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:48,337 [527] [DEBUG] [app] Ending request: urn:request:ed880bbc-bb72-4582-8695-c7982a34b76f (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:48,337 [527] [DEBUG] [app] Ending request: urn:request:ed880bbc-bb72-4582-8695-c7982a34b76f (/health/instance) 2020-03-31 09:18:48,338 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:48,338 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:48,338 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:18:48,338 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:18:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:18:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:54.323352+00:00 (in 4.822790 seconds) 2020-03-31 09:18:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:49 UTC)" (scheduled at 2020-03-31 09:18:49.498966+00:00) 2020-03-31 09:18:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:18:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:54.323352+00:00 (in 4.822790 seconds) 2020-03-31 09:18:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:49 UTC)" (scheduled at 2020-03-31 09:18:49.498966+00:00) 2020-03-31 09:18:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:49 UTC)" executed successfully 2020-03-31 09:18:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:18:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:59.560326+00:00 (in 8.448878 seconds) 2020-03-31 09:18:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:51 UTC)" (scheduled at 2020-03-31 09:18:51.110610+00:00) 2020-03-31 09:18:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:18:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:18:59.560326+00:00 (in 8.448878 seconds) 2020-03-31 09:18:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:51 UTC)" (scheduled at 2020-03-31 09:18:51.110610+00:00) 2020-03-31 09:18:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 51, 112456), True, datetime.datetime(2020, 3, 31, 9, 18, 51, 112456), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:18:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 51, 112456), True, datetime.datetime(2020, 3, 31, 9, 18, 51, 112456), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:18:51,133 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:18:51,133 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:51,134 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:18:51,134 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:51 UTC)" executed successfully 2020-03-31 09:18:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:18:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:01.440707+00:00 (in 9.999182 seconds) 2020-03-31 09:18:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:01 UTC)" (scheduled at 2020-03-31 09:18:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:18:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:01.440707+00:00 (in 9.999182 seconds) 2020-03-31 09:18:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:01 UTC)" (scheduled at 2020-03-31 09:18:51.440707+00:00) 2020-03-31 09:18:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:18:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:18:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:18:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:18:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:18:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:01 UTC)" executed successfully 2020-03-31 09:18:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:18:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:22.113221+00:00 (in 29.999075 seconds) 2020-03-31 09:18:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:22 UTC)" (scheduled at 2020-03-31 09:18:52.113221+00:00) securityworker stdout | 2020-03-31 09:18:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:22.113221+00:00 (in 29.999075 seconds) 2020-03-31 09:18:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:22 UTC)" (scheduled at 2020-03-31 09:18:52.113221+00:00) 2020-03-31 09:18:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:18:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:18:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:18:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:18:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:18:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:18:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:18:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:22 UTC)" executed successfully 2020-03-31 09:18:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:18:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:04.323352+00:00 (in 9.998726 seconds) 2020-03-31 09:18:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:04 UTC)" (scheduled at 2020-03-31 09:18:54.323352+00:00) 2020-03-31 09:18:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:18:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:04.323352+00:00 (in 9.998726 seconds) 2020-03-31 09:18:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:04 UTC)" (scheduled at 2020-03-31 09:18:54.323352+00:00) 2020-03-31 09:18:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 54, 325559), True, datetime.datetime(2020, 3, 31, 9, 18, 54, 325559), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:18:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 54, 325559), True, datetime.datetime(2020, 3, 31, 9, 18, 54, 325559), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:18:54,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:18:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:18:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:04 UTC)" executed successfully 2020-03-31 09:18:54,445 [527] [DEBUG] [app] Starting request: urn:request:1dc70cb7-3d8a-440f-bda0-a24027b9eee2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:54,445 [527] [DEBUG] [app] Starting request: urn:request:1dc70cb7-3d8a-440f-bda0-a24027b9eee2 (/health/instance) 2020-03-31 09:18:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:54,453 [546] [DEBUG] [app] Starting request: urn:request:4bb0731c-403e-47c0-91c8-9ee0e5e97465 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:54,453 [546] [DEBUG] [app] Starting request: urn:request:4bb0731c-403e-47c0-91c8-9ee0e5e97465 (/v1/_internal_ping) 2020-03-31 09:18:54,454 [546] [DEBUG] [app] Ending request: urn:request:4bb0731c-403e-47c0-91c8-9ee0e5e97465 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:54,454 [546] [DEBUG] [app] Ending request: urn:request:4bb0731c-403e-47c0-91c8-9ee0e5e97465 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:18:54,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:18:54,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:18:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:54,457 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:18:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:18:54,457 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:54,458 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:54,458 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:54,459 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:54,459 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:54,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:54,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:54,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:54,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:54,490 [527] [DEBUG] [app] Starting request: urn:request:9059284f-17f6-4461-a8e5-0f0f6502b746 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:54,490 [527] [DEBUG] [app] Starting request: urn:request:9059284f-17f6-4461-a8e5-0f0f6502b746 (/_internal_ping) 2020-03-31 09:18:54,491 [527] [DEBUG] [app] Ending request: urn:request:9059284f-17f6-4461-a8e5-0f0f6502b746 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:18:54,491 [527] [DEBUG] [app] Ending request: urn:request:9059284f-17f6-4461-a8e5-0f0f6502b746 (/_internal_ping) 2020-03-31 09:18:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:18:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:54,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:54,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:18:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:18:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:18:54,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:54,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:54,505 [539] [DEBUG] [app] Starting request: urn:request:2b0f3bb9-d58d-4559-8168-f8e7d2b21982 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:54,505 [539] [DEBUG] [app] Starting request: urn:request:2b0f3bb9-d58d-4559-8168-f8e7d2b21982 (/c1/_internal_ping) 2020-03-31 09:18:54,506 [539] [DEBUG] [app] Ending request: urn:request:2b0f3bb9-d58d-4559-8168-f8e7d2b21982 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:54,506 [539] [DEBUG] [app] Ending request: urn:request:2b0f3bb9-d58d-4559-8168-f8e7d2b21982 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:54,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:54,506 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:54,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:18:54,506 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:54,508 [527] [DEBUG] [app] Ending request: urn:request:1dc70cb7-3d8a-440f-bda0-a24027b9eee2 (/health/instance) 2020-03-31 09:18:54,508 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:54,508 [527] [DEBUG] [app] Ending request: urn:request:1dc70cb7-3d8a-440f-bda0-a24027b9eee2 (/health/instance) 2020-03-31 09:18:54,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:18:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:18:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:18:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:06.924497+00:00 (in 9.995651 seconds) 2020-03-31 09:18:56,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:06 UTC)" (scheduled at 2020-03-31 09:18:56.924497+00:00) 2020-03-31 09:18:56,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:18:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:06.924497+00:00 (in 9.995651 seconds) 2020-03-31 09:18:56,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:06 UTC)" (scheduled at 2020-03-31 09:18:56.924497+00:00) 2020-03-31 09:18:56,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:18:56,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 56, 929873), True, datetime.datetime(2020, 3, 31, 9, 18, 56, 929873), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:18:56,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 18, 56, 929873), True, datetime.datetime(2020, 3, 31, 9, 18, 56, 929873), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:18:56,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:18:56,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:18:56,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:56,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:18:56,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:56,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:06 UTC)" executed successfully 2020-03-31 09:18:58,273 [518] [DEBUG] [app] Starting request: urn:request:e7ac12f3-f2a0-4540-b174-10a0e14be2d0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:18:58,273 [518] [DEBUG] [app] Starting request: urn:request:e7ac12f3-f2a0-4540-b174-10a0e14be2d0 (/health/instance) 2020-03-31 09:18:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:58,281 [555] [DEBUG] [app] Starting request: urn:request:08c8fdc6-b1b8-4ad8-8e99-4e8f9e7c9899 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:58,281 [555] [DEBUG] [app] Starting request: urn:request:08c8fdc6-b1b8-4ad8-8e99-4e8f9e7c9899 (/v1/_internal_ping) 2020-03-31 09:18:58,283 [555] [DEBUG] [app] Ending request: urn:request:08c8fdc6-b1b8-4ad8-8e99-4e8f9e7c9899 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:18:58,283 [555] [DEBUG] [app] Ending request: urn:request:08c8fdc6-b1b8-4ad8-8e99-4e8f9e7c9899 (/v1/_internal_ping) 2020-03-31 09:18:58,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:18:58,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:18:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:18:58,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:18:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:18:58,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:18:58,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:18:58,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:18:58,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:18:58,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:18:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:18:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:18:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:18:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:18:58,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:58,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:58,313 [527] [DEBUG] [app] Starting request: urn:request:bfc708b7-b389-44b2-9a0f-66da71357161 (/_internal_ping) 2020-03-31 09:18:58,314 [527] [DEBUG] [app] Ending request: urn:request:bfc708b7-b389-44b2-9a0f-66da71357161 (/_internal_ping) 2020-03-31 09:18:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:18:58,313 [527] [DEBUG] [app] Starting request: urn:request:bfc708b7-b389-44b2-9a0f-66da71357161 (/_internal_ping) 2020-03-31 09:18:58,314 [527] [DEBUG] [app] Ending request: urn:request:bfc708b7-b389-44b2-9a0f-66da71357161 (/_internal_ping) 2020-03-31 09:18:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:18:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:18:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:18:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:18:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:18:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:18:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:18:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:18:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:18:58,327 [539] [DEBUG] [app] Starting request: urn:request:8359554f-a959-43d6-a7d7-149a86f0d7d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:58,327 [539] [DEBUG] [app] Starting request: urn:request:8359554f-a959-43d6-a7d7-149a86f0d7d2 (/c1/_internal_ping) 2020-03-31 09:18:58,328 [539] [DEBUG] [app] Ending request: urn:request:8359554f-a959-43d6-a7d7-149a86f0d7d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:18:58,328 [539] [DEBUG] [app] Ending request: urn:request:8359554f-a959-43d6-a7d7-149a86f0d7d2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:58,329 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:18:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:18:58,329 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:18:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:18:58,330 [518] [DEBUG] [app] Ending request: urn:request:e7ac12f3-f2a0-4540-b174-10a0e14be2d0 (/health/instance) 2020-03-31 09:18:58,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:18:58,330 [518] [DEBUG] [app] Ending request: urn:request:e7ac12f3-f2a0-4540-b174-10a0e14be2d0 (/health/instance) 2020-03-31 09:18:58,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:18:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:18:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:18:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:18:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:18:59,563 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:18:59,563 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:18:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:51.110610+00:00 (in 51.546950 seconds) 2020-03-31 09:18:59,564 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:59 UTC)" (scheduled at 2020-03-31 09:18:59.560326+00:00) 2020-03-31 09:18:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:18:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:51.110610+00:00 (in 51.546950 seconds) 2020-03-31 09:18:59,564 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:59 UTC)" (scheduled at 2020-03-31 09:18:59.560326+00:00) 2020-03-31 09:18:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:18:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:19:59 UTC)" executed successfully 2020-03-31 09:19:00,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:19:00,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:30.337699+00:00 (in 29.996117 seconds) 2020-03-31 09:19:00,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:30 UTC)" (scheduled at 2020-03-31 09:19:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:19:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:30.337699+00:00 (in 29.996117 seconds) 2020-03-31 09:19:00,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:30 UTC)" (scheduled at 2020-03-31 09:19:00.337699+00:00) 2020-03-31 09:19:00,344 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 19, 0, 342393), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:19:00,344 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 19, 0, 342393), False, 50, 1, 0]) 2020-03-31 09:19:00,359 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:19:00,359 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:19:00,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:00,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:19:00,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:00,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:30 UTC)" executed successfully 2020-03-31 09:19:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:19:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:11.440707+00:00 (in 9.999349 seconds) 2020-03-31 09:19:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:11 UTC)" (scheduled at 2020-03-31 09:19:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:19:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:11.440707+00:00 (in 9.999349 seconds) 2020-03-31 09:19:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:11 UTC)" (scheduled at 2020-03-31 09:19:01.440707+00:00) 2020-03-31 09:19:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:19:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:19:01,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:19:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:19:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:19:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:11 UTC)" executed successfully 2020-03-31 09:19:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:19:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:23.410371+00:00 (in 20.078230 seconds) 2020-03-31 09:19:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:33 UTC)" (scheduled at 2020-03-31 09:19:03.330112+00:00) 2020-03-31 09:19:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:19:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:23.410371+00:00 (in 20.078230 seconds) 2020-03-31 09:19:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:33 UTC)" (scheduled at 2020-03-31 09:19:03.330112+00:00) 2020-03-31 09:19:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:03,336 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 3, 333307), True, datetime.datetime(2020, 3, 31, 9, 19, 3, 333307), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:19:03,336 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 3, 333307), True, datetime.datetime(2020, 3, 31, 9, 19, 3, 333307), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:19:03,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:19:03,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:19:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:33 UTC)" executed successfully 2020-03-31 09:19:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:19:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:14.323352+00:00 (in 9.999210 seconds) 2020-03-31 09:19:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:14 UTC)" (scheduled at 2020-03-31 09:19:04.323352+00:00) 2020-03-31 09:19:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:19:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:14.323352+00:00 (in 9.999210 seconds) 2020-03-31 09:19:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:14 UTC)" (scheduled at 2020-03-31 09:19:04.323352+00:00) 2020-03-31 09:19:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 4, 325104), True, datetime.datetime(2020, 3, 31, 9, 19, 4, 325104), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:19:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 4, 325104), True, datetime.datetime(2020, 3, 31, 9, 19, 4, 325104), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:19:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:04,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:19:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:19:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:14 UTC)" executed successfully 2020-03-31 09:19:04,446 [518] [DEBUG] [app] Starting request: urn:request:83898a32-a8ff-421b-8304-ffb0e77b286b (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:04,446 [518] [DEBUG] [app] Starting request: urn:request:83898a32-a8ff-421b-8304-ffb0e77b286b (/health/instance) 2020-03-31 09:19:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:04,454 [542] [DEBUG] [app] Starting request: urn:request:44572c4e-3a1e-452d-9d6e-b5ebf40e8885 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:04,454 [542] [DEBUG] [app] Starting request: urn:request:44572c4e-3a1e-452d-9d6e-b5ebf40e8885 (/v1/_internal_ping) 2020-03-31 09:19:04,455 [542] [DEBUG] [app] Ending request: urn:request:44572c4e-3a1e-452d-9d6e-b5ebf40e8885 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:04,455 [542] [DEBUG] [app] Ending request: urn:request:44572c4e-3a1e-452d-9d6e-b5ebf40e8885 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:04,456 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:19:04,456 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:04,458 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:19:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:19:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:04,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:04,458 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:04,460 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:04,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:04,460 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:04,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:04,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:04,495 [525] [DEBUG] [app] Starting request: urn:request:e03c02aa-0268-4e7b-8dc4-5597054faadc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:04,495 [525] [DEBUG] [app] Starting request: urn:request:e03c02aa-0268-4e7b-8dc4-5597054faadc (/_internal_ping) 2020-03-31 09:19:04,496 [525] [DEBUG] [app] Ending request: urn:request:e03c02aa-0268-4e7b-8dc4-5597054faadc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:04,496 [525] [DEBUG] [app] Ending request: urn:request:e03c02aa-0268-4e7b-8dc4-5597054faadc (/_internal_ping) 2020-03-31 09:19:04,497 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:04,497 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:19:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:19:04,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:04,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:04,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:04,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:04,511 [541] [DEBUG] [app] Starting request: urn:request:9f87c657-abd4-49fa-8028-3e58356e3a6a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:04,511 [541] [DEBUG] [app] Starting request: urn:request:9f87c657-abd4-49fa-8028-3e58356e3a6a (/c1/_internal_ping) 2020-03-31 09:19:04,511 [541] [DEBUG] [app] Ending request: urn:request:9f87c657-abd4-49fa-8028-3e58356e3a6a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:04,511 [541] [DEBUG] [app] Ending request: urn:request:9f87c657-abd4-49fa-8028-3e58356e3a6a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:04,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:04,512 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:04,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:19:04,512 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:04,514 [518] [DEBUG] [app] Ending request: urn:request:83898a32-a8ff-421b-8304-ffb0e77b286b (/health/instance) 2020-03-31 09:19:04,514 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:04,514 [518] [DEBUG] [app] Ending request: urn:request:83898a32-a8ff-421b-8304-ffb0e77b286b (/health/instance) 2020-03-31 09:19:04,514 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:04,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:19:04,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:19:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:16.924497+00:00 (in 9.999041 seconds) 2020-03-31 09:19:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:16 UTC)" (scheduled at 2020-03-31 09:19:06.924497+00:00) 2020-03-31 09:19:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:19:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:19:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:16.924497+00:00 (in 9.999041 seconds) 2020-03-31 09:19:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:16 UTC)" (scheduled at 2020-03-31 09:19:06.924497+00:00) 2020-03-31 09:19:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 6, 926394), True, datetime.datetime(2020, 3, 31, 9, 19, 6, 926394), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:19:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 6, 926394), True, datetime.datetime(2020, 3, 31, 9, 19, 6, 926394), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:19:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:19:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:06,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:19:06,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:06,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:16 UTC)" executed successfully 2020-03-31 09:19:08,273 [527] [DEBUG] [app] Starting request: urn:request:439b653a-2b6f-4d2d-a1d7-f43acd012eae (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:08,273 [527] [DEBUG] [app] Starting request: urn:request:439b653a-2b6f-4d2d-a1d7-f43acd012eae (/health/instance) 2020-03-31 09:19:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:08,281 [551] [DEBUG] [app] Starting request: urn:request:517e7afa-f349-40f1-967d-88c903b1d4fd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:08,281 [551] [DEBUG] [app] Starting request: urn:request:517e7afa-f349-40f1-967d-88c903b1d4fd (/v1/_internal_ping) 2020-03-31 09:19:08,282 [551] [DEBUG] [app] Ending request: urn:request:517e7afa-f349-40f1-967d-88c903b1d4fd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:08,282 [551] [DEBUG] [app] Ending request: urn:request:517e7afa-f349-40f1-967d-88c903b1d4fd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:08,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:19:08,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:08,286 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:19:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:19:08,286 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:08,287 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:08,287 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:08,288 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:08,288 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:08,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:08,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:08,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:08,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:08,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:08,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:08,312 [527] [DEBUG] [app] Starting request: urn:request:86176446-4095-434a-a7c5-d568d3d91b80 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:08,312 [527] [DEBUG] [app] Starting request: urn:request:86176446-4095-434a-a7c5-d568d3d91b80 (/_internal_ping) 2020-03-31 09:19:08,314 [527] [DEBUG] [app] Ending request: urn:request:86176446-4095-434a-a7c5-d568d3d91b80 (/_internal_ping) 2020-03-31 09:19:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:19:08,314 [527] [DEBUG] [app] Ending request: urn:request:86176446-4095-434a-a7c5-d568d3d91b80 (/_internal_ping) 2020-03-31 09:19:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:08,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:19:08,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:08,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:08,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:19:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:08,328 [541] [DEBUG] [app] Starting request: urn:request:91675243-c8e9-40f0-b521-9aaa0e9f17d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:08,328 [541] [DEBUG] [app] Starting request: urn:request:91675243-c8e9-40f0-b521-9aaa0e9f17d6 (/c1/_internal_ping) 2020-03-31 09:19:08,329 [541] [DEBUG] [app] Ending request: urn:request:91675243-c8e9-40f0-b521-9aaa0e9f17d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:08,329 [541] [DEBUG] [app] Ending request: urn:request:91675243-c8e9-40f0-b521-9aaa0e9f17d6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:08,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:08,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:08,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:19:08,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:08,331 [527] [DEBUG] [app] Ending request: urn:request:439b653a-2b6f-4d2d-a1d7-f43acd012eae (/health/instance) 2020-03-31 09:19:08,331 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:08,331 [527] [DEBUG] [app] Ending request: urn:request:439b653a-2b6f-4d2d-a1d7-f43acd012eae (/health/instance) 2020-03-31 09:19:08,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:08,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:19:08,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:19:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:19:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:21.440707+00:00 (in 9.999256 seconds) 2020-03-31 09:19:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:21 UTC)" (scheduled at 2020-03-31 09:19:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:19:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:21.440707+00:00 (in 9.999256 seconds) 2020-03-31 09:19:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:21 UTC)" (scheduled at 2020-03-31 09:19:11.440707+00:00) 2020-03-31 09:19:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:19:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:19:11,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:19:11,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:19:11,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:11,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:19:11,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:11,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:21 UTC)" executed successfully 2020-03-31 09:19:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:24.323352+00:00 (in 9.999210 seconds) 2020-03-31 09:19:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:24 UTC)" (scheduled at 2020-03-31 09:19:14.323352+00:00) 2020-03-31 09:19:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:19:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:19:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:24.323352+00:00 (in 9.999210 seconds) 2020-03-31 09:19:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:24 UTC)" (scheduled at 2020-03-31 09:19:14.323352+00:00) 2020-03-31 09:19:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 14, 325043), True, datetime.datetime(2020, 3, 31, 9, 19, 14, 325043), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:19:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 14, 325043), True, datetime.datetime(2020, 3, 31, 9, 19, 14, 325043), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:19:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:14,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:19:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:19:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:24 UTC)" executed successfully 2020-03-31 09:19:14,446 [518] [DEBUG] [app] Starting request: urn:request:0c0ad185-5dd2-47a2-b65c-890341ea59e8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:14,446 [518] [DEBUG] [app] Starting request: urn:request:0c0ad185-5dd2-47a2-b65c-890341ea59e8 (/health/instance) 2020-03-31 09:19:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:14,455 [555] [DEBUG] [app] Starting request: urn:request:ba25a067-9471-4d1a-a95b-7f419e87e1b1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:14,455 [555] [DEBUG] [app] Starting request: urn:request:ba25a067-9471-4d1a-a95b-7f419e87e1b1 (/v1/_internal_ping) 2020-03-31 09:19:14,456 [555] [DEBUG] [app] Ending request: urn:request:ba25a067-9471-4d1a-a95b-7f419e87e1b1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:14,456 [555] [DEBUG] [app] Ending request: urn:request:ba25a067-9471-4d1a-a95b-7f419e87e1b1 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:19:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:19:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:19:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:19:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:14,486 [518] [DEBUG] [app] Starting request: urn:request:860f7e01-953a-4059-9550-9ff0e6387700 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:14,486 [518] [DEBUG] [app] Starting request: urn:request:860f7e01-953a-4059-9550-9ff0e6387700 (/_internal_ping) 2020-03-31 09:19:14,487 [518] [DEBUG] [app] Ending request: urn:request:860f7e01-953a-4059-9550-9ff0e6387700 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:14,487 [518] [DEBUG] [app] Ending request: urn:request:860f7e01-953a-4059-9550-9ff0e6387700 (/_internal_ping) 2020-03-31 09:19:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:19:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:19:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:19:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:14,502 [539] [DEBUG] [app] Starting request: urn:request:fcbfc39a-c67a-48a7-9114-182b86a96dee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:14,502 [539] [DEBUG] [app] Starting request: urn:request:fcbfc39a-c67a-48a7-9114-182b86a96dee (/c1/_internal_ping) 2020-03-31 09:19:14,503 [539] [DEBUG] [app] Ending request: urn:request:fcbfc39a-c67a-48a7-9114-182b86a96dee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:14,503 [539] [DEBUG] [app] Ending request: urn:request:fcbfc39a-c67a-48a7-9114-182b86a96dee (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:14,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:19:14,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:14,505 [518] [DEBUG] [app] Ending request: urn:request:0c0ad185-5dd2-47a2-b65c-890341ea59e8 (/health/instance) 2020-03-31 09:19:14,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:14,505 [518] [DEBUG] [app] Ending request: urn:request:0c0ad185-5dd2-47a2-b65c-890341ea59e8 (/health/instance) 2020-03-31 09:19:14,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:19:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:19:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:19:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:45.064514+00:00 (in 29.999084 seconds) 2020-03-31 09:19:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:45 UTC)" (scheduled at 2020-03-31 09:19:15.064514+00:00) gcworker stdout | 2020-03-31 09:19:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:45.064514+00:00 (in 29.999084 seconds) 2020-03-31 09:19:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:45 UTC)" (scheduled at 2020-03-31 09:19:15.064514+00:00) 2020-03-31 09:19:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436755, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:19:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436755, 1209600, 500, 1, 0]) 2020-03-31 09:19:15,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:19:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:19:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:19:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:45 UTC)" executed successfully 2020-03-31 09:19:16,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:19:16,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:16,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:26.924497+00:00 (in 9.997278 seconds) 2020-03-31 09:19:16,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:26 UTC)" (scheduled at 2020-03-31 09:19:16.924497+00:00) 2020-03-31 09:19:16,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:19:16,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:26.924497+00:00 (in 9.997278 seconds) 2020-03-31 09:19:16,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:26 UTC)" (scheduled at 2020-03-31 09:19:16.924497+00:00) 2020-03-31 09:19:16,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:16,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 16, 928009), True, datetime.datetime(2020, 3, 31, 9, 19, 16, 928009), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:19:16,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 16, 928009), True, datetime.datetime(2020, 3, 31, 9, 19, 16, 928009), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:19:16,963 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:19:16,963 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:16,963 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:16,963 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:19:16,963 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:16,963 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:26 UTC)" executed successfully 2020-03-31 09:19:18,272 [518] [DEBUG] [app] Starting request: urn:request:d5b6b820-ee92-4fd5-a741-a4076dcadb79 (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:18,272 [518] [DEBUG] [app] Starting request: urn:request:d5b6b820-ee92-4fd5-a741-a4076dcadb79 (/health/instance) 2020-03-31 09:19:18,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:18,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:18,277 [538] [DEBUG] [app] Starting request: urn:request:e62105fe-2990-44fd-9dcc-cbe69904c256 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:18,277 [538] [DEBUG] [app] Starting request: urn:request:e62105fe-2990-44fd-9dcc-cbe69904c256 (/v1/_internal_ping) 2020-03-31 09:19:18,278 [538] [DEBUG] [app] Ending request: urn:request:e62105fe-2990-44fd-9dcc-cbe69904c256 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:18,278 [538] [DEBUG] [app] Ending request: urn:request:e62105fe-2990-44fd-9dcc-cbe69904c256 (/v1/_internal_ping) 2020-03-31 09:19:18,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:19:18,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:19:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:19:18,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:19:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:18,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:18,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:18,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:18,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:18,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:18,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:18,309 [518] [DEBUG] [app] Starting request: urn:request:a8f4e7fc-23f7-43d3-bb4d-a92999edc7c1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:18,309 [518] [DEBUG] [app] Starting request: urn:request:a8f4e7fc-23f7-43d3-bb4d-a92999edc7c1 (/_internal_ping) 2020-03-31 09:19:18,310 [518] [DEBUG] [app] Ending request: urn:request:a8f4e7fc-23f7-43d3-bb4d-a92999edc7c1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:18,310 [518] [DEBUG] [app] Ending request: urn:request:a8f4e7fc-23f7-43d3-bb4d-a92999edc7c1 (/_internal_ping) 2020-03-31 09:19:18,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:18,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:18,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:19:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:19:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:18,321 [539] [DEBUG] [app] Starting request: urn:request:9d47ce29-fee5-4e87-99cc-4fa7835ed354 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:18,321 [539] [DEBUG] [app] Starting request: urn:request:9d47ce29-fee5-4e87-99cc-4fa7835ed354 (/c1/_internal_ping) 2020-03-31 09:19:18,322 [539] [DEBUG] [app] Ending request: urn:request:9d47ce29-fee5-4e87-99cc-4fa7835ed354 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:18,322 [539] [DEBUG] [app] Ending request: urn:request:9d47ce29-fee5-4e87-99cc-4fa7835ed354 (/c1/_internal_ping) 2020-03-31 09:19:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:18,323 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:18,324 [518] [DEBUG] [app] Ending request: urn:request:d5b6b820-ee92-4fd5-a741-a4076dcadb79 (/health/instance) 2020-03-31 09:19:18,324 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:19:18,323 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:18,325 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.053 49 0.053) gunicorn-web stdout | 2020-03-31 09:19:18,324 [518] [DEBUG] [app] Ending request: urn:request:d5b6b820-ee92-4fd5-a741-a4076dcadb79 (/health/instance) 2020-03-31 09:19:18,324 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:18,325 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.053 49 0.053) 2020-03-31 09:19:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:19:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:31.440707+00:00 (in 9.998081 seconds) 2020-03-31 09:19:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:31 UTC)" (scheduled at 2020-03-31 09:19:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:19:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:31.440707+00:00 (in 9.998081 seconds) 2020-03-31 09:19:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:31 UTC)" (scheduled at 2020-03-31 09:19:21.440707+00:00) 2020-03-31 09:19:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:19:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:19:21,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:19:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:19:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:19:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:31 UTC)" executed successfully 2020-03-31 09:19:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:19:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:52.113221+00:00 (in 29.999118 seconds) 2020-03-31 09:19:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:52 UTC)" (scheduled at 2020-03-31 09:19:22.113221+00:00) securityworker stdout | 2020-03-31 09:19:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:52.113221+00:00 (in 29.999118 seconds) 2020-03-31 09:19:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:52 UTC)" (scheduled at 2020-03-31 09:19:22.113221+00:00) 2020-03-31 09:19:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:19:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:19:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:19:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:19:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:19:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:19:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:19:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:19:52 UTC)" executed successfully 2020-03-31 09:19:23,414 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:19:23,414 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:23,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:24.410927+00:00 (in 0.996513 seconds) 2020-03-31 09:19:23,414 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:23 UTC)" (scheduled at 2020-03-31 09:19:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:19:23,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:24.410927+00:00 (in 0.996513 seconds) 2020-03-31 09:19:23,414 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:23 UTC)" (scheduled at 2020-03-31 09:19:23.410371+00:00) 2020-03-31 09:19:23,416 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:19:23,416 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), u'security_notification/%']) 2020-03-31 09:19:23,431 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), True, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:19:23,431 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), True, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), 0, u'security_notification/%']) 2020-03-31 09:19:23,436 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), True, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:19:23,436 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), True, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 19, 23, 415158), u'security_notification/%']) 2020-03-31 09:19:23,439 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:19:23,439 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:23,440 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:19:23,440 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:23 UTC)" executed successfully 2020-03-31 09:19:23,443 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:19:23,443 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:23,444 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:19:23,444 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:23,445 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:19:23,445 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:19:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:19:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:34.323352+00:00 (in 9.999140 seconds) 2020-03-31 09:19:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:34 UTC)" (scheduled at 2020-03-31 09:19:24.323352+00:00) 2020-03-31 09:19:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:19:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:34.323352+00:00 (in 9.999140 seconds) 2020-03-31 09:19:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:34 UTC)" (scheduled at 2020-03-31 09:19:24.323352+00:00) 2020-03-31 09:19:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 24, 325061), True, datetime.datetime(2020, 3, 31, 9, 19, 24, 325061), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:19:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 24, 325061), True, datetime.datetime(2020, 3, 31, 9, 19, 24, 325061), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:19:24,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:19:24,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:19:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:34 UTC)" executed successfully 2020-03-31 09:19:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:33.330112+00:00 (in 8.918504 seconds) 2020-03-31 09:19:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:24 UTC)" (scheduled at 2020-03-31 09:19:24.410927+00:00) 2020-03-31 09:19:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:19:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:19:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:33.330112+00:00 (in 8.918504 seconds) 2020-03-31 09:19:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:24 UTC)" (scheduled at 2020-03-31 09:19:24.410927+00:00) 2020-03-31 09:19:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:24 UTC)" executed successfully 2020-03-31 09:19:24,446 [518] [DEBUG] [app] Starting request: urn:request:7db0a92e-83da-4ba3-a30d-4b7c2a9b3e3c (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:24,446 [518] [DEBUG] [app] Starting request: urn:request:7db0a92e-83da-4ba3-a30d-4b7c2a9b3e3c (/health/instance) 2020-03-31 09:19:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:24,454 [538] [DEBUG] [app] Starting request: urn:request:94eb546f-9f50-4283-b7e5-d9dacf2b4aee (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:24,454 [538] [DEBUG] [app] Starting request: urn:request:94eb546f-9f50-4283-b7e5-d9dacf2b4aee (/v1/_internal_ping) 2020-03-31 09:19:24,455 [538] [DEBUG] [app] Ending request: urn:request:94eb546f-9f50-4283-b7e5-d9dacf2b4aee (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:24,455 [538] [DEBUG] [app] Ending request: urn:request:94eb546f-9f50-4283-b7e5-d9dacf2b4aee (/v1/_internal_ping) 2020-03-31 09:19:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:19:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:19:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:19:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:24,489 [518] [DEBUG] [app] Starting request: urn:request:4b485392-6694-48f1-8062-4497991bd507 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:24,489 [518] [DEBUG] [app] Starting request: urn:request:4b485392-6694-48f1-8062-4497991bd507 (/_internal_ping) 2020-03-31 09:19:24,490 [518] [DEBUG] [app] Ending request: urn:request:4b485392-6694-48f1-8062-4497991bd507 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:24,490 [518] [DEBUG] [app] Ending request: urn:request:4b485392-6694-48f1-8062-4497991bd507 (/_internal_ping) 2020-03-31 09:19:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:19:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:19:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:24,505 [539] [DEBUG] [app] Starting request: urn:request:7ac73e49-45f6-4da9-8869-66ae64ddee20 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:24,505 [539] [DEBUG] [app] Starting request: urn:request:7ac73e49-45f6-4da9-8869-66ae64ddee20 (/c1/_internal_ping) 2020-03-31 09:19:24,506 [539] [DEBUG] [app] Ending request: urn:request:7ac73e49-45f6-4da9-8869-66ae64ddee20 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:24,506 [539] [DEBUG] [app] Ending request: urn:request:7ac73e49-45f6-4da9-8869-66ae64ddee20 (/c1/_internal_ping) 2020-03-31 09:19:24,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:24,506 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:24,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:24,508 [518] [DEBUG] [app] Ending request: urn:request:7db0a92e-83da-4ba3-a30d-4b7c2a9b3e3c (/health/instance) 2020-03-31 09:19:24,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:19:24,506 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:24,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:19:24,508 [518] [DEBUG] [app] Ending request: urn:request:7db0a92e-83da-4ba3-a30d-4b7c2a9b3e3c (/health/instance) 2020-03-31 09:19:24,509 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:19:24,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:19:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:19:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:36.924497+00:00 (in 9.999182 seconds) 2020-03-31 09:19:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:36 UTC)" (scheduled at 2020-03-31 09:19:26.924497+00:00) 2020-03-31 09:19:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:19:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:36.924497+00:00 (in 9.999182 seconds) 2020-03-31 09:19:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:36 UTC)" (scheduled at 2020-03-31 09:19:26.924497+00:00) 2020-03-31 09:19:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 26, 926024), True, datetime.datetime(2020, 3, 31, 9, 19, 26, 926024), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:19:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 26, 926024), True, datetime.datetime(2020, 3, 31, 9, 19, 26, 926024), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:19:26,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:19:26,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:19:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:36 UTC)" executed successfully 2020-03-31 09:19:28,273 [527] [DEBUG] [app] Starting request: urn:request:d9ff6b74-a318-4394-b672-8647a0af597d (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:28,273 [527] [DEBUG] [app] Starting request: urn:request:d9ff6b74-a318-4394-b672-8647a0af597d (/health/instance) 2020-03-31 09:19:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:28,280 [554] [DEBUG] [app] Starting request: urn:request:03bf6ba4-9119-436f-9608-c75a901b07c9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:28,280 [554] [DEBUG] [app] Starting request: urn:request:03bf6ba4-9119-436f-9608-c75a901b07c9 (/v1/_internal_ping) 2020-03-31 09:19:28,281 [554] [DEBUG] [app] Ending request: urn:request:03bf6ba4-9119-436f-9608-c75a901b07c9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:28,281 [554] [DEBUG] [app] Ending request: urn:request:03bf6ba4-9119-436f-9608-c75a901b07c9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:28,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:19:28,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:28,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:19:28,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:28,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:19:28,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:28,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:19:28,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:28,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:28,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:28,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:28,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:28,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:28,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:28,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:28,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:28,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:28,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:28,312 [518] [DEBUG] [app] Starting request: urn:request:aba888da-38c8-4093-a7b6-b76cb51a4c4c (/_internal_ping) 2020-03-31 09:19:28,313 [518] [DEBUG] [app] Ending request: urn:request:aba888da-38c8-4093-a7b6-b76cb51a4c4c (/_internal_ping) 2020-03-31 09:19:28,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:19:28,312 [518] [DEBUG] [app] Starting request: urn:request:aba888da-38c8-4093-a7b6-b76cb51a4c4c (/_internal_ping) 2020-03-31 09:19:28,313 [518] [DEBUG] [app] Ending request: urn:request:aba888da-38c8-4093-a7b6-b76cb51a4c4c (/_internal_ping) 2020-03-31 09:19:28,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:28,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:19:28,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:28,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:28,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:28,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:28,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:28,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:19:28,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:19:28,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:28,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:28,330 [541] [DEBUG] [app] Starting request: urn:request:7e1a90e8-526c-4841-b938-6d353b13b7ed (/c1/_internal_ping) 2020-03-31 09:19:28,331 [541] [DEBUG] [app] Ending request: urn:request:7e1a90e8-526c-4841-b938-6d353b13b7ed (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:28,330 [541] [DEBUG] [app] Starting request: urn:request:7e1a90e8-526c-4841-b938-6d353b13b7ed (/c1/_internal_ping) 2020-03-31 09:19:28,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:19:28,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:19:28,331 [541] [DEBUG] [app] Ending request: urn:request:7e1a90e8-526c-4841-b938-6d353b13b7ed (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:28,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:19:28,334 [527] [DEBUG] [app] Ending request: urn:request:d9ff6b74-a318-4394-b672-8647a0af597d (/health/instance) 2020-03-31 09:19:28,335 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:19:28,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:28,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:19:28,334 [527] [DEBUG] [app] Ending request: urn:request:d9ff6b74-a318-4394-b672-8647a0af597d (/health/instance) 2020-03-31 09:19:28,335 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:28,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:19:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:19:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:00.337699+00:00 (in 29.999194 seconds) 2020-03-31 09:19:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:00 UTC)" (scheduled at 2020-03-31 09:19:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:19:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:00.337699+00:00 (in 29.999194 seconds) 2020-03-31 09:19:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:00 UTC)" (scheduled at 2020-03-31 09:19:30.337699+00:00) 2020-03-31 09:19:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 19, 30, 339237), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:19:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 19, 30, 339237), False, 50, 1, 0]) 2020-03-31 09:19:30,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:19:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:19:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:19:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:00 UTC)" executed successfully 2020-03-31 09:19:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:19:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:41.440707+00:00 (in 9.999205 seconds) 2020-03-31 09:19:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:41 UTC)" (scheduled at 2020-03-31 09:19:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:19:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:41.440707+00:00 (in 9.999205 seconds) 2020-03-31 09:19:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:41 UTC)" (scheduled at 2020-03-31 09:19:31.440707+00:00) 2020-03-31 09:19:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:19:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:19:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:19:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:19:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:19:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:41 UTC)" executed successfully 2020-03-31 09:19:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:19:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:03.330112+00:00 (in 29.999271 seconds) 2020-03-31 09:19:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:03 UTC)" (scheduled at 2020-03-31 09:19:33.330112+00:00) 2020-03-31 09:19:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:19:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:03.330112+00:00 (in 29.999271 seconds) 2020-03-31 09:19:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:03 UTC)" (scheduled at 2020-03-31 09:19:33.330112+00:00) 2020-03-31 09:19:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 33, 331759), True, datetime.datetime(2020, 3, 31, 9, 19, 33, 331759), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:19:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 33, 331759), True, datetime.datetime(2020, 3, 31, 9, 19, 33, 331759), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:19:33,358 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:33,358 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:19:33,358 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:33,358 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:33,358 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:19:33,358 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:03 UTC)" executed successfully 2020-03-31 09:19:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:19:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:43.539999+00:00 (in 10.172137 seconds) exportactionlogsworker stdout | 2020-03-31 09:19:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:43.539999+00:00 (in 10.172137 seconds) 2020-03-31 09:19:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:33 UTC)" (scheduled at 2020-03-31 09:19:33.366791+00:00) 2020-03-31 09:19:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:19:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:33 UTC)" (scheduled at 2020-03-31 09:19:33.366791+00:00) 2020-03-31 09:19:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 33, 368876), True, datetime.datetime(2020, 3, 31, 9, 19, 33, 368876), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:19:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 33, 368876), True, datetime.datetime(2020, 3, 31, 9, 19, 33, 368876), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:19:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:19:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:19:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:33 UTC)" executed successfully 2020-03-31 09:19:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:19:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:44.323352+00:00 (in 9.999258 seconds) 2020-03-31 09:19:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:44 UTC)" (scheduled at 2020-03-31 09:19:34.323352+00:00) 2020-03-31 09:19:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:19:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:44.323352+00:00 (in 9.999258 seconds) 2020-03-31 09:19:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:44 UTC)" (scheduled at 2020-03-31 09:19:34.323352+00:00) 2020-03-31 09:19:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 34, 325274), True, datetime.datetime(2020, 3, 31, 9, 19, 34, 325274), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:19:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 34, 325274), True, datetime.datetime(2020, 3, 31, 9, 19, 34, 325274), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:19:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:19:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:19:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:44 UTC)" executed successfully 2020-03-31 09:19:34,446 [526] [DEBUG] [app] Starting request: urn:request:64fc8e08-3463-4291-b1c0-6473e821841e (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:34,446 [526] [DEBUG] [app] Starting request: urn:request:64fc8e08-3463-4291-b1c0-6473e821841e (/health/instance) 2020-03-31 09:19:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:34,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:34,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:34,454 [538] [DEBUG] [app] Starting request: urn:request:3bfd2c20-9b45-4765-80fa-b00967fe52ef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:34,454 [538] [DEBUG] [app] Starting request: urn:request:3bfd2c20-9b45-4765-80fa-b00967fe52ef (/v1/_internal_ping) 2020-03-31 09:19:34,455 [538] [DEBUG] [app] Ending request: urn:request:3bfd2c20-9b45-4765-80fa-b00967fe52ef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:34,455 [538] [DEBUG] [app] Ending request: urn:request:3bfd2c20-9b45-4765-80fa-b00967fe52ef (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:19:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:19:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:19:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:19:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:34,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:34,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:34,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:34,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:34,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:34,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:34,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:34,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:34,490 [518] [DEBUG] [app] Starting request: urn:request:a21fc26b-08b2-4384-8bf0-138c4f084724 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:34,490 [518] [DEBUG] [app] Starting request: urn:request:a21fc26b-08b2-4384-8bf0-138c4f084724 (/_internal_ping) 2020-03-31 09:19:34,491 [518] [DEBUG] [app] Ending request: urn:request:a21fc26b-08b2-4384-8bf0-138c4f084724 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:34,491 [518] [DEBUG] [app] Ending request: urn:request:a21fc26b-08b2-4384-8bf0-138c4f084724 (/_internal_ping) 2020-03-31 09:19:34,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:34,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:19:34,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:34,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:19:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:34,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:19:34,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:19:34,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:34,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:19:34,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:34,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:34,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:34,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:34,505 [541] [DEBUG] [app] Starting request: urn:request:97df8ff2-c6ee-49ba-b7d0-757591c14ca9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:34,505 [541] [DEBUG] [app] Starting request: urn:request:97df8ff2-c6ee-49ba-b7d0-757591c14ca9 (/c1/_internal_ping) 2020-03-31 09:19:34,505 [541] [DEBUG] [app] Ending request: urn:request:97df8ff2-c6ee-49ba-b7d0-757591c14ca9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:34,505 [541] [DEBUG] [app] Ending request: urn:request:97df8ff2-c6ee-49ba-b7d0-757591c14ca9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:34,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:34,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:19:34,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:34,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:34,507 [526] [DEBUG] [app] Ending request: urn:request:64fc8e08-3463-4291-b1c0-6473e821841e (/health/instance) 2020-03-31 09:19:34,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:34,507 [526] [DEBUG] [app] Ending request: urn:request:64fc8e08-3463-4291-b1c0-6473e821841e (/health/instance) 2020-03-31 09:19:34,508 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:34,508 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:19:34,508 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:19:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:19:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:38.010934+00:00 (in 1.085633 seconds) 2020-03-31 09:19:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:46 UTC)" (scheduled at 2020-03-31 09:19:36.924497+00:00) 2020-03-31 09:19:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:19:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:38.010934+00:00 (in 1.085633 seconds) 2020-03-31 09:19:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:46 UTC)" (scheduled at 2020-03-31 09:19:36.924497+00:00) 2020-03-31 09:19:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 36, 926324), True, datetime.datetime(2020, 3, 31, 9, 19, 36, 926324), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:19:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 36, 926324), True, datetime.datetime(2020, 3, 31, 9, 19, 36, 926324), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:19:37,088 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:19:37,088 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:37,089 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:37,089 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:19:37,089 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:37,089 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:46 UTC)" executed successfully 2020-03-31 09:19:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:19:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:46.924497+00:00 (in 8.912417 seconds) 2020-03-31 09:19:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:38 UTC)" (scheduled at 2020-03-31 09:19:38.010934+00:00) 2020-03-31 09:19:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:19:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:46.924497+00:00 (in 8.912417 seconds) 2020-03-31 09:19:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:38 UTC)" (scheduled at 2020-03-31 09:19:38.010934+00:00) 2020-03-31 09:19:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:38 UTC)" executed successfully 2020-03-31 09:19:38,273 [518] [DEBUG] [app] Starting request: urn:request:41220ea3-d03b-4d76-97d1-35f575567cde (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:38,273 [518] [DEBUG] [app] Starting request: urn:request:41220ea3-d03b-4d76-97d1-35f575567cde (/health/instance) 2020-03-31 09:19:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:38,280 [554] [DEBUG] [app] Starting request: urn:request:52d02075-0857-449c-a10c-583f5fa50c29 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:38,280 [554] [DEBUG] [app] Starting request: urn:request:52d02075-0857-449c-a10c-583f5fa50c29 (/v1/_internal_ping) 2020-03-31 09:19:38,281 [554] [DEBUG] [app] Ending request: urn:request:52d02075-0857-449c-a10c-583f5fa50c29 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:38,281 [554] [DEBUG] [app] Ending request: urn:request:52d02075-0857-449c-a10c-583f5fa50c29 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:38,282 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:38,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:19:38,282 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:38,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:19:38,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:38,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:19:38,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:19:38,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:38,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:38,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:38,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:38,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:38,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:38,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:38,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:38,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:38,313 [525] [DEBUG] [app] Starting request: urn:request:5e88dd61-4b9c-4ee6-ab29-e8faa2f56f58 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:38,313 [525] [DEBUG] [app] Starting request: urn:request:5e88dd61-4b9c-4ee6-ab29-e8faa2f56f58 (/_internal_ping) 2020-03-31 09:19:38,314 [525] [DEBUG] [app] Ending request: urn:request:5e88dd61-4b9c-4ee6-ab29-e8faa2f56f58 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:38,314 [525] [DEBUG] [app] Ending request: urn:request:5e88dd61-4b9c-4ee6-ab29-e8faa2f56f58 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:19:38,315 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:19:38,315 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:19:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:19:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:19:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:38,328 [537] [DEBUG] [app] Starting request: urn:request:4e149e48-9f5b-4605-a344-8cff412289c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:38,328 [537] [DEBUG] [app] Starting request: urn:request:4e149e48-9f5b-4605-a344-8cff412289c2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:38,329 [537] [DEBUG] [app] Ending request: urn:request:4e149e48-9f5b-4605-a344-8cff412289c2 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:38,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:19:38,329 [537] [DEBUG] [app] Ending request: urn:request:4e149e48-9f5b-4605-a344-8cff412289c2 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:38,331 [518] [DEBUG] [app] Ending request: urn:request:41220ea3-d03b-4d76-97d1-35f575567cde (/health/instance) 2020-03-31 09:19:38,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-verbs stdout | 2020-03-31 09:19:38,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:38,331 [518] [DEBUG] [app] Ending request: urn:request:41220ea3-d03b-4d76-97d1-35f575567cde (/health/instance) 2020-03-31 09:19:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:19:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:19:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:51.440707+00:00 (in 9.998889 seconds) 2020-03-31 09:19:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:51 UTC)" (scheduled at 2020-03-31 09:19:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:19:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:51.440707+00:00 (in 9.998889 seconds) 2020-03-31 09:19:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:51 UTC)" (scheduled at 2020-03-31 09:19:41.440707+00:00) 2020-03-31 09:19:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:19:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:19:41,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:19:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:19:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:19:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:51 UTC)" executed successfully 2020-03-31 09:19:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:19:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:14.539408+00:00 (in 30.998427 seconds) 2020-03-31 09:19:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:43 UTC)" (scheduled at 2020-03-31 09:19:43.539999+00:00) 2020-03-31 09:19:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:19:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:14.539408+00:00 (in 30.998427 seconds) 2020-03-31 09:19:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:43 UTC)" (scheduled at 2020-03-31 09:19:43.539999+00:00) 2020-03-31 09:19:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:43 UTC)" executed successfully 2020-03-31 09:19:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:19:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:49.498966+00:00 (in 5.174650 seconds) 2020-03-31 09:19:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:54 UTC)" (scheduled at 2020-03-31 09:19:44.323352+00:00) 2020-03-31 09:19:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:19:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:49.498966+00:00 (in 5.174650 seconds) 2020-03-31 09:19:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:54 UTC)" (scheduled at 2020-03-31 09:19:44.323352+00:00) 2020-03-31 09:19:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 44, 325038), True, datetime.datetime(2020, 3, 31, 9, 19, 44, 325038), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:19:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 44, 325038), True, datetime.datetime(2020, 3, 31, 9, 19, 44, 325038), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:19:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:19:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:19:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:54 UTC)" executed successfully 2020-03-31 09:19:44,446 [527] [DEBUG] [app] Starting request: urn:request:f5cff4ff-fd59-4b28-916b-ea7f09f46c82 (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:44,446 [527] [DEBUG] [app] Starting request: urn:request:f5cff4ff-fd59-4b28-916b-ea7f09f46c82 (/health/instance) 2020-03-31 09:19:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:44,454 [542] [DEBUG] [app] Starting request: urn:request:9e1472b3-e31c-4fc4-9ca8-06420138120b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:44,454 [542] [DEBUG] [app] Starting request: urn:request:9e1472b3-e31c-4fc4-9ca8-06420138120b (/v1/_internal_ping) 2020-03-31 09:19:44,455 [542] [DEBUG] [app] Ending request: urn:request:9e1472b3-e31c-4fc4-9ca8-06420138120b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:44,455 [542] [DEBUG] [app] Ending request: urn:request:9e1472b3-e31c-4fc4-9ca8-06420138120b (/v1/_internal_ping) 2020-03-31 09:19:44,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:19:44,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:44,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:19:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:44,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:19:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:44,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:44,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:19:44,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:44,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:44,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:44,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:44,488 [527] [DEBUG] [app] Starting request: urn:request:c746bfdc-8325-4708-a74b-7cef8869f201 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:44,488 [527] [DEBUG] [app] Starting request: urn:request:c746bfdc-8325-4708-a74b-7cef8869f201 (/_internal_ping) 2020-03-31 09:19:44,488 [527] [DEBUG] [app] Ending request: urn:request:c746bfdc-8325-4708-a74b-7cef8869f201 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:44,488 [527] [DEBUG] [app] Ending request: urn:request:c746bfdc-8325-4708-a74b-7cef8869f201 (/_internal_ping) 2020-03-31 09:19:44,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:44,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:44,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:44,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:19:44,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:44,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:44,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:44,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:44,504 [537] [DEBUG] [app] Starting request: urn:request:cb5e3f42-3599-4c4e-8f5a-78af96268311 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:44,504 [537] [DEBUG] [app] Starting request: urn:request:cb5e3f42-3599-4c4e-8f5a-78af96268311 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:44,505 [537] [DEBUG] [app] Ending request: urn:request:cb5e3f42-3599-4c4e-8f5a-78af96268311 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:44,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:44,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:19:44,505 [537] [DEBUG] [app] Ending request: urn:request:cb5e3f42-3599-4c4e-8f5a-78af96268311 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:44,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:44,507 [527] [DEBUG] [app] Ending request: urn:request:f5cff4ff-fd59-4b28-916b-ea7f09f46c82 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:19:44,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:44,508 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:44,507 [527] [DEBUG] [app] Ending request: urn:request:f5cff4ff-fd59-4b28-916b-ea7f09f46c82 (/health/instance) 2020-03-31 09:19:44,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:19:44,508 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:44,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:19:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:19:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:15.064514+00:00 (in 29.999149 seconds) 2020-03-31 09:19:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:15 UTC)" (scheduled at 2020-03-31 09:19:45.064514+00:00) gcworker stdout | 2020-03-31 09:19:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:15.064514+00:00 (in 29.999149 seconds) 2020-03-31 09:19:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:15 UTC)" (scheduled at 2020-03-31 09:19:45.064514+00:00) 2020-03-31 09:19:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436785, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:19:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436785, 1209600, 500, 1, 0]) 2020-03-31 09:19:45,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:19:45,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:19:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:19:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:15 UTC)" executed successfully 2020-03-31 09:19:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:19:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:56.924497+00:00 (in 9.999102 seconds) 2020-03-31 09:19:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:56 UTC)" (scheduled at 2020-03-31 09:19:46.924497+00:00) 2020-03-31 09:19:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:19:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:56.924497+00:00 (in 9.999102 seconds) 2020-03-31 09:19:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:56 UTC)" (scheduled at 2020-03-31 09:19:46.924497+00:00) 2020-03-31 09:19:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 46, 926358), True, datetime.datetime(2020, 3, 31, 9, 19, 46, 926358), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:19:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 46, 926358), True, datetime.datetime(2020, 3, 31, 9, 19, 46, 926358), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:19:46,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:19:46,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:46,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:19:46,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:19:56 UTC)" executed successfully 2020-03-31 09:19:48,273 [527] [DEBUG] [app] Starting request: urn:request:f64dd3a8-e48b-4cf2-a46e-0683ddd03308 (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:48,273 [527] [DEBUG] [app] Starting request: urn:request:f64dd3a8-e48b-4cf2-a46e-0683ddd03308 (/health/instance) 2020-03-31 09:19:48,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:48,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:48,279 [555] [DEBUG] [app] Starting request: urn:request:df1aa886-eca9-48a1-a702-c41d48e9d041 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:48,279 [555] [DEBUG] [app] Starting request: urn:request:df1aa886-eca9-48a1-a702-c41d48e9d041 (/v1/_internal_ping) 2020-03-31 09:19:48,280 [555] [DEBUG] [app] Ending request: urn:request:df1aa886-eca9-48a1-a702-c41d48e9d041 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:48,280 [555] [DEBUG] [app] Ending request: urn:request:df1aa886-eca9-48a1-a702-c41d48e9d041 (/v1/_internal_ping) 2020-03-31 09:19:48,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:19:48,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:48,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:48,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:19:48,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:48,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:48,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:19:48,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:48,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:48,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:48,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:48,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:48,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:48,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:48,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:48,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:48,310 [526] [DEBUG] [app] Starting request: urn:request:6996da77-f074-4d04-bd00-8ae81c640e05 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:48,310 [526] [DEBUG] [app] Starting request: urn:request:6996da77-f074-4d04-bd00-8ae81c640e05 (/_internal_ping) 2020-03-31 09:19:48,311 [526] [DEBUG] [app] Ending request: urn:request:6996da77-f074-4d04-bd00-8ae81c640e05 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:48,311 [526] [DEBUG] [app] Ending request: urn:request:6996da77-f074-4d04-bd00-8ae81c640e05 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:19:48,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:48,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:19:48,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:48,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:48,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:19:48,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:19:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:19:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:48,324 [537] [DEBUG] [app] Starting request: urn:request:4a27c6c9-9e0d-4ced-834a-83eb9ebac4ae (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:48,324 [537] [DEBUG] [app] Starting request: urn:request:4a27c6c9-9e0d-4ced-834a-83eb9ebac4ae (/c1/_internal_ping) 2020-03-31 09:19:48,324 [537] [DEBUG] [app] Ending request: urn:request:4a27c6c9-9e0d-4ced-834a-83eb9ebac4ae (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:48,324 [537] [DEBUG] [app] Ending request: urn:request:4a27c6c9-9e0d-4ced-834a-83eb9ebac4ae (/c1/_internal_ping) 2020-03-31 09:19:48,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:48,325 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:48,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:48,327 [527] [DEBUG] [app] Ending request: urn:request:f64dd3a8-e48b-4cf2-a46e-0683ddd03308 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:19:48,325 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:48,327 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:19:48,327 [527] [DEBUG] [app] Ending request: urn:request:f64dd3a8-e48b-4cf2-a46e-0683ddd03308 (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:48,327 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:19:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:19:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:49,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:54.323352+00:00 (in 4.823684 seconds) 2020-03-31 09:19:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:49 UTC)" (scheduled at 2020-03-31 09:19:49.498966+00:00) 2020-03-31 09:19:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:19:49,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:54.323352+00:00 (in 4.823684 seconds) 2020-03-31 09:19:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:49 UTC)" (scheduled at 2020-03-31 09:19:49.498966+00:00) 2020-03-31 09:19:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:19:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:49 UTC)" executed successfully 2020-03-31 09:19:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:19:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:53.556282+00:00 (in 2.444723 seconds) 2020-03-31 09:19:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:51 UTC)" (scheduled at 2020-03-31 09:19:51.110610+00:00) 2020-03-31 09:19:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:19:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:53.556282+00:00 (in 2.444723 seconds) 2020-03-31 09:19:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:51 UTC)" (scheduled at 2020-03-31 09:19:51.110610+00:00) 2020-03-31 09:19:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 51, 112581), True, datetime.datetime(2020, 3, 31, 9, 19, 51, 112581), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:19:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 51, 112581), True, datetime.datetime(2020, 3, 31, 9, 19, 51, 112581), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:19:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:19:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:19:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:51 UTC)" executed successfully 2020-03-31 09:19:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:01.440707+00:00 (in 9.999151 seconds) repositoryactioncounter stdout | 2020-03-31 09:19:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:01.440707+00:00 (in 9.999151 seconds) 2020-03-31 09:19:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:01 UTC)" (scheduled at 2020-03-31 09:19:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:19:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:01 UTC)" (scheduled at 2020-03-31 09:19:51.440707+00:00) 2020-03-31 09:19:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:19:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:19:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:19:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:19:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:19:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:01 UTC)" executed successfully 2020-03-31 09:19:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:19:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:22 UTC)" (scheduled at 2020-03-31 09:19:52.113221+00:00) securityworker stdout | 2020-03-31 09:19:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:22 UTC)" (scheduled at 2020-03-31 09:19:52.113221+00:00) 2020-03-31 09:19:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:22.113221+00:00 (in 29.998927 seconds) 2020-03-31 09:19:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:19:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:22.113221+00:00 (in 29.998927 seconds) 2020-03-31 09:19:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:19:52,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:19:52,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:19:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:19:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:19:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:19:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:22 UTC)" executed successfully 2020-03-31 09:19:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:19:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:59.560326+00:00 (in 6.003268 seconds) 2020-03-31 09:19:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:53 UTC)" (scheduled at 2020-03-31 09:19:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:19:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:19:59.560326+00:00 (in 6.003268 seconds) 2020-03-31 09:19:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:53 UTC)" (scheduled at 2020-03-31 09:19:53.556282+00:00) 2020-03-31 09:19:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:19:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), u'namespacegc/%']) 2020-03-31 09:19:53,572 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), True, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:19:53,572 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), True, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), 0, u'namespacegc/%']) 2020-03-31 09:19:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), True, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:19:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), True, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 19, 53, 557756), u'namespacegc/%']) 2020-03-31 09:19:53,582 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:19:53,582 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:53,583 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:19:53,583 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:24:53 UTC)" executed successfully 2020-03-31 09:19:53,585 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:19:53,585 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:19:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:53,586 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:19:53,586 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:19:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:19:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:04.323352+00:00 (in 9.999201 seconds) 2020-03-31 09:19:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:04 UTC)" (scheduled at 2020-03-31 09:19:54.323352+00:00) 2020-03-31 09:19:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:19:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:04.323352+00:00 (in 9.999201 seconds) 2020-03-31 09:19:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:04 UTC)" (scheduled at 2020-03-31 09:19:54.323352+00:00) 2020-03-31 09:19:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 54, 324991), True, datetime.datetime(2020, 3, 31, 9, 19, 54, 324991), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:19:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 54, 324991), True, datetime.datetime(2020, 3, 31, 9, 19, 54, 324991), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:19:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:19:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:19:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:04 UTC)" executed successfully 2020-03-31 09:19:54,446 [518] [DEBUG] [app] Starting request: urn:request:2bf98783-60b0-40e8-8808-88492fb15373 (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:54,446 [518] [DEBUG] [app] Starting request: urn:request:2bf98783-60b0-40e8-8808-88492fb15373 (/health/instance) 2020-03-31 09:19:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:54,453 [554] [DEBUG] [app] Starting request: urn:request:87fa28a2-d984-4b40-be3f-a5e4ace8989d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:54,453 [554] [DEBUG] [app] Starting request: urn:request:87fa28a2-d984-4b40-be3f-a5e4ace8989d (/v1/_internal_ping) 2020-03-31 09:19:54,455 [554] [DEBUG] [app] Ending request: urn:request:87fa28a2-d984-4b40-be3f-a5e4ace8989d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:54,455 [554] [DEBUG] [app] Ending request: urn:request:87fa28a2-d984-4b40-be3f-a5e4ace8989d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:54,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:19:54,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:19:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:19:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:54,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:19:54,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:54,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:54,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:54,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:54,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:54,485 [518] [DEBUG] [app] Starting request: urn:request:4aa7c1ef-4f4d-41bd-902b-8aedd7aa362b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:54,485 [518] [DEBUG] [app] Starting request: urn:request:4aa7c1ef-4f4d-41bd-902b-8aedd7aa362b (/_internal_ping) 2020-03-31 09:19:54,486 [518] [DEBUG] [app] Ending request: urn:request:4aa7c1ef-4f4d-41bd-902b-8aedd7aa362b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:54,486 [518] [DEBUG] [app] Ending request: urn:request:4aa7c1ef-4f4d-41bd-902b-8aedd7aa362b (/_internal_ping) 2020-03-31 09:19:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:19:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:19:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:54,498 [539] [DEBUG] [app] Starting request: urn:request:c0846154-2330-4972-93bc-5de7eaa95902 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:54,498 [539] [DEBUG] [app] Starting request: urn:request:c0846154-2330-4972-93bc-5de7eaa95902 (/c1/_internal_ping) 2020-03-31 09:19:54,499 [539] [DEBUG] [app] Ending request: urn:request:c0846154-2330-4972-93bc-5de7eaa95902 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:54,499 [539] [DEBUG] [app] Ending request: urn:request:c0846154-2330-4972-93bc-5de7eaa95902 (/c1/_internal_ping) 2020-03-31 09:19:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:54,500 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:19:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:54,501 [518] [DEBUG] [app] Ending request: urn:request:2bf98783-60b0-40e8-8808-88492fb15373 (/health/instance) 2020-03-31 09:19:54,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:19:54,500 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:19:54,501 [518] [DEBUG] [app] Ending request: urn:request:2bf98783-60b0-40e8-8808-88492fb15373 (/health/instance) 2020-03-31 09:19:54,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:19:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:19:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:06.924497+00:00 (in 9.995892 seconds) 2020-03-31 09:19:56,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:06 UTC)" (scheduled at 2020-03-31 09:19:56.924497+00:00) 2020-03-31 09:19:56,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:19:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:06.924497+00:00 (in 9.995892 seconds) 2020-03-31 09:19:56,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:06 UTC)" (scheduled at 2020-03-31 09:19:56.924497+00:00) 2020-03-31 09:19:56,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:19:56,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 56, 929715), True, datetime.datetime(2020, 3, 31, 9, 19, 56, 929715), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:19:56,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 19, 56, 929715), True, datetime.datetime(2020, 3, 31, 9, 19, 56, 929715), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:19:56,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:19:56,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:19:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:19:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:19:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:06 UTC)" executed successfully 2020-03-31 09:19:58,273 [527] [DEBUG] [app] Starting request: urn:request:f5416b03-d97e-4c1b-9b52-2bc1362f1018 (/health/instance) gunicorn-web stdout | 2020-03-31 09:19:58,273 [527] [DEBUG] [app] Starting request: urn:request:f5416b03-d97e-4c1b-9b52-2bc1362f1018 (/health/instance) 2020-03-31 09:19:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:58,280 [552] [DEBUG] [app] Starting request: urn:request:5fd58141-59c2-4c03-8b88-97516a20d1b7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:58,280 [552] [DEBUG] [app] Starting request: urn:request:5fd58141-59c2-4c03-8b88-97516a20d1b7 (/v1/_internal_ping) 2020-03-31 09:19:58,281 [552] [DEBUG] [app] Ending request: urn:request:5fd58141-59c2-4c03-8b88-97516a20d1b7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:19:58,281 [552] [DEBUG] [app] Ending request: urn:request:5fd58141-59c2-4c03-8b88-97516a20d1b7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:19:58,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:19:58,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:19:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:19:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:19:58,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:19:58,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:19:58,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:19:58,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:19:58,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:19:58,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:19:58,300 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:19:58,300 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:19:58,323 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:19:58,323 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:19:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:58,328 [527] [DEBUG] [app] Starting request: urn:request:3d3eea46-026f-44d3-820d-8db5ac7318c7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:58,328 [527] [DEBUG] [app] Starting request: urn:request:3d3eea46-026f-44d3-820d-8db5ac7318c7 (/_internal_ping) 2020-03-31 09:19:58,329 [527] [DEBUG] [app] Ending request: urn:request:3d3eea46-026f-44d3-820d-8db5ac7318c7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:58,329 [527] [DEBUG] [app] Ending request: urn:request:3d3eea46-026f-44d3-820d-8db5ac7318c7 (/_internal_ping) 2020-03-31 09:19:58,330 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:58,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:19:58,330 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:58,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:19:58,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:58,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:58,334 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:19:58,334 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:19:58,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:58,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:19:58,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:19:58,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:19:58,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:19:58,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:19:58,342 [539] [DEBUG] [app] Starting request: urn:request:cf3c4dd0-28e4-4d0d-95d1-8e9e8e3fdf34 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:19:58,342 [539] [DEBUG] [app] Starting request: urn:request:cf3c4dd0-28e4-4d0d-95d1-8e9e8e3fdf34 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:58,343 [539] [DEBUG] [app] Ending request: urn:request:cf3c4dd0-28e4-4d0d-95d1-8e9e8e3fdf34 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:19:58,343 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:58,343 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:19:58,343 [539] [DEBUG] [app] Ending request: urn:request:cf3c4dd0-28e4-4d0d-95d1-8e9e8e3fdf34 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:19:58,343 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:19:58,345 [527] [DEBUG] [app] Ending request: urn:request:f5416b03-d97e-4c1b-9b52-2bc1362f1018 (/health/instance) 2020-03-31 09:19:58,346 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:19:58,343 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:19:58,346 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) gunicorn-web stdout | 2020-03-31 09:19:58,345 [527] [DEBUG] [app] Ending request: urn:request:f5416b03-d97e-4c1b-9b52-2bc1362f1018 (/health/instance) 2020-03-31 09:19:58,346 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:19:58,346 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:19:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:19:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) 2020-03-31 09:19:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:19:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:19:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:51.110610+00:00 (in 51.549317 seconds) 2020-03-31 09:19:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:59 UTC)" (scheduled at 2020-03-31 09:19:59.560326+00:00) 2020-03-31 09:19:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:19:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:51.110610+00:00 (in 51.549317 seconds) 2020-03-31 09:19:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:59 UTC)" (scheduled at 2020-03-31 09:19:59.560326+00:00) 2020-03-31 09:19:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:19:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:59 UTC)" executed successfully 2020-03-31 09:20:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:20:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:30.337699+00:00 (in 29.999098 seconds) 2020-03-31 09:20:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:30 UTC)" (scheduled at 2020-03-31 09:20:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:20:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:30.337699+00:00 (in 29.999098 seconds) 2020-03-31 09:20:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:30 UTC)" (scheduled at 2020-03-31 09:20:00.337699+00:00) 2020-03-31 09:20:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 20, 0, 339477), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:20:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 20, 0, 339477), False, 50, 1, 0]) 2020-03-31 09:20:00,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:20:00,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:20:00,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:20:00,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:30 UTC)" executed successfully 2020-03-31 09:20:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:20:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:11.440707+00:00 (in 9.999162 seconds) 2020-03-31 09:20:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:11 UTC)" (scheduled at 2020-03-31 09:20:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:20:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:11.440707+00:00 (in 9.999162 seconds) 2020-03-31 09:20:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:11 UTC)" (scheduled at 2020-03-31 09:20:01.440707+00:00) 2020-03-31 09:20:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:20:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:20:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:20:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:20:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:20:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:11 UTC)" executed successfully 2020-03-31 09:20:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:20:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:24.410927+00:00 (in 21.079885 seconds) 2020-03-31 09:20:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:33 UTC)" (scheduled at 2020-03-31 09:20:03.330112+00:00) 2020-03-31 09:20:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:20:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:24.410927+00:00 (in 21.079885 seconds) 2020-03-31 09:20:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:33 UTC)" (scheduled at 2020-03-31 09:20:03.330112+00:00) 2020-03-31 09:20:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 3, 331878), True, datetime.datetime(2020, 3, 31, 9, 20, 3, 331878), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:20:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 3, 331878), True, datetime.datetime(2020, 3, 31, 9, 20, 3, 331878), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:20:03,354 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:20:03,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:20:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:33 UTC)" executed successfully 2020-03-31 09:20:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:20:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:14.323352+00:00 (in 9.999185 seconds) 2020-03-31 09:20:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:14 UTC)" (scheduled at 2020-03-31 09:20:04.323352+00:00) 2020-03-31 09:20:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:20:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:14.323352+00:00 (in 9.999185 seconds) 2020-03-31 09:20:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:14 UTC)" (scheduled at 2020-03-31 09:20:04.323352+00:00) 2020-03-31 09:20:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 4, 325227), True, datetime.datetime(2020, 3, 31, 9, 20, 4, 325227), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:20:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 4, 325227), True, datetime.datetime(2020, 3, 31, 9, 20, 4, 325227), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:20:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:20:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:20:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:14 UTC)" executed successfully 2020-03-31 09:20:04,446 [526] [DEBUG] [app] Starting request: urn:request:fe547904-60fb-4714-8fef-312b0b784b34 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:04,446 [526] [DEBUG] [app] Starting request: urn:request:fe547904-60fb-4714-8fef-312b0b784b34 (/health/instance) 2020-03-31 09:20:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:04,449 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:04,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:04,453 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:04,454 [545] [DEBUG] [app] Starting request: urn:request:3bab7c3f-1ee3-4b7c-81ba-e6416066c1bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:04,454 [545] [DEBUG] [app] Starting request: urn:request:3bab7c3f-1ee3-4b7c-81ba-e6416066c1bd (/v1/_internal_ping) 2020-03-31 09:20:04,456 [545] [DEBUG] [app] Ending request: urn:request:3bab7c3f-1ee3-4b7c-81ba-e6416066c1bd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:04,456 [545] [DEBUG] [app] Ending request: urn:request:3bab7c3f-1ee3-4b7c-81ba-e6416066c1bd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:04,456 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:04,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:20:04,456 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:04,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:20:04,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:04,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:04,459 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:04,459 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:04,460 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:04,460 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:04,461 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:04,461 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:04,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:04,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:04,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:04,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:04,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:04,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:04,492 [526] [DEBUG] [app] Starting request: urn:request:9f2a54e2-627f-4384-a78e-0af85d53fbbf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:04,492 [526] [DEBUG] [app] Starting request: urn:request:9f2a54e2-627f-4384-a78e-0af85d53fbbf (/_internal_ping) 2020-03-31 09:20:04,493 [526] [DEBUG] [app] Ending request: urn:request:9f2a54e2-627f-4384-a78e-0af85d53fbbf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:04,493 [526] [DEBUG] [app] Ending request: urn:request:9f2a54e2-627f-4384-a78e-0af85d53fbbf (/_internal_ping) 2020-03-31 09:20:04,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:04,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:04,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:04,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:04,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:04,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:20:04,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:04,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:04,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:04,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:04,506 [539] [DEBUG] [app] Starting request: urn:request:d4632d4b-4927-4e66-aee4-7e22826320f9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:04,506 [539] [DEBUG] [app] Starting request: urn:request:d4632d4b-4927-4e66-aee4-7e22826320f9 (/c1/_internal_ping) 2020-03-31 09:20:04,506 [539] [DEBUG] [app] Ending request: urn:request:d4632d4b-4927-4e66-aee4-7e22826320f9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:04,506 [539] [DEBUG] [app] Ending request: urn:request:d4632d4b-4927-4e66-aee4-7e22826320f9 (/c1/_internal_ping) 2020-03-31 09:20:04,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:04,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:04,507 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:04,508 [526] [DEBUG] [app] Ending request: urn:request:fe547904-60fb-4714-8fef-312b0b784b34 (/health/instance) 2020-03-31 09:20:04,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:20:04,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:04,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:20:04,508 [526] [DEBUG] [app] Ending request: urn:request:fe547904-60fb-4714-8fef-312b0b784b34 (/health/instance) 2020-03-31 09:20:04,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:04,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:20:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:20:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:16.924497+00:00 (in 9.998604 seconds) 2020-03-31 09:20:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:16 UTC)" (scheduled at 2020-03-31 09:20:06.924497+00:00) 2020-03-31 09:20:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:20:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:16.924497+00:00 (in 9.998604 seconds) 2020-03-31 09:20:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:16 UTC)" (scheduled at 2020-03-31 09:20:06.924497+00:00) 2020-03-31 09:20:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 6, 926740), True, datetime.datetime(2020, 3, 31, 9, 20, 6, 926740), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:20:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 6, 926740), True, datetime.datetime(2020, 3, 31, 9, 20, 6, 926740), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:20:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:20:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:20:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:16 UTC)" executed successfully 2020-03-31 09:20:08,273 [518] [DEBUG] [app] Starting request: urn:request:b0a6a0ab-c78f-4872-879a-456f5ecf2596 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:08,273 [518] [DEBUG] [app] Starting request: urn:request:b0a6a0ab-c78f-4872-879a-456f5ecf2596 (/health/instance) 2020-03-31 09:20:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:08,280 [551] [DEBUG] [app] Starting request: urn:request:25ce5d7a-5d51-4958-83cb-e7addea4859a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:08,280 [551] [DEBUG] [app] Starting request: urn:request:25ce5d7a-5d51-4958-83cb-e7addea4859a (/v1/_internal_ping) 2020-03-31 09:20:08,281 [551] [DEBUG] [app] Ending request: urn:request:25ce5d7a-5d51-4958-83cb-e7addea4859a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:08,281 [551] [DEBUG] [app] Ending request: urn:request:25ce5d7a-5d51-4958-83cb-e7addea4859a (/v1/_internal_ping) 2020-03-31 09:20:08,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:20:08,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:20:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:08,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:08,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:08,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:08,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:08,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:08,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:08,312 [527] [DEBUG] [app] Starting request: urn:request:df09237e-1c28-4e23-a1f6-a51ae063fd7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:08,312 [527] [DEBUG] [app] Starting request: urn:request:df09237e-1c28-4e23-a1f6-a51ae063fd7b (/_internal_ping) 2020-03-31 09:20:08,313 [527] [DEBUG] [app] Ending request: urn:request:df09237e-1c28-4e23-a1f6-a51ae063fd7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:08,313 [527] [DEBUG] [app] Ending request: urn:request:df09237e-1c28-4e23-a1f6-a51ae063fd7b (/_internal_ping) 2020-03-31 09:20:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:20:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:20:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:20:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:20:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:08,328 [539] [DEBUG] [app] Starting request: urn:request:94457191-6273-47a9-86b6-bb5a1ee22752 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:08,328 [539] [DEBUG] [app] Starting request: urn:request:94457191-6273-47a9-86b6-bb5a1ee22752 (/c1/_internal_ping) 2020-03-31 09:20:08,329 [539] [DEBUG] [app] Ending request: urn:request:94457191-6273-47a9-86b6-bb5a1ee22752 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:08,329 [539] [DEBUG] [app] Ending request: urn:request:94457191-6273-47a9-86b6-bb5a1ee22752 (/c1/_internal_ping) 2020-03-31 09:20:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:08,330 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:08,332 [518] [DEBUG] [app] Ending request: urn:request:b0a6a0ab-c78f-4872-879a-456f5ecf2596 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:20:08,330 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:20:08,332 [518] [DEBUG] [app] Ending request: urn:request:b0a6a0ab-c78f-4872-879a-456f5ecf2596 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:20:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:20:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:21.440707+00:00 (in 9.999101 seconds) 2020-03-31 09:20:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:21 UTC)" (scheduled at 2020-03-31 09:20:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:20:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:21.440707+00:00 (in 9.999101 seconds) 2020-03-31 09:20:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:21 UTC)" (scheduled at 2020-03-31 09:20:11.440707+00:00) 2020-03-31 09:20:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:20:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:20:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:20:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:20:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:20:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:21 UTC)" executed successfully 2020-03-31 09:20:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:20:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:24.323352+00:00 (in 9.998706 seconds) 2020-03-31 09:20:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:24 UTC)" (scheduled at 2020-03-31 09:20:14.323352+00:00) 2020-03-31 09:20:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:20:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:24.323352+00:00 (in 9.998706 seconds) 2020-03-31 09:20:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:24 UTC)" (scheduled at 2020-03-31 09:20:14.323352+00:00) 2020-03-31 09:20:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 14, 325704), True, datetime.datetime(2020, 3, 31, 9, 20, 14, 325704), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:20:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 14, 325704), True, datetime.datetime(2020, 3, 31, 9, 20, 14, 325704), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:20:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:20:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:20:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:24 UTC)" executed successfully 2020-03-31 09:20:14,446 [526] [DEBUG] [app] Starting request: urn:request:bbe02e7f-721d-4a2d-9202-ed61f1fed9e6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:14,446 [526] [DEBUG] [app] Starting request: urn:request:bbe02e7f-721d-4a2d-9202-ed61f1fed9e6 (/health/instance) 2020-03-31 09:20:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:14,453 [545] [DEBUG] [app] Starting request: urn:request:a783580d-b508-45c9-a934-d696c9589d27 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:14,453 [545] [DEBUG] [app] Starting request: urn:request:a783580d-b508-45c9-a934-d696c9589d27 (/v1/_internal_ping) 2020-03-31 09:20:14,455 [545] [DEBUG] [app] Ending request: urn:request:a783580d-b508-45c9-a934-d696c9589d27 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:14,455 [545] [DEBUG] [app] Ending request: urn:request:a783580d-b508-45c9-a934-d696c9589d27 (/v1/_internal_ping) 2020-03-31 09:20:14,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:20:14,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:20:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:14,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:20:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:20:14,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:14,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:14,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:14,459 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:14,459 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:14,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:14,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:14,490 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:14,490 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:14,495 [527] [DEBUG] [app] Starting request: urn:request:3565bebe-f561-4590-9125-e70745826006 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:14,495 [527] [DEBUG] [app] Starting request: urn:request:3565bebe-f561-4590-9125-e70745826006 (/_internal_ping) 2020-03-31 09:20:14,496 [527] [DEBUG] [app] Ending request: urn:request:3565bebe-f561-4590-9125-e70745826006 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:14,496 [527] [DEBUG] [app] Ending request: urn:request:3565bebe-f561-4590-9125-e70745826006 (/_internal_ping) 2020-03-31 09:20:14,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:14,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:20:14,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:14,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:20:14,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:14,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:14,502 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:14,502 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:14,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:14,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:20:14,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:14,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:14,510 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:14,510 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:14,511 [537] [DEBUG] [app] Starting request: urn:request:13abc5bd-2753-4af5-8150-8a851ba05452 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:14,511 [537] [DEBUG] [app] Starting request: urn:request:13abc5bd-2753-4af5-8150-8a851ba05452 (/c1/_internal_ping) 2020-03-31 09:20:14,512 [537] [DEBUG] [app] Ending request: urn:request:13abc5bd-2753-4af5-8150-8a851ba05452 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:14,512 [537] [DEBUG] [app] Ending request: urn:request:13abc5bd-2753-4af5-8150-8a851ba05452 (/c1/_internal_ping) 2020-03-31 09:20:14,513 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:14,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:14,513 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:20:14,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:14,514 [526] [DEBUG] [app] Ending request: urn:request:bbe02e7f-721d-4a2d-9202-ed61f1fed9e6 (/health/instance) 2020-03-31 09:20:14,515 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:14,514 [526] [DEBUG] [app] Ending request: urn:request:bbe02e7f-721d-4a2d-9202-ed61f1fed9e6 (/health/instance) 2020-03-31 09:20:14,515 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:14,515 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:20:14,515 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:20:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:20:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:33.366791+00:00 (in 18.825794 seconds) 2020-03-31 09:20:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:25:14 UTC)" (scheduled at 2020-03-31 09:20:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:20:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:33.366791+00:00 (in 18.825794 seconds) 2020-03-31 09:20:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:25:14 UTC)" (scheduled at 2020-03-31 09:20:14.539408+00:00) 2020-03-31 09:20:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:20:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), u'exportactionlogs/%']) 2020-03-31 09:20:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), True, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:20:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), True, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), 0, u'exportactionlogs/%']) 2020-03-31 09:20:14,564 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), True, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:20:14,564 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), True, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 20, 14, 541845), u'exportactionlogs/%']) 2020-03-31 09:20:14,567 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:20:14,567 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:14,569 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:20:14,569 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:14,570 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:25:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:20:14,570 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:25:14 UTC)" executed successfully 2020-03-31 09:20:14,571 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:20:14,571 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:14,572 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:20:14,572 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:20:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:20:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:45 UTC)" (scheduled at 2020-03-31 09:20:15.064514+00:00) gcworker stdout | 2020-03-31 09:20:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:45 UTC)" (scheduled at 2020-03-31 09:20:15.064514+00:00) 2020-03-31 09:20:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436815, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:20:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436815, 1209600, 500, 1, 0]) 2020-03-31 09:20:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:45.064514+00:00 (in 29.998998 seconds) gcworker stdout | 2020-03-31 09:20:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:45.064514+00:00 (in 29.998998 seconds) 2020-03-31 09:20:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:20:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:20:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:20:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:45 UTC)" executed successfully 2020-03-31 09:20:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:20:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:26.924497+00:00 (in 9.999080 seconds) 2020-03-31 09:20:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:26 UTC)" (scheduled at 2020-03-31 09:20:16.924497+00:00) 2020-03-31 09:20:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:20:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:26.924497+00:00 (in 9.999080 seconds) 2020-03-31 09:20:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:26 UTC)" (scheduled at 2020-03-31 09:20:16.924497+00:00) 2020-03-31 09:20:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 16, 926096), True, datetime.datetime(2020, 3, 31, 9, 20, 16, 926096), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:20:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 16, 926096), True, datetime.datetime(2020, 3, 31, 9, 20, 16, 926096), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:20:16,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:20:16,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:16,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:20:16,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:26 UTC)" executed successfully 2020-03-31 09:20:18,272 [526] [DEBUG] [app] Starting request: urn:request:4269e27e-eb4b-44a2-8037-2e9efbb83420 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:18,272 [526] [DEBUG] [app] Starting request: urn:request:4269e27e-eb4b-44a2-8037-2e9efbb83420 (/health/instance) 2020-03-31 09:20:18,276 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:18,276 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:18,278 [552] [DEBUG] [app] Starting request: urn:request:2a887a11-9dd7-42c7-a2a0-aed72f02ec06 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:18,278 [552] [DEBUG] [app] Starting request: urn:request:2a887a11-9dd7-42c7-a2a0-aed72f02ec06 (/v1/_internal_ping) 2020-03-31 09:20:18,279 [552] [DEBUG] [app] Ending request: urn:request:2a887a11-9dd7-42c7-a2a0-aed72f02ec06 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:18,279 [552] [DEBUG] [app] Ending request: urn:request:2a887a11-9dd7-42c7-a2a0-aed72f02ec06 (/v1/_internal_ping) 2020-03-31 09:20:18,280 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:18,280 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:20:18,280 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:18,281 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:20:18,280 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:18,281 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:20:18,281 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:18,284 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:18,281 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:18,286 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:18,284 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:18,286 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:18,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:18,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:18,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:18,306 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:18,310 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:18,310 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:18,312 [527] [DEBUG] [app] Starting request: urn:request:873c9e05-eaa0-4733-b20b-352dae529f28 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:18,312 [527] [DEBUG] [app] Starting request: urn:request:873c9e05-eaa0-4733-b20b-352dae529f28 (/_internal_ping) 2020-03-31 09:20:18,313 [527] [DEBUG] [app] Ending request: urn:request:873c9e05-eaa0-4733-b20b-352dae529f28 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:18,313 [527] [DEBUG] [app] Ending request: urn:request:873c9e05-eaa0-4733-b20b-352dae529f28 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:20:18,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:18,313 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:18,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:18,313 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:18,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:18,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:18,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:18,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:18,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:18,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:20:18,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:18,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:18,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:18,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:18,325 [534] [DEBUG] [app] Starting request: urn:request:f88f988d-bd06-4a1c-af2a-329248d126a9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:18,325 [534] [DEBUG] [app] Starting request: urn:request:f88f988d-bd06-4a1c-af2a-329248d126a9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:18,326 [534] [DEBUG] [app] Ending request: urn:request:f88f988d-bd06-4a1c-af2a-329248d126a9 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:18,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:18,326 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:20:18,326 [534] [DEBUG] [app] Ending request: urn:request:f88f988d-bd06-4a1c-af2a-329248d126a9 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:18,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:18,327 [526] [DEBUG] [app] Ending request: urn:request:4269e27e-eb4b-44a2-8037-2e9efbb83420 (/health/instance) 2020-03-31 09:20:18,328 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:20:18,326 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:18,328 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:20:18,327 [526] [DEBUG] [app] Ending request: urn:request:4269e27e-eb4b-44a2-8037-2e9efbb83420 (/health/instance) 2020-03-31 09:20:18,328 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:18,328 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:20:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:20:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:31 UTC)" (scheduled at 2020-03-31 09:20:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:20:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:31 UTC)" (scheduled at 2020-03-31 09:20:21.440707+00:00) 2020-03-31 09:20:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:20:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:20:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:31.440707+00:00 (in 9.998812 seconds) repositoryactioncounter stdout | 2020-03-31 09:20:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:31.440707+00:00 (in 9.998812 seconds) 2020-03-31 09:20:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:20:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:20:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:20:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:31 UTC)" executed successfully 2020-03-31 09:20:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:20:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:52.113221+00:00 (in 29.997980 seconds) 2020-03-31 09:20:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:52 UTC)" (scheduled at 2020-03-31 09:20:22.113221+00:00) securityworker stdout | 2020-03-31 09:20:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:52.113221+00:00 (in 29.997980 seconds) 2020-03-31 09:20:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:52 UTC)" (scheduled at 2020-03-31 09:20:22.113221+00:00) 2020-03-31 09:20:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:20:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:20:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:20:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:20:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:20:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:20:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:20:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:20:52 UTC)" executed successfully 2020-03-31 09:20:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:20:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:34 UTC)" (scheduled at 2020-03-31 09:20:24.323352+00:00) 2020-03-31 09:20:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:20:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:34 UTC)" (scheduled at 2020-03-31 09:20:24.323352+00:00) 2020-03-31 09:20:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:34.323352+00:00 (in 9.999232 seconds) notificationworker stdout | 2020-03-31 09:20:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:34.323352+00:00 (in 9.999232 seconds) 2020-03-31 09:20:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 24, 324946), True, datetime.datetime(2020, 3, 31, 9, 20, 24, 324946), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:20:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 24, 324946), True, datetime.datetime(2020, 3, 31, 9, 20, 24, 324946), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:20:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:20:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:20:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:34 UTC)" executed successfully 2020-03-31 09:20:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:20:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:33.330112+00:00 (in 8.918414 seconds) 2020-03-31 09:20:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:24 UTC)" (scheduled at 2020-03-31 09:20:24.410927+00:00) 2020-03-31 09:20:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:20:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:33.330112+00:00 (in 8.918414 seconds) 2020-03-31 09:20:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:24 UTC)" (scheduled at 2020-03-31 09:20:24.410927+00:00) 2020-03-31 09:20:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:20:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:20:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:24 UTC)" executed successfully 2020-03-31 09:20:24,445 [518] [DEBUG] [app] Starting request: urn:request:96a285cc-dd4a-4f89-93e2-2c905bf057f5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:24,445 [518] [DEBUG] [app] Starting request: urn:request:96a285cc-dd4a-4f89-93e2-2c905bf057f5 (/health/instance) 2020-03-31 09:20:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:24,452 [545] [DEBUG] [app] Starting request: urn:request:ec1df5e1-4e6a-404f-ab1e-8556d6120b99 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:24,452 [545] [DEBUG] [app] Starting request: urn:request:ec1df5e1-4e6a-404f-ab1e-8556d6120b99 (/v1/_internal_ping) 2020-03-31 09:20:24,453 [545] [DEBUG] [app] Ending request: urn:request:ec1df5e1-4e6a-404f-ab1e-8556d6120b99 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:24,453 [545] [DEBUG] [app] Ending request: urn:request:ec1df5e1-4e6a-404f-ab1e-8556d6120b99 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:24,454 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:24,456 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:24,454 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:24,458 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:20:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:24,458 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:24,456 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:24,458 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:24,458 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:24,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:24,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:24,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:24,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:24,485 [518] [DEBUG] [app] Starting request: urn:request:661fa75e-71ad-42ad-8d4d-98d7e9a6a0ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:24,485 [518] [DEBUG] [app] Starting request: urn:request:661fa75e-71ad-42ad-8d4d-98d7e9a6a0ce (/_internal_ping) 2020-03-31 09:20:24,486 [518] [DEBUG] [app] Ending request: urn:request:661fa75e-71ad-42ad-8d4d-98d7e9a6a0ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:24,486 [518] [DEBUG] [app] Ending request: urn:request:661fa75e-71ad-42ad-8d4d-98d7e9a6a0ce (/_internal_ping) 2020-03-31 09:20:24,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:20:24,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:20:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:20:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:24,501 [534] [DEBUG] [app] Starting request: urn:request:53dcaf67-93df-4499-84f5-74d8379b059b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:24,501 [534] [DEBUG] [app] Starting request: urn:request:53dcaf67-93df-4499-84f5-74d8379b059b (/c1/_internal_ping) 2020-03-31 09:20:24,501 [534] [DEBUG] [app] Ending request: urn:request:53dcaf67-93df-4499-84f5-74d8379b059b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:24,501 [534] [DEBUG] [app] Ending request: urn:request:53dcaf67-93df-4499-84f5-74d8379b059b (/c1/_internal_ping) 2020-03-31 09:20:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:24,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:24,504 [518] [DEBUG] [app] Ending request: urn:request:96a285cc-dd4a-4f89-93e2-2c905bf057f5 (/health/instance) 2020-03-31 09:20:24,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:20:24,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:24,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:20:24,504 [518] [DEBUG] [app] Ending request: urn:request:96a285cc-dd4a-4f89-93e2-2c905bf057f5 (/health/instance) 2020-03-31 09:20:24,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:24,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:20:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:20:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:36.924497+00:00 (in 9.999315 seconds) 2020-03-31 09:20:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:36 UTC)" (scheduled at 2020-03-31 09:20:26.924497+00:00) 2020-03-31 09:20:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:20:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:36.924497+00:00 (in 9.999315 seconds) 2020-03-31 09:20:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:36 UTC)" (scheduled at 2020-03-31 09:20:26.924497+00:00) 2020-03-31 09:20:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 26, 925796), True, datetime.datetime(2020, 3, 31, 9, 20, 26, 925796), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:20:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 26, 925796), True, datetime.datetime(2020, 3, 31, 9, 20, 26, 925796), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:20:26,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:20:26,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:20:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:36 UTC)" executed successfully 2020-03-31 09:20:28,273 [518] [DEBUG] [app] Starting request: urn:request:e224cd4b-56cb-4565-816c-6a98a17adc31 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:28,273 [518] [DEBUG] [app] Starting request: urn:request:e224cd4b-56cb-4565-816c-6a98a17adc31 (/health/instance) 2020-03-31 09:20:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:28,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:28,279 [538] [DEBUG] [app] Starting request: urn:request:6558c767-303a-4461-a692-7790bcdd54d9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:28,279 [538] [DEBUG] [app] Starting request: urn:request:6558c767-303a-4461-a692-7790bcdd54d9 (/v1/_internal_ping) 2020-03-31 09:20:28,281 [538] [DEBUG] [app] Ending request: urn:request:6558c767-303a-4461-a692-7790bcdd54d9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:28,281 [538] [DEBUG] [app] Ending request: urn:request:6558c767-303a-4461-a692-7790bcdd54d9 (/v1/_internal_ping) 2020-03-31 09:20:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:20:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:20:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:20:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:20:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:28,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:28,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:28,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:28,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:28,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:28,314 [518] [DEBUG] [app] Starting request: urn:request:91d317e6-563b-428f-ac79-3b204603f917 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:28,314 [518] [DEBUG] [app] Starting request: urn:request:91d317e6-563b-428f-ac79-3b204603f917 (/_internal_ping) 2020-03-31 09:20:28,316 [518] [DEBUG] [app] Ending request: urn:request:91d317e6-563b-428f-ac79-3b204603f917 (/_internal_ping) 2020-03-31 09:20:28,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:20:28,316 [518] [DEBUG] [app] Ending request: urn:request:91d317e6-563b-428f-ac79-3b204603f917 (/_internal_ping) 2020-03-31 09:20:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:28,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:20:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:28,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:20:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:28,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:28,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:28,334 [537] [DEBUG] [app] Starting request: urn:request:d1833ad6-906f-4f04-90f1-c295c9288464 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:28,334 [537] [DEBUG] [app] Starting request: urn:request:d1833ad6-906f-4f04-90f1-c295c9288464 (/c1/_internal_ping) 2020-03-31 09:20:28,335 [537] [DEBUG] [app] Ending request: urn:request:d1833ad6-906f-4f04-90f1-c295c9288464 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:28,335 [537] [DEBUG] [app] Ending request: urn:request:d1833ad6-906f-4f04-90f1-c295c9288464 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:28,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:28,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:20:28,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:28,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:28,338 [518] [DEBUG] [app] Ending request: urn:request:e224cd4b-56cb-4565-816c-6a98a17adc31 (/health/instance) 2020-03-31 09:20:28,338 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:28,338 [518] [DEBUG] [app] Ending request: urn:request:e224cd4b-56cb-4565-816c-6a98a17adc31 (/health/instance) 2020-03-31 09:20:28,338 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:28,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:20:28,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:20:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:20:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:00.337699+00:00 (in 29.999073 seconds) 2020-03-31 09:20:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:00 UTC)" (scheduled at 2020-03-31 09:20:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:20:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:00.337699+00:00 (in 29.999073 seconds) 2020-03-31 09:20:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:00 UTC)" (scheduled at 2020-03-31 09:20:30.337699+00:00) 2020-03-31 09:20:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 20, 30, 339337), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:20:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 20, 30, 339337), False, 50, 1, 0]) 2020-03-31 09:20:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:20:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:20:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:20:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:00 UTC)" executed successfully 2020-03-31 09:20:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:20:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:41.440707+00:00 (in 9.996034 seconds) 2020-03-31 09:20:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:41 UTC)" (scheduled at 2020-03-31 09:20:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:20:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:41.440707+00:00 (in 9.996034 seconds) 2020-03-31 09:20:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:41 UTC)" (scheduled at 2020-03-31 09:20:31.440707+00:00) 2020-03-31 09:20:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:20:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:20:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:20:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:20:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:20:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:41 UTC)" executed successfully 2020-03-31 09:20:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:20:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:03.330112+00:00 (in 29.999216 seconds) 2020-03-31 09:20:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:03 UTC)" (scheduled at 2020-03-31 09:20:33.330112+00:00) 2020-03-31 09:20:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:20:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:03.330112+00:00 (in 29.999216 seconds) 2020-03-31 09:20:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:03 UTC)" (scheduled at 2020-03-31 09:20:33.330112+00:00) 2020-03-31 09:20:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 33, 331851), True, datetime.datetime(2020, 3, 31, 9, 20, 33, 331851), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:20:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 33, 331851), True, datetime.datetime(2020, 3, 31, 9, 20, 33, 331851), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:20:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:20:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:20:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:03 UTC)" executed successfully 2020-03-31 09:20:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:20:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:33 UTC)" (scheduled at 2020-03-31 09:20:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:20:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:20:33 UTC)" (scheduled at 2020-03-31 09:20:33.366791+00:00) 2020-03-31 09:20:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:43.539999+00:00 (in 10.172368 seconds) 2020-03-31 09:20:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:20:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:43.539999+00:00 (in 10.172368 seconds) 2020-03-31 09:20:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 33, 368688), True, datetime.datetime(2020, 3, 31, 9, 20, 33, 368688), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:20:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 33, 368688), True, datetime.datetime(2020, 3, 31, 9, 20, 33, 368688), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:20:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:20:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:20:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:33 UTC)" executed successfully 2020-03-31 09:20:34,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:20:34,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:44.323352+00:00 (in 9.997696 seconds) 2020-03-31 09:20:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:44 UTC)" (scheduled at 2020-03-31 09:20:34.323352+00:00) 2020-03-31 09:20:34,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:20:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:44.323352+00:00 (in 9.997696 seconds) 2020-03-31 09:20:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:44 UTC)" (scheduled at 2020-03-31 09:20:34.323352+00:00) 2020-03-31 09:20:34,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 34, 326829), True, datetime.datetime(2020, 3, 31, 9, 20, 34, 326829), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:20:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 34, 326829), True, datetime.datetime(2020, 3, 31, 9, 20, 34, 326829), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:20:34,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:20:34,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:34,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:20:34,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:44 UTC)" executed successfully 2020-03-31 09:20:34,446 [527] [DEBUG] [app] Starting request: urn:request:9e459311-d3ac-4e6f-a8bb-27acb41aa842 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:34,446 [527] [DEBUG] [app] Starting request: urn:request:9e459311-d3ac-4e6f-a8bb-27acb41aa842 (/health/instance) 2020-03-31 09:20:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:34,455 [544] [DEBUG] [app] Starting request: urn:request:88e3ea30-92c3-4917-9fc6-ce4e9d096f70 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:34,455 [544] [DEBUG] [app] Starting request: urn:request:88e3ea30-92c3-4917-9fc6-ce4e9d096f70 (/v1/_internal_ping) 2020-03-31 09:20:34,456 [544] [DEBUG] [app] Ending request: urn:request:88e3ea30-92c3-4917-9fc6-ce4e9d096f70 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:34,456 [544] [DEBUG] [app] Ending request: urn:request:88e3ea30-92c3-4917-9fc6-ce4e9d096f70 (/v1/_internal_ping) 2020-03-31 09:20:34,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:34,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:20:34,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:34,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:34,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:20:34,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:34,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:34,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:34,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:34,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:34,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:34,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:34,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:34,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:34,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:34,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:34,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:34,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:34,488 [527] [DEBUG] [app] Starting request: urn:request:030d9f8b-3a95-42b8-94b6-d5a4f37981ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:34,488 [527] [DEBUG] [app] Starting request: urn:request:030d9f8b-3a95-42b8-94b6-d5a4f37981ac (/_internal_ping) 2020-03-31 09:20:34,488 [527] [DEBUG] [app] Ending request: urn:request:030d9f8b-3a95-42b8-94b6-d5a4f37981ac (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:34,488 [527] [DEBUG] [app] Ending request: urn:request:030d9f8b-3a95-42b8-94b6-d5a4f37981ac (/_internal_ping) 2020-03-31 09:20:34,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:20:34,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:34,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:20:34,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:34,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:34,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:34,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:34,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:20:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:34,501 [543] [DEBUG] [app] Starting request: urn:request:d1ddea25-440b-413f-ac0e-29d530eb92d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:34,501 [543] [DEBUG] [app] Starting request: urn:request:d1ddea25-440b-413f-ac0e-29d530eb92d6 (/c1/_internal_ping) 2020-03-31 09:20:34,502 [543] [DEBUG] [app] Ending request: urn:request:d1ddea25-440b-413f-ac0e-29d530eb92d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:34,502 [543] [DEBUG] [app] Ending request: urn:request:d1ddea25-440b-413f-ac0e-29d530eb92d6 (/c1/_internal_ping) 2020-03-31 09:20:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:34,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:34,504 [527] [DEBUG] [app] Ending request: urn:request:9e459311-d3ac-4e6f-a8bb-27acb41aa842 (/health/instance) 2020-03-31 09:20:34,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:20:34,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:34,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:20:34,504 [527] [DEBUG] [app] Ending request: urn:request:9e459311-d3ac-4e6f-a8bb-27acb41aa842 (/health/instance) 2020-03-31 09:20:34,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:34,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:20:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:20:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:38.010934+00:00 (in 1.085764 seconds) 2020-03-31 09:20:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:46 UTC)" (scheduled at 2020-03-31 09:20:36.924497+00:00) 2020-03-31 09:20:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:20:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:38.010934+00:00 (in 1.085764 seconds) 2020-03-31 09:20:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:46 UTC)" (scheduled at 2020-03-31 09:20:36.924497+00:00) 2020-03-31 09:20:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 36, 926154), True, datetime.datetime(2020, 3, 31, 9, 20, 36, 926154), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:20:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 36, 926154), True, datetime.datetime(2020, 3, 31, 9, 20, 36, 926154), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:20:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:20:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:20:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:46 UTC)" executed successfully 2020-03-31 09:20:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:20:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:46.924497+00:00 (in 8.912812 seconds) 2020-03-31 09:20:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:38 UTC)" (scheduled at 2020-03-31 09:20:38.010934+00:00) 2020-03-31 09:20:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:20:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:20:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:46.924497+00:00 (in 8.912812 seconds) 2020-03-31 09:20:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:38 UTC)" (scheduled at 2020-03-31 09:20:38.010934+00:00) 2020-03-31 09:20:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:20:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:38 UTC)" executed successfully 2020-03-31 09:20:38,273 [518] [DEBUG] [app] Starting request: urn:request:926a04e7-79b0-496b-858f-2c578b01a087 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:38,273 [518] [DEBUG] [app] Starting request: urn:request:926a04e7-79b0-496b-858f-2c578b01a087 (/health/instance) 2020-03-31 09:20:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:38,280 [545] [DEBUG] [app] Starting request: urn:request:7a6370a6-e99f-4df4-8337-18e1ba506754 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:38,280 [545] [DEBUG] [app] Starting request: urn:request:7a6370a6-e99f-4df4-8337-18e1ba506754 (/v1/_internal_ping) 2020-03-31 09:20:38,281 [545] [DEBUG] [app] Ending request: urn:request:7a6370a6-e99f-4df4-8337-18e1ba506754 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:38,281 [545] [DEBUG] [app] Ending request: urn:request:7a6370a6-e99f-4df4-8337-18e1ba506754 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:38,282 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:20:38,282 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:20:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:38,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:38,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:38,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:38,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:38,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:38,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:38,311 [518] [DEBUG] [app] Starting request: urn:request:1b17e054-ece3-46f9-845e-556c4d1e9dd2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:38,311 [518] [DEBUG] [app] Starting request: urn:request:1b17e054-ece3-46f9-845e-556c4d1e9dd2 (/_internal_ping) 2020-03-31 09:20:38,312 [518] [DEBUG] [app] Ending request: urn:request:1b17e054-ece3-46f9-845e-556c4d1e9dd2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:38,312 [518] [DEBUG] [app] Ending request: urn:request:1b17e054-ece3-46f9-845e-556c4d1e9dd2 (/_internal_ping) 2020-03-31 09:20:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:20:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:20:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:38,326 [541] [DEBUG] [app] Starting request: urn:request:b3b02f98-f671-4b5a-9a63-75f0d59a3315 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:38,326 [541] [DEBUG] [app] Starting request: urn:request:b3b02f98-f671-4b5a-9a63-75f0d59a3315 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:38,327 [541] [DEBUG] [app] Ending request: urn:request:b3b02f98-f671-4b5a-9a63-75f0d59a3315 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:38,327 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:20:38,327 [541] [DEBUG] [app] Ending request: urn:request:b3b02f98-f671-4b5a-9a63-75f0d59a3315 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:38,328 [518] [DEBUG] [app] Ending request: urn:request:926a04e7-79b0-496b-858f-2c578b01a087 (/health/instance) 2020-03-31 09:20:38,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:20:38,327 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:38,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:20:38,328 [518] [DEBUG] [app] Ending request: urn:request:926a04e7-79b0-496b-858f-2c578b01a087 (/health/instance) 2020-03-31 09:20:38,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:38,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:20:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:20:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:51.440707+00:00 (in 9.999061 seconds) 2020-03-31 09:20:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:51 UTC)" (scheduled at 2020-03-31 09:20:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:20:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:51.440707+00:00 (in 9.999061 seconds) 2020-03-31 09:20:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:51 UTC)" (scheduled at 2020-03-31 09:20:41.440707+00:00) 2020-03-31 09:20:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:20:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:20:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:20:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:20:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:20:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:51 UTC)" executed successfully 2020-03-31 09:20:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:20:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:43,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:33.366791+00:00 (in 49.824856 seconds) 2020-03-31 09:20:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:43 UTC)" (scheduled at 2020-03-31 09:20:43.539999+00:00) 2020-03-31 09:20:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:20:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:20:43,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:33.366791+00:00 (in 49.824856 seconds) 2020-03-31 09:20:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:43 UTC)" (scheduled at 2020-03-31 09:20:43.539999+00:00) 2020-03-31 09:20:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:20:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:43 UTC)" executed successfully 2020-03-31 09:20:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:20:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:49.498966+00:00 (in 5.174333 seconds) 2020-03-31 09:20:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:54 UTC)" (scheduled at 2020-03-31 09:20:44.323352+00:00) 2020-03-31 09:20:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:20:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:49.498966+00:00 (in 5.174333 seconds) 2020-03-31 09:20:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:54 UTC)" (scheduled at 2020-03-31 09:20:44.323352+00:00) 2020-03-31 09:20:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 44, 325756), True, datetime.datetime(2020, 3, 31, 9, 20, 44, 325756), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:20:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 44, 325756), True, datetime.datetime(2020, 3, 31, 9, 20, 44, 325756), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:20:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:20:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:20:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:54 UTC)" executed successfully 2020-03-31 09:20:44,446 [527] [DEBUG] [app] Starting request: urn:request:97708f69-7e96-456c-9910-1c287f820665 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:44,446 [527] [DEBUG] [app] Starting request: urn:request:97708f69-7e96-456c-9910-1c287f820665 (/health/instance) 2020-03-31 09:20:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:44,453 [540] [DEBUG] [app] Starting request: urn:request:0dc2d9f0-ba1c-419b-af07-367cf649bd5d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:44,453 [540] [DEBUG] [app] Starting request: urn:request:0dc2d9f0-ba1c-419b-af07-367cf649bd5d (/v1/_internal_ping) 2020-03-31 09:20:44,454 [540] [DEBUG] [app] Ending request: urn:request:0dc2d9f0-ba1c-419b-af07-367cf649bd5d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:44,454 [540] [DEBUG] [app] Ending request: urn:request:0dc2d9f0-ba1c-419b-af07-367cf649bd5d (/v1/_internal_ping) 2020-03-31 09:20:44,455 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:20:44,455 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:44,457 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:20:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:44,458 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:44,457 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:44,459 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:44,458 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:44,459 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:44,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:44,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:44,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:44,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:44,485 [527] [DEBUG] [app] Starting request: urn:request:5acce9a3-bc4c-4ea0-8ea4-182f471d7e56 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:44,485 [527] [DEBUG] [app] Starting request: urn:request:5acce9a3-bc4c-4ea0-8ea4-182f471d7e56 (/_internal_ping) 2020-03-31 09:20:44,486 [527] [DEBUG] [app] Ending request: urn:request:5acce9a3-bc4c-4ea0-8ea4-182f471d7e56 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:44,486 [527] [DEBUG] [app] Ending request: urn:request:5acce9a3-bc4c-4ea0-8ea4-182f471d7e56 (/_internal_ping) 2020-03-31 09:20:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:44,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:44,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:20:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:20:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:20:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:44,499 [539] [DEBUG] [app] Starting request: urn:request:9e48485f-d459-4fc3-8c9b-3ed62278c87b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:44,499 [539] [DEBUG] [app] Starting request: urn:request:9e48485f-d459-4fc3-8c9b-3ed62278c87b (/c1/_internal_ping) 2020-03-31 09:20:44,500 [539] [DEBUG] [app] Ending request: urn:request:9e48485f-d459-4fc3-8c9b-3ed62278c87b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:44,500 [539] [DEBUG] [app] Ending request: urn:request:9e48485f-d459-4fc3-8c9b-3ed62278c87b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:44,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:20:44,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:44,503 [527] [DEBUG] [app] Ending request: urn:request:97708f69-7e96-456c-9910-1c287f820665 (/health/instance) 2020-03-31 09:20:44,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:44,503 [527] [DEBUG] [app] Ending request: urn:request:97708f69-7e96-456c-9910-1c287f820665 (/health/instance) 2020-03-31 09:20:44,503 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:44,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:20:44,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:20:45,069 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:20:45,069 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:45,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:15.064514+00:00 (in 29.995094 seconds) 2020-03-31 09:20:45,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:15 UTC)" (scheduled at 2020-03-31 09:20:45.064514+00:00) gcworker stdout | 2020-03-31 09:20:45,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:15.064514+00:00 (in 29.995094 seconds) 2020-03-31 09:20:45,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:15 UTC)" (scheduled at 2020-03-31 09:20:45.064514+00:00) 2020-03-31 09:20:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436845, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:20:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436845, 1209600, 500, 1, 0]) 2020-03-31 09:20:45,088 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:20:45,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:20:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:20:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:15 UTC)" executed successfully 2020-03-31 09:20:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:56.924497+00:00 (in 9.999198 seconds) 2020-03-31 09:20:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:56 UTC)" (scheduled at 2020-03-31 09:20:46.924497+00:00) 2020-03-31 09:20:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:20:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:20:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:56.924497+00:00 (in 9.999198 seconds) 2020-03-31 09:20:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:56 UTC)" (scheduled at 2020-03-31 09:20:46.924497+00:00) 2020-03-31 09:20:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 46, 926235), True, datetime.datetime(2020, 3, 31, 9, 20, 46, 926235), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:20:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 46, 926235), True, datetime.datetime(2020, 3, 31, 9, 20, 46, 926235), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:20:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:20:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:20:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:20:56 UTC)" executed successfully 2020-03-31 09:20:48,273 [527] [DEBUG] [app] Starting request: urn:request:3842393c-74ab-4749-a3e6-b47a92fc9130 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:48,273 [527] [DEBUG] [app] Starting request: urn:request:3842393c-74ab-4749-a3e6-b47a92fc9130 (/health/instance) 2020-03-31 09:20:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:48,277 [553] [DEBUG] [app] Starting request: urn:request:853fb05c-997a-4c6a-8dee-735293c55ef3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:48,277 [553] [DEBUG] [app] Starting request: urn:request:853fb05c-997a-4c6a-8dee-735293c55ef3 (/v1/_internal_ping) 2020-03-31 09:20:48,278 [553] [DEBUG] [app] Ending request: urn:request:853fb05c-997a-4c6a-8dee-735293c55ef3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:48,278 [553] [DEBUG] [app] Ending request: urn:request:853fb05c-997a-4c6a-8dee-735293c55ef3 (/v1/_internal_ping) 2020-03-31 09:20:48,279 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:20:48,279 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:48,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:48,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:20:48,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:48,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:48,281 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:48,281 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:48,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:48,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:48,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:48,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:48,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:48,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:48,307 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:48,307 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:48,309 [518] [DEBUG] [app] Starting request: urn:request:35286617-7139-4063-ade3-39581a61e8e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:48,309 [518] [DEBUG] [app] Starting request: urn:request:35286617-7139-4063-ade3-39581a61e8e3 (/_internal_ping) 2020-03-31 09:20:48,309 [518] [DEBUG] [app] Ending request: urn:request:35286617-7139-4063-ade3-39581a61e8e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:48,309 [518] [DEBUG] [app] Ending request: urn:request:35286617-7139-4063-ade3-39581a61e8e3 (/_internal_ping) 2020-03-31 09:20:48,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:48,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:48,310 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:48,310 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:48,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:48,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:20:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:48,322 [539] [DEBUG] [app] Starting request: urn:request:ee75c618-8a73-41ae-bca9-103e1ad2495b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:48,322 [539] [DEBUG] [app] Starting request: urn:request:ee75c618-8a73-41ae-bca9-103e1ad2495b (/c1/_internal_ping) 2020-03-31 09:20:48,323 [539] [DEBUG] [app] Ending request: urn:request:ee75c618-8a73-41ae-bca9-103e1ad2495b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:48,323 [539] [DEBUG] [app] Ending request: urn:request:ee75c618-8a73-41ae-bca9-103e1ad2495b (/c1/_internal_ping) 2020-03-31 09:20:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:48,323 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:20:48,323 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:48,325 [527] [DEBUG] [app] Ending request: urn:request:3842393c-74ab-4749-a3e6-b47a92fc9130 (/health/instance) 2020-03-31 09:20:48,325 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:48,325 [527] [DEBUG] [app] Ending request: urn:request:3842393c-74ab-4749-a3e6-b47a92fc9130 (/health/instance) 2020-03-31 09:20:48,325 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:48,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) gunicorn-web stdout | 2020-03-31 09:20:48,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) 2020-03-31 09:20:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:20:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:54.323352+00:00 (in 4.823233 seconds) 2020-03-31 09:20:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:49 UTC)" (scheduled at 2020-03-31 09:20:49.498966+00:00) 2020-03-31 09:20:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:20:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:54.323352+00:00 (in 4.823233 seconds) 2020-03-31 09:20:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:49 UTC)" (scheduled at 2020-03-31 09:20:49.498966+00:00) 2020-03-31 09:20:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:20:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:20:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:49 UTC)" executed successfully 2020-03-31 09:20:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:20:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:59.560326+00:00 (in 8.448253 seconds) 2020-03-31 09:20:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:51 UTC)" (scheduled at 2020-03-31 09:20:51.110610+00:00) 2020-03-31 09:20:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:20:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:20:59.560326+00:00 (in 8.448253 seconds) 2020-03-31 09:20:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:51 UTC)" (scheduled at 2020-03-31 09:20:51.110610+00:00) 2020-03-31 09:20:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 51, 113312), True, datetime.datetime(2020, 3, 31, 9, 20, 51, 113312), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:20:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 51, 113312), True, datetime.datetime(2020, 3, 31, 9, 20, 51, 113312), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:20:51,131 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:20:51,131 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:20:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:51 UTC)" executed successfully 2020-03-31 09:20:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:20:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:01.440707+00:00 (in 9.999222 seconds) 2020-03-31 09:20:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:01 UTC)" (scheduled at 2020-03-31 09:20:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:20:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:01.440707+00:00 (in 9.999222 seconds) 2020-03-31 09:20:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:01 UTC)" (scheduled at 2020-03-31 09:20:51.440707+00:00) 2020-03-31 09:20:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:20:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:20:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:20:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:20:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:20:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:01 UTC)" executed successfully 2020-03-31 09:20:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:20:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:22.113221+00:00 (in 29.999087 seconds) 2020-03-31 09:20:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:22 UTC)" (scheduled at 2020-03-31 09:20:52.113221+00:00) securityworker stdout | 2020-03-31 09:20:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:22.113221+00:00 (in 29.999087 seconds) 2020-03-31 09:20:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:22 UTC)" (scheduled at 2020-03-31 09:20:52.113221+00:00) 2020-03-31 09:20:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:20:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:20:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:20:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:20:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:20:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:20:52,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:20:52,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:22 UTC)" executed successfully 2020-03-31 09:20:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:20:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:04.323352+00:00 (in 9.998544 seconds) 2020-03-31 09:20:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:04 UTC)" (scheduled at 2020-03-31 09:20:54.323352+00:00) notificationworker stdout | 2020-03-31 09:20:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:04.323352+00:00 (in 9.998544 seconds) 2020-03-31 09:20:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:04 UTC)" (scheduled at 2020-03-31 09:20:54.323352+00:00) 2020-03-31 09:20:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:20:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 54, 325861), True, datetime.datetime(2020, 3, 31, 9, 20, 54, 325861), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:20:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 54, 325861), True, datetime.datetime(2020, 3, 31, 9, 20, 54, 325861), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:20:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:20:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:20:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:04 UTC)" executed successfully 2020-03-31 09:20:54,446 [518] [DEBUG] [app] Starting request: urn:request:9ec86639-c187-4e41-ba29-85269fe8f57a (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:54,446 [518] [DEBUG] [app] Starting request: urn:request:9ec86639-c187-4e41-ba29-85269fe8f57a (/health/instance) 2020-03-31 09:20:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:54,453 [548] [DEBUG] [app] Starting request: urn:request:c045cad3-6650-4da0-bd77-7d468dfc9597 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:54,453 [548] [DEBUG] [app] Starting request: urn:request:c045cad3-6650-4da0-bd77-7d468dfc9597 (/v1/_internal_ping) 2020-03-31 09:20:54,454 [548] [DEBUG] [app] Ending request: urn:request:c045cad3-6650-4da0-bd77-7d468dfc9597 (/v1/_internal_ping) 2020-03-31 09:20:54,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:20:54,454 [548] [DEBUG] [app] Ending request: urn:request:c045cad3-6650-4da0-bd77-7d468dfc9597 (/v1/_internal_ping) 2020-03-31 09:20:54,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:20:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:54,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:54,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:54,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:54,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:54,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:54,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:54,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:54,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:54,490 [521] [DEBUG] [app] Starting request: urn:request:75af24b7-eb62-446a-9d7c-026022a39f2d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:54,490 [521] [DEBUG] [app] Starting request: urn:request:75af24b7-eb62-446a-9d7c-026022a39f2d (/_internal_ping) 2020-03-31 09:20:54,491 [521] [DEBUG] [app] Ending request: urn:request:75af24b7-eb62-446a-9d7c-026022a39f2d (/_internal_ping) 2020-03-31 09:20:54,491 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:54,491 [521] [DEBUG] [app] Ending request: urn:request:75af24b7-eb62-446a-9d7c-026022a39f2d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:54,491 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:20:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:20:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:54,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:54,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:54,505 [539] [DEBUG] [app] Starting request: urn:request:9bfda221-bba4-4341-9964-47e43c6ab0c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:54,505 [539] [DEBUG] [app] Starting request: urn:request:9bfda221-bba4-4341-9964-47e43c6ab0c7 (/c1/_internal_ping) 2020-03-31 09:20:54,506 [539] [DEBUG] [app] Ending request: urn:request:9bfda221-bba4-4341-9964-47e43c6ab0c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:54,506 [539] [DEBUG] [app] Ending request: urn:request:9bfda221-bba4-4341-9964-47e43c6ab0c7 (/c1/_internal_ping) 2020-03-31 09:20:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:20:54,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:20:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:20:54,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:54,509 [518] [DEBUG] [app] Ending request: urn:request:9ec86639-c187-4e41-ba29-85269fe8f57a (/health/instance) 2020-03-31 09:20:54,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:20:54,509 [518] [DEBUG] [app] Ending request: urn:request:9ec86639-c187-4e41-ba29-85269fe8f57a (/health/instance) 2020-03-31 09:20:54,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:20:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:20:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:20:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:06.924497+00:00 (in 9.999118 seconds) storagereplication stdout | 2020-03-31 09:20:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:06.924497+00:00 (in 9.999118 seconds) 2020-03-31 09:20:56,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:06 UTC)" (scheduled at 2020-03-31 09:20:56.924497+00:00) storagereplication stdout | 2020-03-31 09:20:56,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:06 UTC)" (scheduled at 2020-03-31 09:20:56.924497+00:00) 2020-03-31 09:20:56,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:20:56,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:20:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 56, 927798), True, datetime.datetime(2020, 3, 31, 9, 20, 56, 927798), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:20:56,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 20, 56, 927798), True, datetime.datetime(2020, 3, 31, 9, 20, 56, 927798), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:20:56,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:20:56,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:20:56,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:56,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:20:56,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:20:56,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:06 UTC)" executed successfully 2020-03-31 09:20:58,273 [525] [DEBUG] [app] Starting request: urn:request:3b3c6f23-0356-4b98-87f2-65e25c3256a9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:20:58,273 [525] [DEBUG] [app] Starting request: urn:request:3b3c6f23-0356-4b98-87f2-65e25c3256a9 (/health/instance) 2020-03-31 09:20:58,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:58,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:58,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:58,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:58,280 [549] [DEBUG] [app] Starting request: urn:request:ae9b4016-3078-4e87-83f1-1240e6b14d43 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:58,280 [549] [DEBUG] [app] Starting request: urn:request:ae9b4016-3078-4e87-83f1-1240e6b14d43 (/v1/_internal_ping) 2020-03-31 09:20:58,281 [549] [DEBUG] [app] Ending request: urn:request:ae9b4016-3078-4e87-83f1-1240e6b14d43 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:20:58,281 [549] [DEBUG] [app] Ending request: urn:request:ae9b4016-3078-4e87-83f1-1240e6b14d43 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:58,281 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:58,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:20:58,281 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:58,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:20:58,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:20:58,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:20:58,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:20:58,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:20:58,285 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:20:58,285 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:20:58,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:20:58,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:20:58,292 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:20:58,292 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:20:58,305 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:20:58,305 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:20:58,309 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:58,309 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:58,310 [525] [DEBUG] [app] Starting request: urn:request:d75821b0-f374-4cb3-be57-484438605395 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:58,310 [525] [DEBUG] [app] Starting request: urn:request:d75821b0-f374-4cb3-be57-484438605395 (/_internal_ping) 2020-03-31 09:20:58,311 [525] [DEBUG] [app] Ending request: urn:request:d75821b0-f374-4cb3-be57-484438605395 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:58,311 [525] [DEBUG] [app] Ending request: urn:request:d75821b0-f374-4cb3-be57-484438605395 (/_internal_ping) 2020-03-31 09:20:58,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:58,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:58,312 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:20:58,312 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:58,315 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:58,315 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:58,315 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:20:58,315 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:20:58,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:58,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:20:58,319 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:20:58,319 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:20:58,322 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:20:58,322 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:20:58,324 [543] [DEBUG] [app] Starting request: urn:request:8c3e86ad-50c0-4d80-a793-955d7016ba11 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:20:58,324 [543] [DEBUG] [app] Starting request: urn:request:8c3e86ad-50c0-4d80-a793-955d7016ba11 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:58,324 [543] [DEBUG] [app] Ending request: urn:request:8c3e86ad-50c0-4d80-a793-955d7016ba11 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:20:58,325 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:58,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:20:58,324 [543] [DEBUG] [app] Ending request: urn:request:8c3e86ad-50c0-4d80-a793-955d7016ba11 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:20:58,325 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:20:58,327 [525] [DEBUG] [app] Ending request: urn:request:3b3c6f23-0356-4b98-87f2-65e25c3256a9 (/health/instance) 2020-03-31 09:20:58,327 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:20:58,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:20:58,328 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:20:58,327 [525] [DEBUG] [app] Ending request: urn:request:3b3c6f23-0356-4b98-87f2-65e25c3256a9 (/health/instance) 2020-03-31 09:20:58,327 [525] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:20:58,328 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:20:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:20:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:20:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:20:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:20:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:51.110610+00:00 (in 51.549251 seconds) 2020-03-31 09:20:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:59 UTC)" (scheduled at 2020-03-31 09:20:59.560326+00:00) 2020-03-31 09:20:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:20:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:51.110610+00:00 (in 51.549251 seconds) 2020-03-31 09:20:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:59 UTC)" (scheduled at 2020-03-31 09:20:59.560326+00:00) 2020-03-31 09:20:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:20:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:20:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:21:59 UTC)" executed successfully 2020-03-31 09:21:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:21:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:30.337699+00:00 (in 29.999265 seconds) 2020-03-31 09:21:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:30 UTC)" (scheduled at 2020-03-31 09:21:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:21:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:30.337699+00:00 (in 29.999265 seconds) 2020-03-31 09:21:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:30 UTC)" (scheduled at 2020-03-31 09:21:00.337699+00:00) 2020-03-31 09:21:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 21, 0, 339139), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:21:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 21, 0, 339139), False, 50, 1, 0]) 2020-03-31 09:21:00,363 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:21:00,363 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:21:00,363 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:00,364 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:21:00,363 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:00,364 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:30 UTC)" executed successfully 2020-03-31 09:21:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:21:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:11.440707+00:00 (in 9.999151 seconds) 2020-03-31 09:21:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:11 UTC)" (scheduled at 2020-03-31 09:21:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:21:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:11.440707+00:00 (in 9.999151 seconds) 2020-03-31 09:21:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:11 UTC)" (scheduled at 2020-03-31 09:21:01.440707+00:00) 2020-03-31 09:21:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:21:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:21:01,467 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:01,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:01,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:21:01,467 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:01,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:01,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:11 UTC)" executed successfully 2020-03-31 09:21:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:21:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:24.410927+00:00 (in 21.080048 seconds) 2020-03-31 09:21:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:33 UTC)" (scheduled at 2020-03-31 09:21:03.330112+00:00) 2020-03-31 09:21:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:21:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:24.410927+00:00 (in 21.080048 seconds) 2020-03-31 09:21:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:33 UTC)" (scheduled at 2020-03-31 09:21:03.330112+00:00) 2020-03-31 09:21:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 3, 331819), True, datetime.datetime(2020, 3, 31, 9, 21, 3, 331819), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:21:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 3, 331819), True, datetime.datetime(2020, 3, 31, 9, 21, 3, 331819), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:21:03,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:21:03,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:21:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:33 UTC)" executed successfully 2020-03-31 09:21:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:21:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:14.323352+00:00 (in 9.998904 seconds) 2020-03-31 09:21:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:14 UTC)" (scheduled at 2020-03-31 09:21:04.323352+00:00) 2020-03-31 09:21:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:21:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:14.323352+00:00 (in 9.998904 seconds) 2020-03-31 09:21:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:14 UTC)" (scheduled at 2020-03-31 09:21:04.323352+00:00) 2020-03-31 09:21:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 4, 325133), True, datetime.datetime(2020, 3, 31, 9, 21, 4, 325133), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:21:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 4, 325133), True, datetime.datetime(2020, 3, 31, 9, 21, 4, 325133), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:21:04,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:21:04,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:04,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:21:04,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:14 UTC)" executed successfully 2020-03-31 09:21:04,446 [527] [DEBUG] [app] Starting request: urn:request:a326c2c5-918c-4ca0-b5f2-bc4a000ee411 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:04,446 [527] [DEBUG] [app] Starting request: urn:request:a326c2c5-918c-4ca0-b5f2-bc4a000ee411 (/health/instance) 2020-03-31 09:21:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:04,454 [555] [DEBUG] [app] Starting request: urn:request:fc62826a-685a-4410-be44-795123b0e440 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:04,454 [555] [DEBUG] [app] Starting request: urn:request:fc62826a-685a-4410-be44-795123b0e440 (/v1/_internal_ping) 2020-03-31 09:21:04,455 [555] [DEBUG] [app] Ending request: urn:request:fc62826a-685a-4410-be44-795123b0e440 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:04,455 [555] [DEBUG] [app] Ending request: urn:request:fc62826a-685a-4410-be44-795123b0e440 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:04,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:21:04,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:21:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:04,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:04,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:04,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:04,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:04,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:04,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:04,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:04,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:04,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:04,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:04,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:04,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:04,487 [527] [DEBUG] [app] Starting request: urn:request:55983bc7-84e3-4a2a-86a7-b611960832c3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:04,487 [527] [DEBUG] [app] Starting request: urn:request:55983bc7-84e3-4a2a-86a7-b611960832c3 (/_internal_ping) 2020-03-31 09:21:04,488 [527] [DEBUG] [app] Ending request: urn:request:55983bc7-84e3-4a2a-86a7-b611960832c3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:04,488 [527] [DEBUG] [app] Ending request: urn:request:55983bc7-84e3-4a2a-86a7-b611960832c3 (/_internal_ping) 2020-03-31 09:21:04,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:21:04,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:21:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:21:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:21:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:21:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:21:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:04,502 [534] [DEBUG] [app] Starting request: urn:request:d7faffbf-6985-42da-960d-ced4b725e5d3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:04,502 [534] [DEBUG] [app] Starting request: urn:request:d7faffbf-6985-42da-960d-ced4b725e5d3 (/c1/_internal_ping) 2020-03-31 09:21:04,503 [534] [DEBUG] [app] Ending request: urn:request:d7faffbf-6985-42da-960d-ced4b725e5d3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:04,503 [534] [DEBUG] [app] Ending request: urn:request:d7faffbf-6985-42da-960d-ced4b725e5d3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:04,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:04,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:04,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:21:04,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:04,505 [527] [DEBUG] [app] Ending request: urn:request:a326c2c5-918c-4ca0-b5f2-bc4a000ee411 (/health/instance) 2020-03-31 09:21:04,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:04,505 [527] [DEBUG] [app] Ending request: urn:request:a326c2c5-918c-4ca0-b5f2-bc4a000ee411 (/health/instance) 2020-03-31 09:21:04,505 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:04,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:21:04,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:21:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:21:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:16.924497+00:00 (in 9.999014 seconds) 2020-03-31 09:21:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:16 UTC)" (scheduled at 2020-03-31 09:21:06.924497+00:00) storagereplication stdout | 2020-03-31 09:21:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:16.924497+00:00 (in 9.999014 seconds) 2020-03-31 09:21:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:16 UTC)" (scheduled at 2020-03-31 09:21:06.924497+00:00) 2020-03-31 09:21:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:21:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 6, 926784), True, datetime.datetime(2020, 3, 31, 9, 21, 6, 926784), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:21:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 6, 926784), True, datetime.datetime(2020, 3, 31, 9, 21, 6, 926784), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:21:06,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:06,949 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:21:06,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:06,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:06,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:21:06,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:16 UTC)" executed successfully 2020-03-31 09:21:08,273 [518] [DEBUG] [app] Starting request: urn:request:942bfe27-580d-49c8-83d9-ff9bc4abaff8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:08,273 [518] [DEBUG] [app] Starting request: urn:request:942bfe27-580d-49c8-83d9-ff9bc4abaff8 (/health/instance) 2020-03-31 09:21:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:08,281 [547] [DEBUG] [app] Starting request: urn:request:e1f0eecc-6958-4ba8-9be3-1908522f98ad (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:08,281 [547] [DEBUG] [app] Starting request: urn:request:e1f0eecc-6958-4ba8-9be3-1908522f98ad (/v1/_internal_ping) 2020-03-31 09:21:08,282 [547] [DEBUG] [app] Ending request: urn:request:e1f0eecc-6958-4ba8-9be3-1908522f98ad (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:08,282 [547] [DEBUG] [app] Ending request: urn:request:e1f0eecc-6958-4ba8-9be3-1908522f98ad (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:21:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:08,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:21:08,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:08,287 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:08,287 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:08,289 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:08,289 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:08,289 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:08,289 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:08,312 [526] [DEBUG] [app] Starting request: urn:request:f637ba48-c510-4dbc-a7a5-468b485da397 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:08,312 [526] [DEBUG] [app] Starting request: urn:request:f637ba48-c510-4dbc-a7a5-468b485da397 (/_internal_ping) 2020-03-31 09:21:08,313 [526] [DEBUG] [app] Ending request: urn:request:f637ba48-c510-4dbc-a7a5-468b485da397 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:08,313 [526] [DEBUG] [app] Ending request: urn:request:f637ba48-c510-4dbc-a7a5-468b485da397 (/_internal_ping) 2020-03-31 09:21:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:08,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:08,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:21:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:21:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:21:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:21:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:21:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:08,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:08,328 [534] [DEBUG] [app] Starting request: urn:request:cf12d4ac-34c3-4c0e-b122-fe1ba3ef80cb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:08,328 [534] [DEBUG] [app] Starting request: urn:request:cf12d4ac-34c3-4c0e-b122-fe1ba3ef80cb (/c1/_internal_ping) 2020-03-31 09:21:08,328 [534] [DEBUG] [app] Ending request: urn:request:cf12d4ac-34c3-4c0e-b122-fe1ba3ef80cb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:08,328 [534] [DEBUG] [app] Ending request: urn:request:cf12d4ac-34c3-4c0e-b122-fe1ba3ef80cb (/c1/_internal_ping) 2020-03-31 09:21:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:08,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:08,331 [518] [DEBUG] [app] Ending request: urn:request:942bfe27-580d-49c8-83d9-ff9bc4abaff8 (/health/instance) 2020-03-31 09:21:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:21:08,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:21:08,331 [518] [DEBUG] [app] Ending request: urn:request:942bfe27-580d-49c8-83d9-ff9bc4abaff8 (/health/instance) 2020-03-31 09:21:08,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:21:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:21:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:21.440707+00:00 (in 9.999168 seconds) 2020-03-31 09:21:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:21 UTC)" (scheduled at 2020-03-31 09:21:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:21:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:21.440707+00:00 (in 9.999168 seconds) 2020-03-31 09:21:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:21 UTC)" (scheduled at 2020-03-31 09:21:11.440707+00:00) 2020-03-31 09:21:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:21:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:21:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:11,461 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:21:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:21:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:21 UTC)" executed successfully 2020-03-31 09:21:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:21:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:24.323352+00:00 (in 9.999308 seconds) 2020-03-31 09:21:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:24 UTC)" (scheduled at 2020-03-31 09:21:14.323352+00:00) 2020-03-31 09:21:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:21:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:24.323352+00:00 (in 9.999308 seconds) 2020-03-31 09:21:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:24 UTC)" (scheduled at 2020-03-31 09:21:14.323352+00:00) 2020-03-31 09:21:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 14, 324780), True, datetime.datetime(2020, 3, 31, 9, 21, 14, 324780), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:21:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 14, 324780), True, datetime.datetime(2020, 3, 31, 9, 21, 14, 324780), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:21:14,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:21:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:21:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:24 UTC)" executed successfully 2020-03-31 09:21:14,446 [518] [DEBUG] [app] Starting request: urn:request:a6876f58-fca6-4b49-9f22-261c9127769d (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:14,446 [518] [DEBUG] [app] Starting request: urn:request:a6876f58-fca6-4b49-9f22-261c9127769d (/health/instance) 2020-03-31 09:21:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:14,453 [553] [DEBUG] [app] Starting request: urn:request:db0ea4f6-d47e-4441-a769-97415fea8d01 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:14,453 [553] [DEBUG] [app] Starting request: urn:request:db0ea4f6-d47e-4441-a769-97415fea8d01 (/v1/_internal_ping) 2020-03-31 09:21:14,455 [553] [DEBUG] [app] Ending request: urn:request:db0ea4f6-d47e-4441-a769-97415fea8d01 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:14,455 [553] [DEBUG] [app] Ending request: urn:request:db0ea4f6-d47e-4441-a769-97415fea8d01 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:21:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:21:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:21:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:14,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:14,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:14,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:14,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:14,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:14,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:14,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:14,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:14,491 [518] [DEBUG] [app] Starting request: urn:request:2eb8d915-d5b1-444d-9545-dba618e5a38b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:14,491 [518] [DEBUG] [app] Starting request: urn:request:2eb8d915-d5b1-444d-9545-dba618e5a38b (/_internal_ping) 2020-03-31 09:21:14,492 [518] [DEBUG] [app] Ending request: urn:request:2eb8d915-d5b1-444d-9545-dba618e5a38b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:14,492 [518] [DEBUG] [app] Ending request: urn:request:2eb8d915-d5b1-444d-9545-dba618e5a38b (/_internal_ping) 2020-03-31 09:21:14,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:14,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:14,493 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:14,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:21:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:21:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:21:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:21:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:14,507 [543] [DEBUG] [app] Starting request: urn:request:c90db9f8-1210-46b4-b182-ba37d6555560 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:14,507 [543] [DEBUG] [app] Starting request: urn:request:c90db9f8-1210-46b4-b182-ba37d6555560 (/c1/_internal_ping) 2020-03-31 09:21:14,507 [543] [DEBUG] [app] Ending request: urn:request:c90db9f8-1210-46b4-b182-ba37d6555560 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:14,507 [543] [DEBUG] [app] Ending request: urn:request:c90db9f8-1210-46b4-b182-ba37d6555560 (/c1/_internal_ping) 2020-03-31 09:21:14,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:14,508 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:14,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:14,510 [518] [DEBUG] [app] Ending request: urn:request:a6876f58-fca6-4b49-9f22-261c9127769d (/health/instance) 2020-03-31 09:21:14,510 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:21:14,508 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:14,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:21:14,510 [518] [DEBUG] [app] Ending request: urn:request:a6876f58-fca6-4b49-9f22-261c9127769d (/health/instance) 2020-03-31 09:21:14,510 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:14,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:21:15,064 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:21:15,064 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:45.064514+00:00 (in 29.999387 seconds) 2020-03-31 09:21:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:45 UTC)" (scheduled at 2020-03-31 09:21:15.064514+00:00) gcworker stdout | 2020-03-31 09:21:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:45.064514+00:00 (in 29.999387 seconds) 2020-03-31 09:21:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:45 UTC)" (scheduled at 2020-03-31 09:21:15.064514+00:00) 2020-03-31 09:21:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436875, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:21:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436875, 1209600, 500, 1, 0]) 2020-03-31 09:21:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:21:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:21:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:21:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:45 UTC)" executed successfully 2020-03-31 09:21:16,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:21:16,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:16,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:26.924497+00:00 (in 9.998120 seconds) 2020-03-31 09:21:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:26 UTC)" (scheduled at 2020-03-31 09:21:16.924497+00:00) 2020-03-31 09:21:16,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:21:16,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:26.924497+00:00 (in 9.998120 seconds) 2020-03-31 09:21:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:26 UTC)" (scheduled at 2020-03-31 09:21:16.924497+00:00) 2020-03-31 09:21:16,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:16,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 16, 927064), True, datetime.datetime(2020, 3, 31, 9, 21, 16, 927064), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:21:16,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 16, 927064), True, datetime.datetime(2020, 3, 31, 9, 21, 16, 927064), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:21:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:21:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:21:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:26 UTC)" executed successfully 2020-03-31 09:21:18,272 [518] [DEBUG] [app] Starting request: urn:request:62bd339d-e67a-4265-8589-67c6989d7194 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:18,272 [518] [DEBUG] [app] Starting request: urn:request:62bd339d-e67a-4265-8589-67c6989d7194 (/health/instance) 2020-03-31 09:21:18,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:18,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:18,277 [556] [DEBUG] [app] Starting request: urn:request:4d57804c-0198-430e-be20-6d74658866e5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:18,277 [556] [DEBUG] [app] Starting request: urn:request:4d57804c-0198-430e-be20-6d74658866e5 (/v1/_internal_ping) 2020-03-31 09:21:18,279 [556] [DEBUG] [app] Ending request: urn:request:4d57804c-0198-430e-be20-6d74658866e5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:18,279 [556] [DEBUG] [app] Ending request: urn:request:4d57804c-0198-430e-be20-6d74658866e5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:18,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:21:18,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:18,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:18,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:18,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:18,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:18,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:18,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:18,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:18,303 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:18,307 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:18,308 [518] [DEBUG] [app] Starting request: urn:request:7279a6dd-abc1-4974-92ec-fc11bc05d387 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:18,308 [518] [DEBUG] [app] Starting request: urn:request:7279a6dd-abc1-4974-92ec-fc11bc05d387 (/_internal_ping) 2020-03-31 09:21:18,309 [518] [DEBUG] [app] Ending request: urn:request:7279a6dd-abc1-4974-92ec-fc11bc05d387 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:18,309 [518] [DEBUG] [app] Ending request: urn:request:7279a6dd-abc1-4974-92ec-fc11bc05d387 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:21:18,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:18,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:21:18,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:18,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:21:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:21:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:21:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:21:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:21:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:18,323 [539] [DEBUG] [app] Starting request: urn:request:a18d23f9-d4b9-4299-bea1-3308a9d1fd02 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:18,323 [539] [DEBUG] [app] Starting request: urn:request:a18d23f9-d4b9-4299-bea1-3308a9d1fd02 (/c1/_internal_ping) 2020-03-31 09:21:18,324 [539] [DEBUG] [app] Ending request: urn:request:a18d23f9-d4b9-4299-bea1-3308a9d1fd02 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:18,324 [539] [DEBUG] [app] Ending request: urn:request:a18d23f9-d4b9-4299-bea1-3308a9d1fd02 (/c1/_internal_ping) 2020-03-31 09:21:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:18,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:21:18,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:18,327 [518] [DEBUG] [app] Ending request: urn:request:62bd339d-e67a-4265-8589-67c6989d7194 (/health/instance) 2020-03-31 09:21:18,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:18,327 [518] [DEBUG] [app] Ending request: urn:request:62bd339d-e67a-4265-8589-67c6989d7194 (/health/instance) 2020-03-31 09:21:18,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:18,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:21:18,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:21:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:21:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:31.440707+00:00 (in 9.998797 seconds) 2020-03-31 09:21:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:31 UTC)" (scheduled at 2020-03-31 09:21:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:21:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:31.440707+00:00 (in 9.998797 seconds) 2020-03-31 09:21:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:31 UTC)" (scheduled at 2020-03-31 09:21:21.440707+00:00) 2020-03-31 09:21:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:21:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:21:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:21:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:21:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:31 UTC)" executed successfully 2020-03-31 09:21:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:21:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:52.113221+00:00 (in 29.997778 seconds) 2020-03-31 09:21:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:52 UTC)" (scheduled at 2020-03-31 09:21:22.113221+00:00) securityworker stdout | 2020-03-31 09:21:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:52.113221+00:00 (in 29.997778 seconds) 2020-03-31 09:21:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:52 UTC)" (scheduled at 2020-03-31 09:21:22.113221+00:00) 2020-03-31 09:21:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:21:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:21:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:21:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:21:22,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:21:22,134 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:21:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:21:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:21:52 UTC)" executed successfully 2020-03-31 09:21:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:21:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:34.323352+00:00 (in 9.999183 seconds) 2020-03-31 09:21:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:34 UTC)" (scheduled at 2020-03-31 09:21:24.323352+00:00) 2020-03-31 09:21:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:21:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:34.323352+00:00 (in 9.999183 seconds) 2020-03-31 09:21:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:34 UTC)" (scheduled at 2020-03-31 09:21:24.323352+00:00) 2020-03-31 09:21:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 24, 324916), True, datetime.datetime(2020, 3, 31, 9, 21, 24, 324916), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:21:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 24, 324916), True, datetime.datetime(2020, 3, 31, 9, 21, 24, 324916), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:21:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:21:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:21:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:34 UTC)" executed successfully 2020-03-31 09:21:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:21:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:33.330112+00:00 (in 8.918351 seconds) 2020-03-31 09:21:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" (scheduled at 2020-03-31 09:21:24.410927+00:00) 2020-03-31 09:21:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:21:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:33.330112+00:00 (in 8.918351 seconds) 2020-03-31 09:21:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" (scheduled at 2020-03-31 09:21:24.410927+00:00) 2020-03-31 09:21:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" executed successfully 2020-03-31 09:21:24,446 [518] [DEBUG] [app] Starting request: urn:request:1de63dbb-3980-478a-a85b-822cab05326d (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:24,446 [518] [DEBUG] [app] Starting request: urn:request:1de63dbb-3980-478a-a85b-822cab05326d (/health/instance) 2020-03-31 09:21:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:24,453 [556] [DEBUG] [app] Starting request: urn:request:09c07d94-18fa-4143-b5a7-7d5ae5d8cb00 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:24,453 [556] [DEBUG] [app] Starting request: urn:request:09c07d94-18fa-4143-b5a7-7d5ae5d8cb00 (/v1/_internal_ping) 2020-03-31 09:21:24,454 [556] [DEBUG] [app] Ending request: urn:request:09c07d94-18fa-4143-b5a7-7d5ae5d8cb00 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:24,454 [556] [DEBUG] [app] Ending request: urn:request:09c07d94-18fa-4143-b5a7-7d5ae5d8cb00 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:24,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:24,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:21:24,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:24,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:24,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:21:24,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:24,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:24,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:24,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:24,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:24,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:24,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:24,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:24,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:24,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:24,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:24,486 [526] [DEBUG] [app] Starting request: urn:request:2205f4ec-2d5b-4b4b-8d72-ab9c138b06b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:24,486 [526] [DEBUG] [app] Starting request: urn:request:2205f4ec-2d5b-4b4b-8d72-ab9c138b06b2 (/_internal_ping) 2020-03-31 09:21:24,487 [526] [DEBUG] [app] Ending request: urn:request:2205f4ec-2d5b-4b4b-8d72-ab9c138b06b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:24,487 [526] [DEBUG] [app] Ending request: urn:request:2205f4ec-2d5b-4b4b-8d72-ab9c138b06b2 (/_internal_ping) 2020-03-31 09:21:24,488 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:24,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:24,488 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:24,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:21:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:21:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:21:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:21:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:21:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:21:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:24,500 [534] [DEBUG] [app] Starting request: urn:request:1c8200f8-a880-4819-aae9-2f650051f2c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:24,500 [534] [DEBUG] [app] Starting request: urn:request:1c8200f8-a880-4819-aae9-2f650051f2c2 (/c1/_internal_ping) 2020-03-31 09:21:24,500 [534] [DEBUG] [app] Ending request: urn:request:1c8200f8-a880-4819-aae9-2f650051f2c2 (/c1/_internal_ping) 2020-03-31 09:21:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:24,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:24,503 [518] [DEBUG] [app] Ending request: urn:request:1de63dbb-3980-478a-a85b-822cab05326d (/health/instance) 2020-03-31 09:21:24,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:21:24,500 [534] [DEBUG] [app] Ending request: urn:request:1c8200f8-a880-4819-aae9-2f650051f2c2 (/c1/_internal_ping) 2020-03-31 09:21:24,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:21:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:21:24,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:24,503 [518] [DEBUG] [app] Ending request: urn:request:1de63dbb-3980-478a-a85b-822cab05326d (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:24,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:24,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:21:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:21:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:36.924497+00:00 (in 9.999223 seconds) 2020-03-31 09:21:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:36 UTC)" (scheduled at 2020-03-31 09:21:26.924497+00:00) 2020-03-31 09:21:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:21:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:36.924497+00:00 (in 9.999223 seconds) 2020-03-31 09:21:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:36 UTC)" (scheduled at 2020-03-31 09:21:26.924497+00:00) 2020-03-31 09:21:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 26, 926060), True, datetime.datetime(2020, 3, 31, 9, 21, 26, 926060), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:21:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 26, 926060), True, datetime.datetime(2020, 3, 31, 9, 21, 26, 926060), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:21:26,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:21:26,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:21:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:36 UTC)" executed successfully 2020-03-31 09:21:28,273 [527] [DEBUG] [app] Starting request: urn:request:5ac5118b-6954-4d03-bce1-d78f446d68ba (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:28,273 [527] [DEBUG] [app] Starting request: urn:request:5ac5118b-6954-4d03-bce1-d78f446d68ba (/health/instance) 2020-03-31 09:21:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:28,282 [542] [DEBUG] [app] Starting request: urn:request:b389716c-38e9-415d-885e-169d6e4dd5ed (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:28,282 [542] [DEBUG] [app] Starting request: urn:request:b389716c-38e9-415d-885e-169d6e4dd5ed (/v1/_internal_ping) 2020-03-31 09:21:28,284 [542] [DEBUG] [app] Ending request: urn:request:b389716c-38e9-415d-885e-169d6e4dd5ed (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:28,284 [542] [DEBUG] [app] Ending request: urn:request:b389716c-38e9-415d-885e-169d6e4dd5ed (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:21:28,285 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:21:28,285 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:28,287 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:21:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:21:28,287 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:28,290 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:28,290 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:28,291 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:28,291 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:28,299 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:28,299 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:28,316 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:28,316 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:28,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:28,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:28,322 [521] [DEBUG] [app] Starting request: urn:request:bf2e3c3c-96c2-4217-8117-35486f9abb10 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:28,322 [521] [DEBUG] [app] Starting request: urn:request:bf2e3c3c-96c2-4217-8117-35486f9abb10 (/_internal_ping) 2020-03-31 09:21:28,323 [521] [DEBUG] [app] Ending request: urn:request:bf2e3c3c-96c2-4217-8117-35486f9abb10 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:28,323 [521] [DEBUG] [app] Ending request: urn:request:bf2e3c3c-96c2-4217-8117-35486f9abb10 (/_internal_ping) 2020-03-31 09:21:28,324 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:28,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:28,324 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:28,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:21:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:28,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:28,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:28,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:21:28,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:21:28,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:28,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:28,342 [534] [DEBUG] [app] Starting request: urn:request:cb350725-7fcf-4529-a212-a45e88e03a27 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:28,342 [534] [DEBUG] [app] Starting request: urn:request:cb350725-7fcf-4529-a212-a45e88e03a27 (/c1/_internal_ping) 2020-03-31 09:21:28,343 [534] [DEBUG] [app] Ending request: urn:request:cb350725-7fcf-4529-a212-a45e88e03a27 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:28,343 [534] [DEBUG] [app] Ending request: urn:request:cb350725-7fcf-4529-a212-a45e88e03a27 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:28,344 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:28,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:21:28,344 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:28,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:28,346 [527] [DEBUG] [app] Ending request: urn:request:5ac5118b-6954-4d03-bce1-d78f446d68ba (/health/instance) 2020-03-31 09:21:28,346 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:28,346 [527] [DEBUG] [app] Ending request: urn:request:5ac5118b-6954-4d03-bce1-d78f446d68ba (/health/instance) 2020-03-31 09:21:28,346 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:28,347 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.076 49 0.076) gunicorn-web stdout | 2020-03-31 09:21:28,347 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.076 49 0.076) 2020-03-31 09:21:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:21:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:00.337699+00:00 (in 29.999129 seconds) 2020-03-31 09:21:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:00 UTC)" (scheduled at 2020-03-31 09:21:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:21:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:00.337699+00:00 (in 29.999129 seconds) 2020-03-31 09:21:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:00 UTC)" (scheduled at 2020-03-31 09:21:30.337699+00:00) 2020-03-31 09:21:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 21, 30, 339416), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:21:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 21, 30, 339416), False, 50, 1, 0]) 2020-03-31 09:21:30,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:21:30,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:21:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:21:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:00 UTC)" executed successfully 2020-03-31 09:21:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:21:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:41.440707+00:00 (in 9.996214 seconds) 2020-03-31 09:21:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:41 UTC)" (scheduled at 2020-03-31 09:21:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:21:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:41.440707+00:00 (in 9.996214 seconds) 2020-03-31 09:21:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:41 UTC)" (scheduled at 2020-03-31 09:21:31.440707+00:00) 2020-03-31 09:21:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:21:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:21:31,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:21:31,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:31,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:31,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:21:31,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:31,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:41 UTC)" executed successfully 2020-03-31 09:21:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:21:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:03.330112+00:00 (in 29.999154 seconds) 2020-03-31 09:21:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:03 UTC)" (scheduled at 2020-03-31 09:21:33.330112+00:00) 2020-03-31 09:21:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:21:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:03.330112+00:00 (in 29.999154 seconds) 2020-03-31 09:21:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:03 UTC)" (scheduled at 2020-03-31 09:21:33.330112+00:00) 2020-03-31 09:21:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 33, 331886), True, datetime.datetime(2020, 3, 31, 9, 21, 33, 331886), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:21:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 33, 331886), True, datetime.datetime(2020, 3, 31, 9, 21, 33, 331886), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:21:33,353 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:21:33,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:33,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:33,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:21:33,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:33,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:03 UTC)" executed successfully 2020-03-31 09:21:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:21:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:43.539999+00:00 (in 10.172300 seconds) 2020-03-31 09:21:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:33 UTC)" (scheduled at 2020-03-31 09:21:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:21:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:43.539999+00:00 (in 10.172300 seconds) 2020-03-31 09:21:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:33 UTC)" (scheduled at 2020-03-31 09:21:33.366791+00:00) 2020-03-31 09:21:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:21:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 33, 368556), True, datetime.datetime(2020, 3, 31, 9, 21, 33, 368556), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:21:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 33, 368556), True, datetime.datetime(2020, 3, 31, 9, 21, 33, 368556), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:21:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:21:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:21:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:33 UTC)" executed successfully 2020-03-31 09:21:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:21:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:44.323352+00:00 (in 9.999260 seconds) 2020-03-31 09:21:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:44 UTC)" (scheduled at 2020-03-31 09:21:34.323352+00:00) 2020-03-31 09:21:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:21:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:44.323352+00:00 (in 9.999260 seconds) 2020-03-31 09:21:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:44 UTC)" (scheduled at 2020-03-31 09:21:34.323352+00:00) 2020-03-31 09:21:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 34, 325063), True, datetime.datetime(2020, 3, 31, 9, 21, 34, 325063), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:21:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 34, 325063), True, datetime.datetime(2020, 3, 31, 9, 21, 34, 325063), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:21:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:21:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:21:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:44 UTC)" executed successfully 2020-03-31 09:21:34,446 [526] [DEBUG] [app] Starting request: urn:request:6e2419d7-577a-437d-ada7-8e11e94a14d4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:34,446 [526] [DEBUG] [app] Starting request: urn:request:6e2419d7-577a-437d-ada7-8e11e94a14d4 (/health/instance) 2020-03-31 09:21:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:34,453 [552] [DEBUG] [app] Starting request: urn:request:567a2653-447a-43c7-b0aa-dd176cf2602f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:34,453 [552] [DEBUG] [app] Starting request: urn:request:567a2653-447a-43c7-b0aa-dd176cf2602f (/v1/_internal_ping) 2020-03-31 09:21:34,454 [552] [DEBUG] [app] Ending request: urn:request:567a2653-447a-43c7-b0aa-dd176cf2602f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:34,454 [552] [DEBUG] [app] Ending request: urn:request:567a2653-447a-43c7-b0aa-dd176cf2602f (/v1/_internal_ping) 2020-03-31 09:21:34,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:21:34,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:34,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:21:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:21:34,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:34,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:34,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:34,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:34,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:34,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:34,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:34,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:34,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:34,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:34,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:34,491 [527] [DEBUG] [app] Starting request: urn:request:a18a57f6-f9e8-40d5-818e-5e666607ba63 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:34,491 [527] [DEBUG] [app] Starting request: urn:request:a18a57f6-f9e8-40d5-818e-5e666607ba63 (/_internal_ping) 2020-03-31 09:21:34,492 [527] [DEBUG] [app] Ending request: urn:request:a18a57f6-f9e8-40d5-818e-5e666607ba63 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:34,492 [527] [DEBUG] [app] Ending request: urn:request:a18a57f6-f9e8-40d5-818e-5e666607ba63 (/_internal_ping) 2020-03-31 09:21:34,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:34,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:21:34,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:34,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:21:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:21:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:21:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:21:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:21:34,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:34,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:34,507 [543] [DEBUG] [app] Starting request: urn:request:2c667aa6-8577-4c66-87a7-c8d12064563b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:34,507 [543] [DEBUG] [app] Starting request: urn:request:2c667aa6-8577-4c66-87a7-c8d12064563b (/c1/_internal_ping) 2020-03-31 09:21:34,508 [543] [DEBUG] [app] Ending request: urn:request:2c667aa6-8577-4c66-87a7-c8d12064563b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:34,508 [543] [DEBUG] [app] Ending request: urn:request:2c667aa6-8577-4c66-87a7-c8d12064563b (/c1/_internal_ping) 2020-03-31 09:21:34,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:34,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:34,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:21:34,509 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:34,510 [526] [DEBUG] [app] Ending request: urn:request:6e2419d7-577a-437d-ada7-8e11e94a14d4 (/health/instance) 2020-03-31 09:21:34,510 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:34,510 [526] [DEBUG] [app] Ending request: urn:request:6e2419d7-577a-437d-ada7-8e11e94a14d4 (/health/instance) 2020-03-31 09:21:34,510 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:34,511 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:21:34,511 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:21:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:21:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:38.010934+00:00 (in 1.085708 seconds) storagereplication stdout | 2020-03-31 09:21:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:38.010934+00:00 (in 1.085708 seconds) 2020-03-31 09:21:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:46 UTC)" (scheduled at 2020-03-31 09:21:36.924497+00:00) 2020-03-31 09:21:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 36, 926537), True, datetime.datetime(2020, 3, 31, 9, 21, 36, 926537), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:21:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:46 UTC)" (scheduled at 2020-03-31 09:21:36.924497+00:00) 2020-03-31 09:21:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 36, 926537), True, datetime.datetime(2020, 3, 31, 9, 21, 36, 926537), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:21:36,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:21:36,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:21:36,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:46 UTC)" executed successfully 2020-03-31 09:21:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:21:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:46.924497+00:00 (in 8.912721 seconds) 2020-03-31 09:21:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:38 UTC)" (scheduled at 2020-03-31 09:21:38.010934+00:00) 2020-03-31 09:21:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:21:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:46.924497+00:00 (in 8.912721 seconds) 2020-03-31 09:21:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:38 UTC)" (scheduled at 2020-03-31 09:21:38.010934+00:00) 2020-03-31 09:21:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:38 UTC)" executed successfully 2020-03-31 09:21:38,273 [518] [DEBUG] [app] Starting request: urn:request:0ebae2f1-fefe-4f0e-afd0-637f72aebc8b (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:38,273 [518] [DEBUG] [app] Starting request: urn:request:0ebae2f1-fefe-4f0e-afd0-637f72aebc8b (/health/instance) 2020-03-31 09:21:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:38,280 [551] [DEBUG] [app] Starting request: urn:request:7d631785-ae1c-4ee6-bf21-b66b71819072 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:38,280 [551] [DEBUG] [app] Starting request: urn:request:7d631785-ae1c-4ee6-bf21-b66b71819072 (/v1/_internal_ping) 2020-03-31 09:21:38,281 [551] [DEBUG] [app] Ending request: urn:request:7d631785-ae1c-4ee6-bf21-b66b71819072 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:38,281 [551] [DEBUG] [app] Ending request: urn:request:7d631785-ae1c-4ee6-bf21-b66b71819072 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:38,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:21:38,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:38,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:38,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:38,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:38,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:38,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:38,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:38,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:38,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:38,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:38,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:38,314 [518] [DEBUG] [app] Starting request: urn:request:6d95f890-b301-40c7-a3e2-b933bb236384 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:38,314 [518] [DEBUG] [app] Starting request: urn:request:6d95f890-b301-40c7-a3e2-b933bb236384 (/_internal_ping) 2020-03-31 09:21:38,315 [518] [DEBUG] [app] Ending request: urn:request:6d95f890-b301-40c7-a3e2-b933bb236384 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:38,315 [518] [DEBUG] [app] Ending request: urn:request:6d95f890-b301-40c7-a3e2-b933bb236384 (/_internal_ping) 2020-03-31 09:21:38,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:38,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:21:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:21:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:21:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:21:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:38,328 [534] [DEBUG] [app] Starting request: urn:request:28beb9d0-ee91-4250-8850-e239248b4469 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:38,328 [534] [DEBUG] [app] Starting request: urn:request:28beb9d0-ee91-4250-8850-e239248b4469 (/c1/_internal_ping) 2020-03-31 09:21:38,328 [534] [DEBUG] [app] Ending request: urn:request:28beb9d0-ee91-4250-8850-e239248b4469 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:38,328 [534] [DEBUG] [app] Ending request: urn:request:28beb9d0-ee91-4250-8850-e239248b4469 (/c1/_internal_ping) 2020-03-31 09:21:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:38,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:38,331 [518] [DEBUG] [app] Ending request: urn:request:0ebae2f1-fefe-4f0e-afd0-637f72aebc8b (/health/instance) 2020-03-31 09:21:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:21:38,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:38,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:21:38,331 [518] [DEBUG] [app] Ending request: urn:request:0ebae2f1-fefe-4f0e-afd0-637f72aebc8b (/health/instance) 2020-03-31 09:21:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:38,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:21:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:21:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:51.440707+00:00 (in 9.998739 seconds) 2020-03-31 09:21:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:51 UTC)" (scheduled at 2020-03-31 09:21:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:21:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:51.440707+00:00 (in 9.998739 seconds) 2020-03-31 09:21:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:51 UTC)" (scheduled at 2020-03-31 09:21:41.440707+00:00) 2020-03-31 09:21:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:21:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:21:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:21:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:21:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:51 UTC)" executed successfully 2020-03-31 09:21:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:21:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:43,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:33.366791+00:00 (in 49.826094 seconds) 2020-03-31 09:21:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:43 UTC)" (scheduled at 2020-03-31 09:21:43.539999+00:00) 2020-03-31 09:21:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:21:43,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:33.366791+00:00 (in 49.826094 seconds) 2020-03-31 09:21:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:43 UTC)" (scheduled at 2020-03-31 09:21:43.539999+00:00) 2020-03-31 09:21:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:43 UTC)" executed successfully 2020-03-31 09:21:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:21:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:49.498966+00:00 (in 5.174801 seconds) 2020-03-31 09:21:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:54 UTC)" (scheduled at 2020-03-31 09:21:44.323352+00:00) 2020-03-31 09:21:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:21:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:49.498966+00:00 (in 5.174801 seconds) 2020-03-31 09:21:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:54 UTC)" (scheduled at 2020-03-31 09:21:44.323352+00:00) 2020-03-31 09:21:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 44, 325042), True, datetime.datetime(2020, 3, 31, 9, 21, 44, 325042), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:21:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 44, 325042), True, datetime.datetime(2020, 3, 31, 9, 21, 44, 325042), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:21:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:21:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:21:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:54 UTC)" executed successfully 2020-03-31 09:21:44,446 [527] [DEBUG] [app] Starting request: urn:request:ef7194b1-3d85-4570-b576-358e0cf8a520 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:44,446 [527] [DEBUG] [app] Starting request: urn:request:ef7194b1-3d85-4570-b576-358e0cf8a520 (/health/instance) 2020-03-31 09:21:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:44,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:44,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:44,453 [546] [DEBUG] [app] Starting request: urn:request:d2e51e57-3de3-49a7-82da-7aad395b3a54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:44,453 [546] [DEBUG] [app] Starting request: urn:request:d2e51e57-3de3-49a7-82da-7aad395b3a54 (/v1/_internal_ping) 2020-03-31 09:21:44,454 [546] [DEBUG] [app] Ending request: urn:request:d2e51e57-3de3-49a7-82da-7aad395b3a54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:44,454 [546] [DEBUG] [app] Ending request: urn:request:d2e51e57-3de3-49a7-82da-7aad395b3a54 (/v1/_internal_ping) 2020-03-31 09:21:44,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:21:44,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:21:44,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:21:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:44,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:44,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:44,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:44,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:44,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:44,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:44,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:44,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:44,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:44,484 [527] [DEBUG] [app] Starting request: urn:request:572f43bc-8f12-40d0-a8ca-200b1bb18c24 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:44,484 [527] [DEBUG] [app] Starting request: urn:request:572f43bc-8f12-40d0-a8ca-200b1bb18c24 (/_internal_ping) 2020-03-31 09:21:44,485 [527] [DEBUG] [app] Ending request: urn:request:572f43bc-8f12-40d0-a8ca-200b1bb18c24 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:44,485 [527] [DEBUG] [app] Ending request: urn:request:572f43bc-8f12-40d0-a8ca-200b1bb18c24 (/_internal_ping) 2020-03-31 09:21:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:44,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:44,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:44,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:21:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:44,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:21:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:21:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:21:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:21:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:44,499 [539] [DEBUG] [app] Starting request: urn:request:b712d5bd-5c50-4128-a8fd-c2aa4942dacd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:44,499 [539] [DEBUG] [app] Starting request: urn:request:b712d5bd-5c50-4128-a8fd-c2aa4942dacd (/c1/_internal_ping) 2020-03-31 09:21:44,500 [539] [DEBUG] [app] Ending request: urn:request:b712d5bd-5c50-4128-a8fd-c2aa4942dacd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:44,500 [539] [DEBUG] [app] Ending request: urn:request:b712d5bd-5c50-4128-a8fd-c2aa4942dacd (/c1/_internal_ping) 2020-03-31 09:21:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:44,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:44,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:44,502 [527] [DEBUG] [app] Ending request: urn:request:ef7194b1-3d85-4570-b576-358e0cf8a520 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:21:44,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:44,502 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:44,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:21:44,502 [527] [DEBUG] [app] Ending request: urn:request:ef7194b1-3d85-4570-b576-358e0cf8a520 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:44,502 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:44,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:21:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:21:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:15.064514+00:00 (in 29.998894 seconds) 2020-03-31 09:21:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:15 UTC)" (scheduled at 2020-03-31 09:21:45.064514+00:00) gcworker stdout | 2020-03-31 09:21:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:15.064514+00:00 (in 29.998894 seconds) 2020-03-31 09:21:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:15 UTC)" (scheduled at 2020-03-31 09:21:45.064514+00:00) 2020-03-31 09:21:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436905, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:21:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436905, 1209600, 500, 1, 0]) 2020-03-31 09:21:45,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:21:45,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:21:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:21:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:15 UTC)" executed successfully 2020-03-31 09:21:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:21:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:56.924497+00:00 (in 9.998850 seconds) 2020-03-31 09:21:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:56 UTC)" (scheduled at 2020-03-31 09:21:46.924497+00:00) 2020-03-31 09:21:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:21:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:56.924497+00:00 (in 9.998850 seconds) 2020-03-31 09:21:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:56 UTC)" (scheduled at 2020-03-31 09:21:46.924497+00:00) 2020-03-31 09:21:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 46, 926754), True, datetime.datetime(2020, 3, 31, 9, 21, 46, 926754), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:21:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 46, 926754), True, datetime.datetime(2020, 3, 31, 9, 21, 46, 926754), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:21:46,950 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:21:46,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:46,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:46,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:21:46,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:46,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:56 UTC)" executed successfully 2020-03-31 09:21:48,274 [526] [DEBUG] [app] Starting request: urn:request:97084372-e465-4f88-beb2-2b029b6b4e4b (/health/instance) 2020-03-31 09:21:48,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:48,274 [526] [DEBUG] [app] Starting request: urn:request:97084372-e465-4f88-beb2-2b029b6b4e4b (/health/instance) 2020-03-31 09:21:48,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:48,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:48,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:48,282 [555] [DEBUG] [app] Starting request: urn:request:2b531433-67f0-46ad-8b29-2c311ca87f29 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:48,282 [555] [DEBUG] [app] Starting request: urn:request:2b531433-67f0-46ad-8b29-2c311ca87f29 (/v1/_internal_ping) 2020-03-31 09:21:48,283 [555] [DEBUG] [app] Ending request: urn:request:2b531433-67f0-46ad-8b29-2c311ca87f29 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:48,283 [555] [DEBUG] [app] Ending request: urn:request:2b531433-67f0-46ad-8b29-2c311ca87f29 (/v1/_internal_ping) 2020-03-31 09:21:48,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:48,285 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:21:48,286 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:48,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:48,284 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:48,285 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:48,289 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:21:48,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:21:48,286 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:21:48,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:48,289 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:48,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:48,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:48,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:48,312 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:48,312 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:48,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:48,316 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:48,317 [526] [DEBUG] [app] Starting request: urn:request:5441871c-0e94-43a8-85d2-cc9e1292cf67 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:48,317 [526] [DEBUG] [app] Starting request: urn:request:5441871c-0e94-43a8-85d2-cc9e1292cf67 (/_internal_ping) 2020-03-31 09:21:48,318 [526] [DEBUG] [app] Ending request: urn:request:5441871c-0e94-43a8-85d2-cc9e1292cf67 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:48,318 [526] [DEBUG] [app] Ending request: urn:request:5441871c-0e94-43a8-85d2-cc9e1292cf67 (/_internal_ping) 2020-03-31 09:21:48,319 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:48,319 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:48,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:21:48,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:21:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:48,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:48,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:48,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:21:48,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:21:48,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:48,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:48,332 [537] [DEBUG] [app] Starting request: urn:request:840eb32a-0e3d-457a-abd0-62dec25a808b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:48,332 [537] [DEBUG] [app] Starting request: urn:request:840eb32a-0e3d-457a-abd0-62dec25a808b (/c1/_internal_ping) 2020-03-31 09:21:48,333 [537] [DEBUG] [app] Ending request: urn:request:840eb32a-0e3d-457a-abd0-62dec25a808b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:48,333 [537] [DEBUG] [app] Ending request: urn:request:840eb32a-0e3d-457a-abd0-62dec25a808b (/c1/_internal_ping) 2020-03-31 09:21:48,334 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:48,334 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:48,334 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:21:48,334 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:48,336 [526] [DEBUG] [app] Ending request: urn:request:97084372-e465-4f88-beb2-2b029b6b4e4b (/health/instance) 2020-03-31 09:21:48,336 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:48,336 [526] [DEBUG] [app] Ending request: urn:request:97084372-e465-4f88-beb2-2b029b6b4e4b (/health/instance) 2020-03-31 09:21:48,336 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:48,337 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:21:48,337 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:21:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:21:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:49 UTC)" (scheduled at 2020-03-31 09:21:49.498966+00:00) 2020-03-31 09:21:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:21:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:49 UTC)" (scheduled at 2020-03-31 09:21:49.498966+00:00) 2020-03-31 09:21:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:49 UTC)" executed successfully 2020-03-31 09:21:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:54.323352+00:00 (in 4.823472 seconds) notificationworker stdout | 2020-03-31 09:21:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:54.323352+00:00 (in 4.823472 seconds) 2020-03-31 09:21:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:21:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:59.560326+00:00 (in 8.448355 seconds) 2020-03-31 09:21:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:51 UTC)" (scheduled at 2020-03-31 09:21:51.110610+00:00) 2020-03-31 09:21:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:21:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:21:59.560326+00:00 (in 8.448355 seconds) 2020-03-31 09:21:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:51 UTC)" (scheduled at 2020-03-31 09:21:51.110610+00:00) 2020-03-31 09:21:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 51, 113025), True, datetime.datetime(2020, 3, 31, 9, 21, 51, 113025), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:21:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 51, 113025), True, datetime.datetime(2020, 3, 31, 9, 21, 51, 113025), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:21:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:21:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:21:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:51 UTC)" executed successfully 2020-03-31 09:21:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:21:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:01 UTC)" (scheduled at 2020-03-31 09:21:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:21:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:01 UTC)" (scheduled at 2020-03-31 09:21:51.440707+00:00) 2020-03-31 09:21:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:21:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:21:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:01.440707+00:00 (in 9.998999 seconds) repositoryactioncounter stdout | 2020-03-31 09:21:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:01.440707+00:00 (in 9.998999 seconds) 2020-03-31 09:21:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:21:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:21:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:21:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:01 UTC)" executed successfully 2020-03-31 09:21:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:21:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:22.113221+00:00 (in 29.998963 seconds) 2020-03-31 09:21:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:22 UTC)" (scheduled at 2020-03-31 09:21:52.113221+00:00) securityworker stdout | 2020-03-31 09:21:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:22.113221+00:00 (in 29.998963 seconds) 2020-03-31 09:21:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:22 UTC)" (scheduled at 2020-03-31 09:21:52.113221+00:00) 2020-03-31 09:21:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:21:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:21:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:21:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:21:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:21:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:21:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:21:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:22 UTC)" executed successfully 2020-03-31 09:21:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:21:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:04.323352+00:00 (in 9.998264 seconds) 2020-03-31 09:21:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:04 UTC)" (scheduled at 2020-03-31 09:21:54.323352+00:00) 2020-03-31 09:21:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:21:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:04.323352+00:00 (in 9.998264 seconds) 2020-03-31 09:21:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:04 UTC)" (scheduled at 2020-03-31 09:21:54.323352+00:00) 2020-03-31 09:21:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 54, 325985), True, datetime.datetime(2020, 3, 31, 9, 21, 54, 325985), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:21:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 54, 325985), True, datetime.datetime(2020, 3, 31, 9, 21, 54, 325985), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:21:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:21:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:21:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:04 UTC)" executed successfully 2020-03-31 09:21:54,446 [521] [DEBUG] [app] Starting request: urn:request:ac248b4d-20c0-4cb7-b848-479848e7f443 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:54,446 [521] [DEBUG] [app] Starting request: urn:request:ac248b4d-20c0-4cb7-b848-479848e7f443 (/health/instance) 2020-03-31 09:21:54,448 [521] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:54,448 [521] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:54,452 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:54,452 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:54,453 [550] [DEBUG] [app] Starting request: urn:request:a7d7ede9-5493-45ec-ae6f-23b630bf5654 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:54,453 [550] [DEBUG] [app] Starting request: urn:request:a7d7ede9-5493-45ec-ae6f-23b630bf5654 (/v1/_internal_ping) 2020-03-31 09:21:54,454 [550] [DEBUG] [app] Ending request: urn:request:a7d7ede9-5493-45ec-ae6f-23b630bf5654 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:54,454 [550] [DEBUG] [app] Ending request: urn:request:a7d7ede9-5493-45ec-ae6f-23b630bf5654 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:54,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:54,455 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:21:54,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:54,456 [521] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:21:54,455 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:21:54,456 [521] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:21:54,457 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:21:54,457 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:54,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:54,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:54,459 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:54,459 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:54,467 [521] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:54,467 [521] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:54,482 [521] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:54,482 [521] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:54,486 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:54,486 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:54,487 [527] [DEBUG] [app] Starting request: urn:request:61b9445a-e9ed-40dc-b4a1-b3eecb4a8f2a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:54,487 [527] [DEBUG] [app] Starting request: urn:request:61b9445a-e9ed-40dc-b4a1-b3eecb4a8f2a (/_internal_ping) 2020-03-31 09:21:54,488 [527] [DEBUG] [app] Ending request: urn:request:61b9445a-e9ed-40dc-b4a1-b3eecb4a8f2a (/_internal_ping) 2020-03-31 09:21:54,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:54,489 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:21:54,488 [527] [DEBUG] [app] Ending request: urn:request:61b9445a-e9ed-40dc-b4a1-b3eecb4a8f2a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:54,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:54,489 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:21:54,493 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:54,493 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:54,494 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:21:54,494 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:21:54,498 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:54,498 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:54,499 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:21:54,499 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:21:54,503 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:54,503 [521] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:54,505 [537] [DEBUG] [app] Starting request: urn:request:90cd334d-fcf8-4e9b-b61c-b26c9361e81f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:54,505 [537] [DEBUG] [app] Starting request: urn:request:90cd334d-fcf8-4e9b-b61c-b26c9361e81f (/c1/_internal_ping) 2020-03-31 09:21:54,505 [537] [DEBUG] [app] Ending request: urn:request:90cd334d-fcf8-4e9b-b61c-b26c9361e81f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:54,505 [537] [DEBUG] [app] Ending request: urn:request:90cd334d-fcf8-4e9b-b61c-b26c9361e81f (/c1/_internal_ping) 2020-03-31 09:21:54,506 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:54,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:54,506 [521] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:54,508 [521] [DEBUG] [app] Ending request: urn:request:ac248b4d-20c0-4cb7-b848-479848e7f443 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:21:54,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:54,508 [521] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:54,509 [521] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:21:54,508 [521] [DEBUG] [app] Ending request: urn:request:ac248b4d-20c0-4cb7-b848-479848e7f443 (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:54,508 [521] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:54,509 [521] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:21:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:21:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:56 UTC)" (scheduled at 2020-03-31 09:21:56.924497+00:00) 2020-03-31 09:21:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:21:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:21:56 UTC)" (scheduled at 2020-03-31 09:21:56.924497+00:00) 2020-03-31 09:21:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:21:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 56, 925844), True, datetime.datetime(2020, 3, 31, 9, 21, 56, 925844), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:21:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 21, 56, 925844), True, datetime.datetime(2020, 3, 31, 9, 21, 56, 925844), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:21:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:03.010318+00:00 (in 6.085005 seconds) storagereplication stdout | 2020-03-31 09:21:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:03.010318+00:00 (in 6.085005 seconds) 2020-03-31 09:21:56,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:21:56,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:21:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:21:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:06 UTC)" executed successfully 2020-03-31 09:21:58,273 [518] [DEBUG] [app] Starting request: urn:request:4ffd27a2-58f7-4c0a-a4d0-b92e307dfcdc (/health/instance) gunicorn-web stdout | 2020-03-31 09:21:58,273 [518] [DEBUG] [app] Starting request: urn:request:4ffd27a2-58f7-4c0a-a4d0-b92e307dfcdc (/health/instance) 2020-03-31 09:21:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:58,281 [556] [DEBUG] [app] Starting request: urn:request:da07b446-4dad-4353-87fe-2cd7ff4bf279 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:58,281 [556] [DEBUG] [app] Starting request: urn:request:da07b446-4dad-4353-87fe-2cd7ff4bf279 (/v1/_internal_ping) 2020-03-31 09:21:58,282 [556] [DEBUG] [app] Ending request: urn:request:da07b446-4dad-4353-87fe-2cd7ff4bf279 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:21:58,282 [556] [DEBUG] [app] Ending request: urn:request:da07b446-4dad-4353-87fe-2cd7ff4bf279 (/v1/_internal_ping) 2020-03-31 09:21:58,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:21:58,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:21:58,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:21:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:21:58,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:21:58,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:21:58,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:21:58,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:21:58,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:21:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:21:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:21:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:21:58,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:21:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:58,313 [518] [DEBUG] [app] Starting request: urn:request:a5d64d6a-cd12-4b16-af67-f4a65240f8a4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:58,313 [518] [DEBUG] [app] Starting request: urn:request:a5d64d6a-cd12-4b16-af67-f4a65240f8a4 (/_internal_ping) 2020-03-31 09:21:58,314 [518] [DEBUG] [app] Ending request: urn:request:a5d64d6a-cd12-4b16-af67-f4a65240f8a4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:21:58,314 [518] [DEBUG] [app] Ending request: urn:request:a5d64d6a-cd12-4b16-af67-f4a65240f8a4 (/_internal_ping) 2020-03-31 09:21:58,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:21:58,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:21:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:21:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:21:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:21:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:21:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:21:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:21:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:21:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:21:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:21:58,327 [539] [DEBUG] [app] Starting request: urn:request:e84e8b1e-298d-433b-9135-db59cc0a4940 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:58,327 [539] [DEBUG] [app] Starting request: urn:request:e84e8b1e-298d-433b-9135-db59cc0a4940 (/c1/_internal_ping) 2020-03-31 09:21:58,328 [539] [DEBUG] [app] Ending request: urn:request:e84e8b1e-298d-433b-9135-db59cc0a4940 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:21:58,328 [539] [DEBUG] [app] Ending request: urn:request:e84e8b1e-298d-433b-9135-db59cc0a4940 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:58,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:21:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:21:58,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:21:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:21:58,330 [518] [DEBUG] [app] Ending request: urn:request:4ffd27a2-58f7-4c0a-a4d0-b92e307dfcdc (/health/instance) 2020-03-31 09:21:58,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:21:58,330 [518] [DEBUG] [app] Ending request: urn:request:4ffd27a2-58f7-4c0a-a4d0-b92e307dfcdc (/health/instance) 2020-03-31 09:21:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:21:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:21:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:21:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:21:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:21:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:21:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:21:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:51.110610+00:00 (in 51.549378 seconds) 2020-03-31 09:21:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:59 UTC)" (scheduled at 2020-03-31 09:21:59.560326+00:00) 2020-03-31 09:21:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:21:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:51.110610+00:00 (in 51.549378 seconds) 2020-03-31 09:21:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:59 UTC)" (scheduled at 2020-03-31 09:21:59.560326+00:00) 2020-03-31 09:21:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:21:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:59 UTC)" executed successfully 2020-03-31 09:22:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:22:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:00 UTC)" (scheduled at 2020-03-31 09:22:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:22:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:00 UTC)" (scheduled at 2020-03-31 09:22:00.337699+00:00) 2020-03-31 09:22:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:30.337699+00:00 (in 29.998733 seconds) buildlogsarchiver stdout | 2020-03-31 09:22:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:30.337699+00:00 (in 29.998733 seconds) 2020-03-31 09:22:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 22, 0, 339665), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:22:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 22, 0, 339665), False, 50, 1, 0]) 2020-03-31 09:22:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:22:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:22:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:22:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:30 UTC)" executed successfully 2020-03-31 09:22:01,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:22:01,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:11.440707+00:00 (in 9.995902 seconds) 2020-03-31 09:22:01,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:11 UTC)" (scheduled at 2020-03-31 09:22:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:22:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:11.440707+00:00 (in 9.995902 seconds) 2020-03-31 09:22:01,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:11 UTC)" (scheduled at 2020-03-31 09:22:01.440707+00:00) 2020-03-31 09:22:01,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:22:01,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:22:01,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:22:01,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:22:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:22:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:11 UTC)" executed successfully 2020-03-31 09:22:03,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:03,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:03,014 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:03 UTC)" (scheduled at 2020-03-31 09:22:03.010318+00:00) 2020-03-31 09:22:03,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:06.924497+00:00 (in 3.910268 seconds) storagereplication stdout | 2020-03-31 09:22:03,014 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:22:03 UTC)" (scheduled at 2020-03-31 09:22:03.010318+00:00) 2020-03-31 09:22:03,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:06.924497+00:00 (in 3.910268 seconds) 2020-03-31 09:22:03,017 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:22:03,017 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), u'imagestoragereplication/%']) 2020-03-31 09:22:03,030 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), True, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:22:03,030 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), True, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), 0, u'imagestoragereplication/%']) 2020-03-31 09:22:03,034 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), True, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:22:03,034 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), True, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 22, 3, 15220), u'imagestoragereplication/%']) 2020-03-31 09:22:03,038 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:22:03,038 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:03,039 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:27:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:03,039 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:27:03 UTC)" executed successfully 2020-03-31 09:22:03,041 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:22:03,041 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:03,042 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:22:03,042 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:03,042 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 09:22:03,042 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:22:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:22:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:24.410927+00:00 (in 21.079883 seconds) 2020-03-31 09:22:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:33 UTC)" (scheduled at 2020-03-31 09:22:03.330112+00:00) 2020-03-31 09:22:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:22:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:24.410927+00:00 (in 21.079883 seconds) 2020-03-31 09:22:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:33 UTC)" (scheduled at 2020-03-31 09:22:03.330112+00:00) 2020-03-31 09:22:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 3, 332360), True, datetime.datetime(2020, 3, 31, 9, 22, 3, 332360), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:22:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 3, 332360), True, datetime.datetime(2020, 3, 31, 9, 22, 3, 332360), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:22:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:22:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:22:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:33 UTC)" executed successfully 2020-03-31 09:22:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:14 UTC)" (scheduled at 2020-03-31 09:22:04.323352+00:00) 2020-03-31 09:22:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:22:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:14 UTC)" (scheduled at 2020-03-31 09:22:04.323352+00:00) 2020-03-31 09:22:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 4, 325019), True, datetime.datetime(2020, 3, 31, 9, 22, 4, 325019), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:22:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 4, 325019), True, datetime.datetime(2020, 3, 31, 9, 22, 4, 325019), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:22:04,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:14.323352+00:00 (in 9.999036 seconds) notificationworker stdout | 2020-03-31 09:22:04,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:14.323352+00:00 (in 9.999036 seconds) 2020-03-31 09:22:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:22:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:14 UTC)" executed successfully 2020-03-31 09:22:04,445 [526] [DEBUG] [app] Starting request: urn:request:123db4dc-28d1-42d2-b8e5-6ff59b21a109 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:04,445 [526] [DEBUG] [app] Starting request: urn:request:123db4dc-28d1-42d2-b8e5-6ff59b21a109 (/health/instance) 2020-03-31 09:22:04,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:04,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:04,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:04,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:04,453 [554] [DEBUG] [app] Starting request: urn:request:b17c71f1-2996-4aae-a95f-262ce8657192 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:04,453 [554] [DEBUG] [app] Starting request: urn:request:b17c71f1-2996-4aae-a95f-262ce8657192 (/v1/_internal_ping) 2020-03-31 09:22:04,454 [554] [DEBUG] [app] Ending request: urn:request:b17c71f1-2996-4aae-a95f-262ce8657192 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:04,454 [554] [DEBUG] [app] Ending request: urn:request:b17c71f1-2996-4aae-a95f-262ce8657192 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:04,454 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:04,454 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:04,454 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:04,455 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:04,454 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:04,455 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:22:04,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:22:04,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:04,458 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:04,458 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:04,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:04,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:04,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:04,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:04,489 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:04,489 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:04,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:04,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:04,494 [518] [DEBUG] [app] Starting request: urn:request:4a4528b2-77f3-4de3-b606-b6bc4fadb95e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:04,494 [518] [DEBUG] [app] Starting request: urn:request:4a4528b2-77f3-4de3-b606-b6bc4fadb95e (/_internal_ping) 2020-03-31 09:22:04,495 [518] [DEBUG] [app] Ending request: urn:request:4a4528b2-77f3-4de3-b606-b6bc4fadb95e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:04,495 [518] [DEBUG] [app] Ending request: urn:request:4a4528b2-77f3-4de3-b606-b6bc4fadb95e (/_internal_ping) 2020-03-31 09:22:04,496 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:04,496 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:22:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:04,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:22:04,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:04,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:04,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:22:04,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:22:04,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:22:04,510 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:04,510 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:04,511 [534] [DEBUG] [app] Starting request: urn:request:96e3f972-e7bc-40d0-a47c-e533580c3126 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:04,511 [534] [DEBUG] [app] Starting request: urn:request:96e3f972-e7bc-40d0-a47c-e533580c3126 (/c1/_internal_ping) 2020-03-31 09:22:04,512 [534] [DEBUG] [app] Ending request: urn:request:96e3f972-e7bc-40d0-a47c-e533580c3126 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:04,512 [534] [DEBUG] [app] Ending request: urn:request:96e3f972-e7bc-40d0-a47c-e533580c3126 (/c1/_internal_ping) 2020-03-31 09:22:04,513 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:04,513 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:22:04,513 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:22:04,513 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:04,515 [526] [DEBUG] [app] Ending request: urn:request:123db4dc-28d1-42d2-b8e5-6ff59b21a109 (/health/instance) 2020-03-31 09:22:04,515 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:04,515 [526] [DEBUG] [app] Ending request: urn:request:123db4dc-28d1-42d2-b8e5-6ff59b21a109 (/health/instance) 2020-03-31 09:22:04,515 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:04,516 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:22:04,516 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:22:06,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:06,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:16.924497+00:00 (in 9.996736 seconds) 2020-03-31 09:22:06,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:16 UTC)" (scheduled at 2020-03-31 09:22:06.924497+00:00) 2020-03-31 09:22:06,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:22:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:16.924497+00:00 (in 9.996736 seconds) 2020-03-31 09:22:06,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:16 UTC)" (scheduled at 2020-03-31 09:22:06.924497+00:00) 2020-03-31 09:22:06,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:06,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 6, 928552), True, datetime.datetime(2020, 3, 31, 9, 22, 6, 928552), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:22:06,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 6, 928552), True, datetime.datetime(2020, 3, 31, 9, 22, 6, 928552), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:22:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:22:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:16 UTC)" executed successfully 2020-03-31 09:22:08,272 [527] [DEBUG] [app] Starting request: urn:request:a04b338d-1810-4928-acec-2efe469c37a2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:08,272 [527] [DEBUG] [app] Starting request: urn:request:a04b338d-1810-4928-acec-2efe469c37a2 (/health/instance) 2020-03-31 09:22:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:08,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:08,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:08,280 [556] [DEBUG] [app] Starting request: urn:request:1faa1aaa-8fa6-4704-9ad0-6eeb7e3dbf86 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:08,280 [556] [DEBUG] [app] Starting request: urn:request:1faa1aaa-8fa6-4704-9ad0-6eeb7e3dbf86 (/v1/_internal_ping) 2020-03-31 09:22:08,281 [556] [DEBUG] [app] Ending request: urn:request:1faa1aaa-8fa6-4704-9ad0-6eeb7e3dbf86 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:08,281 [556] [DEBUG] [app] Ending request: urn:request:1faa1aaa-8fa6-4704-9ad0-6eeb7e3dbf86 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:08,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:08,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:08,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:08,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:22:08,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:22:08,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:08,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:08,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:08,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:08,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:08,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:08,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:08,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:08,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:08,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:08,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:08,311 [527] [DEBUG] [app] Starting request: urn:request:6bdd3cac-fb81-4dc3-bbf3-7bf4d545307b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:08,311 [527] [DEBUG] [app] Starting request: urn:request:6bdd3cac-fb81-4dc3-bbf3-7bf4d545307b (/_internal_ping) 2020-03-31 09:22:08,312 [527] [DEBUG] [app] Ending request: urn:request:6bdd3cac-fb81-4dc3-bbf3-7bf4d545307b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:08,312 [527] [DEBUG] [app] Ending request: urn:request:6bdd3cac-fb81-4dc3-bbf3-7bf4d545307b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:22:08,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:08,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:08,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:08,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:08,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:08,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:08,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:22:08,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:08,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:22:08,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:22:08,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:08,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:08,326 [534] [DEBUG] [app] Starting request: urn:request:7cf59d12-5f3c-4049-bbbf-c7ee6c482d1d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:08,326 [534] [DEBUG] [app] Starting request: urn:request:7cf59d12-5f3c-4049-bbbf-c7ee6c482d1d (/c1/_internal_ping) 2020-03-31 09:22:08,327 [534] [DEBUG] [app] Ending request: urn:request:7cf59d12-5f3c-4049-bbbf-c7ee6c482d1d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:08,327 [534] [DEBUG] [app] Ending request: urn:request:7cf59d12-5f3c-4049-bbbf-c7ee6c482d1d (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:08,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:08,328 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:22:08,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:22:08,328 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:08,329 [527] [DEBUG] [app] Ending request: urn:request:a04b338d-1810-4928-acec-2efe469c37a2 (/health/instance) 2020-03-31 09:22:08,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:08,329 [527] [DEBUG] [app] Ending request: urn:request:a04b338d-1810-4928-acec-2efe469c37a2 (/health/instance) 2020-03-31 09:22:08,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:08,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:22:08,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:22:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:22:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:21.440707+00:00 (in 9.998936 seconds) 2020-03-31 09:22:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:21 UTC)" (scheduled at 2020-03-31 09:22:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:22:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:21.440707+00:00 (in 9.998936 seconds) 2020-03-31 09:22:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:21 UTC)" (scheduled at 2020-03-31 09:22:11.440707+00:00) 2020-03-31 09:22:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:22:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:22:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:22:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:22:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:22:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:21 UTC)" executed successfully 2020-03-31 09:22:14,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:14,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:14,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:24 UTC)" (scheduled at 2020-03-31 09:22:14.323352+00:00) 2020-03-31 09:22:14,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:24.323352+00:00 (in 9.996232 seconds) notificationworker stdout | 2020-03-31 09:22:14,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:24 UTC)" (scheduled at 2020-03-31 09:22:14.323352+00:00) 2020-03-31 09:22:14,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:24.323352+00:00 (in 9.996232 seconds) 2020-03-31 09:22:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 14, 328121), True, datetime.datetime(2020, 3, 31, 9, 22, 14, 328121), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:22:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 14, 328121), True, datetime.datetime(2020, 3, 31, 9, 22, 14, 328121), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:22:14,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:22:14,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:14,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:14,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:14,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:14,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:24 UTC)" executed successfully 2020-03-31 09:22:14,445 [525] [DEBUG] [app] Starting request: urn:request:2ab420a2-eaf5-4a74-b2c7-847fb21b4f28 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:14,445 [525] [DEBUG] [app] Starting request: urn:request:2ab420a2-eaf5-4a74-b2c7-847fb21b4f28 (/health/instance) 2020-03-31 09:22:14,447 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:14,447 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:14,451 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:14,451 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:14,453 [544] [DEBUG] [app] Starting request: urn:request:8ccadd8b-6d9b-4a9c-81f6-2ca711e5dd87 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:14,453 [544] [DEBUG] [app] Starting request: urn:request:8ccadd8b-6d9b-4a9c-81f6-2ca711e5dd87 (/v1/_internal_ping) 2020-03-31 09:22:14,454 [544] [DEBUG] [app] Ending request: urn:request:8ccadd8b-6d9b-4a9c-81f6-2ca711e5dd87 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:14,454 [544] [DEBUG] [app] Ending request: urn:request:8ccadd8b-6d9b-4a9c-81f6-2ca711e5dd87 (/v1/_internal_ping) 2020-03-31 09:22:14,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:14,455 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:22:14,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:14,456 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:14,455 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:22:14,456 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:22:14,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:22:14,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:14,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:14,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:14,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:14,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:14,465 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:14,465 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:14,479 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:14,479 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:14,482 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:14,482 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:14,484 [518] [DEBUG] [app] Starting request: urn:request:01e10e8b-50eb-4ced-adb1-33de58b706a8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:14,484 [518] [DEBUG] [app] Starting request: urn:request:01e10e8b-50eb-4ced-adb1-33de58b706a8 (/_internal_ping) 2020-03-31 09:22:14,484 [518] [DEBUG] [app] Ending request: urn:request:01e10e8b-50eb-4ced-adb1-33de58b706a8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:14,484 [518] [DEBUG] [app] Ending request: urn:request:01e10e8b-50eb-4ced-adb1-33de58b706a8 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:22:14,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:14,485 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:14,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:14,485 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:14,489 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:14,489 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:14,490 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:22:14,490 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:14,493 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:14,493 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:14,494 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:22:14,494 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:22:14,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:14,498 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:14,500 [541] [DEBUG] [app] Starting request: urn:request:d43fa49b-3037-494f-a162-a803c1eb2ed1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:14,500 [541] [DEBUG] [app] Starting request: urn:request:d43fa49b-3037-494f-a162-a803c1eb2ed1 (/c1/_internal_ping) 2020-03-31 09:22:14,500 [541] [DEBUG] [app] Ending request: urn:request:d43fa49b-3037-494f-a162-a803c1eb2ed1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:14,500 [541] [DEBUG] [app] Ending request: urn:request:d43fa49b-3037-494f-a162-a803c1eb2ed1 (/c1/_internal_ping) 2020-03-31 09:22:14,501 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:14,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:22:14,501 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:14,503 [525] [DEBUG] [app] Ending request: urn:request:2ab420a2-eaf5-4a74-b2c7-847fb21b4f28 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:22:14,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:14,503 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:14,504 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:22:14,503 [525] [DEBUG] [app] Ending request: urn:request:2ab420a2-eaf5-4a74-b2c7-847fb21b4f28 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:14,503 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:14,504 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:22:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:22:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:45.064514+00:00 (in 29.999054 seconds) 2020-03-31 09:22:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:45 UTC)" (scheduled at 2020-03-31 09:22:15.064514+00:00) gcworker stdout | 2020-03-31 09:22:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:45.064514+00:00 (in 29.999054 seconds) 2020-03-31 09:22:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:45 UTC)" (scheduled at 2020-03-31 09:22:15.064514+00:00) 2020-03-31 09:22:15,067 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2020-03-31 09:22:15,067 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) 2020-03-31 09:22:15,082 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436935, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:22:15,082 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436935, 1209600, 500, 1, 0]) 2020-03-31 09:22:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:22:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:22:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:22:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:45 UTC)" executed successfully 2020-03-31 09:22:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:26.924497+00:00 (in 9.999233 seconds) 2020-03-31 09:22:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:26 UTC)" (scheduled at 2020-03-31 09:22:16.924497+00:00) 2020-03-31 09:22:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:22:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:26.924497+00:00 (in 9.999233 seconds) 2020-03-31 09:22:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:26 UTC)" (scheduled at 2020-03-31 09:22:16.924497+00:00) 2020-03-31 09:22:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 16, 925964), True, datetime.datetime(2020, 3, 31, 9, 22, 16, 925964), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:22:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 16, 925964), True, datetime.datetime(2020, 3, 31, 9, 22, 16, 925964), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:22:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:22:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:16,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:16,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:26 UTC)" executed successfully 2020-03-31 09:22:18,273 [518] [DEBUG] [app] Starting request: urn:request:1100129f-76d1-4ca8-ba1a-df12d5ad64b4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:18,273 [518] [DEBUG] [app] Starting request: urn:request:1100129f-76d1-4ca8-ba1a-df12d5ad64b4 (/health/instance) 2020-03-31 09:22:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:18,280 [552] [DEBUG] [app] Starting request: urn:request:2c392036-21d4-4c60-a407-e84f28806df9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:18,280 [552] [DEBUG] [app] Starting request: urn:request:2c392036-21d4-4c60-a407-e84f28806df9 (/v1/_internal_ping) 2020-03-31 09:22:18,281 [552] [DEBUG] [app] Ending request: urn:request:2c392036-21d4-4c60-a407-e84f28806df9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:18,281 [552] [DEBUG] [app] Ending request: urn:request:2c392036-21d4-4c60-a407-e84f28806df9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:18,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:18,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:22:18,284 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:22:18,284 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:18,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:18,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:18,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:18,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:18,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:18,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:18,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:18,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:18,314 [518] [DEBUG] [app] Starting request: urn:request:0db37522-95cb-49dd-9530-b755eb98b7b5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:18,314 [518] [DEBUG] [app] Starting request: urn:request:0db37522-95cb-49dd-9530-b755eb98b7b5 (/_internal_ping) 2020-03-31 09:22:18,315 [518] [DEBUG] [app] Ending request: urn:request:0db37522-95cb-49dd-9530-b755eb98b7b5 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:18,315 [518] [DEBUG] [app] Ending request: urn:request:0db37522-95cb-49dd-9530-b755eb98b7b5 (/_internal_ping) 2020-03-31 09:22:18,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:18,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:22:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:22:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:22:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:22:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:18,333 [532] [DEBUG] [app] Starting request: urn:request:ef116b6e-c367-4e3f-8dae-359a2d1afa3f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:18,333 [532] [DEBUG] [app] Starting request: urn:request:ef116b6e-c367-4e3f-8dae-359a2d1afa3f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:18,334 [532] [DEBUG] [app] Ending request: urn:request:ef116b6e-c367-4e3f-8dae-359a2d1afa3f (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:18,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:18,335 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:22:18,334 [532] [DEBUG] [app] Ending request: urn:request:ef116b6e-c367-4e3f-8dae-359a2d1afa3f (/c1/_internal_ping) 2020-03-31 09:22:18,336 [518] [DEBUG] [app] Ending request: urn:request:1100129f-76d1-4ca8-ba1a-df12d5ad64b4 (/health/instance) 2020-03-31 09:22:18,337 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:18,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:18,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:22:18,335 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:22:18,336 [518] [DEBUG] [app] Ending request: urn:request:1100129f-76d1-4ca8-ba1a-df12d5ad64b4 (/health/instance) 2020-03-31 09:22:18,337 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:18,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:22:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:22:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:31.440707+00:00 (in 9.999106 seconds) repositoryactioncounter stdout | 2020-03-31 09:22:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:31.440707+00:00 (in 9.999106 seconds) 2020-03-31 09:22:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:31 UTC)" (scheduled at 2020-03-31 09:22:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:22:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:31 UTC)" (scheduled at 2020-03-31 09:22:21.440707+00:00) 2020-03-31 09:22:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:22:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:22:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:22:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:22:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:22:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:31 UTC)" executed successfully 2020-03-31 09:22:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:22:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:52.113221+00:00 (in 29.999063 seconds) 2020-03-31 09:22:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:52 UTC)" (scheduled at 2020-03-31 09:22:22.113221+00:00) securityworker stdout | 2020-03-31 09:22:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:52.113221+00:00 (in 29.999063 seconds) 2020-03-31 09:22:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:52 UTC)" (scheduled at 2020-03-31 09:22:22.113221+00:00) 2020-03-31 09:22:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:22:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:22:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:22:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:22:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:22:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:22:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:22:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:22:52 UTC)" executed successfully 2020-03-31 09:22:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:31.498397+00:00 (in 7.170491 seconds) 2020-03-31 09:22:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:34 UTC)" (scheduled at 2020-03-31 09:22:24.323352+00:00) 2020-03-31 09:22:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:22:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:31.498397+00:00 (in 7.170491 seconds) 2020-03-31 09:22:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:34 UTC)" (scheduled at 2020-03-31 09:22:24.323352+00:00) 2020-03-31 09:22:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 24, 328655), True, datetime.datetime(2020, 3, 31, 9, 22, 24, 328655), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:22:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 24, 328655), True, datetime.datetime(2020, 3, 31, 9, 22, 24, 328655), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:22:24,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:22:24,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:24,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:24,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:34 UTC)" executed successfully 2020-03-31 09:22:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" (scheduled at 2020-03-31 09:22:24.410927+00:00) 2020-03-31 09:22:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:22:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" executed successfully 2020-03-31 09:22:24,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:33.330112+00:00 (in 8.917680 seconds) security_notification_worker stdout | 2020-03-31 09:22:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" (scheduled at 2020-03-31 09:22:24.410927+00:00) 2020-03-31 09:22:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:22:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:22:24 UTC)" executed successfully 2020-03-31 09:22:24,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:33.330112+00:00 (in 8.917680 seconds) 2020-03-31 09:22:24,446 [527] [DEBUG] [app] Starting request: urn:request:f453d0e1-1925-415f-8847-088cc8607cfb (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:24,446 [527] [DEBUG] [app] Starting request: urn:request:f453d0e1-1925-415f-8847-088cc8607cfb (/health/instance) 2020-03-31 09:22:24,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:24,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:24,454 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:24,454 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:24,456 [552] [DEBUG] [app] Starting request: urn:request:8c7885fa-d256-4d27-8e45-b040075caa01 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:24,456 [552] [DEBUG] [app] Starting request: urn:request:8c7885fa-d256-4d27-8e45-b040075caa01 (/v1/_internal_ping) 2020-03-31 09:22:24,457 [552] [DEBUG] [app] Ending request: urn:request:8c7885fa-d256-4d27-8e45-b040075caa01 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:24,457 [552] [DEBUG] [app] Ending request: urn:request:8c7885fa-d256-4d27-8e45-b040075caa01 (/v1/_internal_ping) 2020-03-31 09:22:24,458 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:22:24,458 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:24,459 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:22:24,459 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:22:24,460 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:22:24,460 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:24,463 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:22:24,463 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:24,465 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:24,465 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:24,465 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:24,465 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:24,478 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:24,478 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:24,495 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:24,495 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:24,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:24,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:24,502 [527] [DEBUG] [app] Starting request: urn:request:a5033ce0-0f62-4196-8c17-cd4dbde60206 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:24,502 [527] [DEBUG] [app] Starting request: urn:request:a5033ce0-0f62-4196-8c17-cd4dbde60206 (/_internal_ping) 2020-03-31 09:22:24,503 [527] [DEBUG] [app] Ending request: urn:request:a5033ce0-0f62-4196-8c17-cd4dbde60206 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:24,503 [527] [DEBUG] [app] Ending request: urn:request:a5033ce0-0f62-4196-8c17-cd4dbde60206 (/_internal_ping) 2020-03-31 09:22:24,504 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:24,504 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:24,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:24,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:24,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:24,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:24,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:22:24,510 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:24,514 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:24,514 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:24,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:22:24,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:22:24,519 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:24,519 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:24,521 [534] [DEBUG] [app] Starting request: urn:request:f2f6df86-daad-43d2-bc56-320de194aa76 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:24,521 [534] [DEBUG] [app] Starting request: urn:request:f2f6df86-daad-43d2-bc56-320de194aa76 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:24,522 [534] [DEBUG] [app] Ending request: urn:request:f2f6df86-daad-43d2-bc56-320de194aa76 (/c1/_internal_ping) 2020-03-31 09:22:24,522 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:24,523 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:22:24,522 [534] [DEBUG] [app] Ending request: urn:request:f2f6df86-daad-43d2-bc56-320de194aa76 (/c1/_internal_ping) 2020-03-31 09:22:24,522 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:22:24,523 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:24,525 [527] [DEBUG] [app] Ending request: urn:request:f453d0e1-1925-415f-8847-088cc8607cfb (/health/instance) 2020-03-31 09:22:24,525 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:24,525 [527] [DEBUG] [app] Ending request: urn:request:f453d0e1-1925-415f-8847-088cc8607cfb (/health/instance) 2020-03-31 09:22:24,525 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:24,526 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) gunicorn-web stdout | 2020-03-31 09:22:24,526 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) 2020-03-31 09:22:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:26 UTC)" (scheduled at 2020-03-31 09:22:26.924497+00:00) 2020-03-31 09:22:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:22:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:26 UTC)" (scheduled at 2020-03-31 09:22:26.924497+00:00) 2020-03-31 09:22:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:36.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:22:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 26, 926075), True, datetime.datetime(2020, 3, 31, 9, 22, 26, 926075), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:22:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:36.924497+00:00 (in 9.999199 seconds) storagereplication stdout | 2020-03-31 09:22:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 26, 926075), True, datetime.datetime(2020, 3, 31, 9, 22, 26, 926075), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:22:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:22:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:36 UTC)" executed successfully 2020-03-31 09:22:28,272 [518] [DEBUG] [app] Starting request: urn:request:ede49f1f-847c-4fb3-b880-14e094871053 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:28,272 [518] [DEBUG] [app] Starting request: urn:request:ede49f1f-847c-4fb3-b880-14e094871053 (/health/instance) 2020-03-31 09:22:28,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:28,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:28,280 [547] [DEBUG] [app] Starting request: urn:request:523ebbf5-ad7f-404b-8566-9d68d868759b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:28,280 [547] [DEBUG] [app] Starting request: urn:request:523ebbf5-ad7f-404b-8566-9d68d868759b (/v1/_internal_ping) 2020-03-31 09:22:28,281 [547] [DEBUG] [app] Ending request: urn:request:523ebbf5-ad7f-404b-8566-9d68d868759b (/v1/_internal_ping) 2020-03-31 09:22:28,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:22:28,281 [547] [DEBUG] [app] Ending request: urn:request:523ebbf5-ad7f-404b-8566-9d68d868759b (/v1/_internal_ping) 2020-03-31 09:22:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:28,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:28,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:22:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:22:28,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:28,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:28,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:28,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:28,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:28,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:28,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:28,318 [518] [DEBUG] [app] Starting request: urn:request:a7d3a17d-c282-4f57-a9e4-304ebeee77b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:28,318 [518] [DEBUG] [app] Starting request: urn:request:a7d3a17d-c282-4f57-a9e4-304ebeee77b2 (/_internal_ping) 2020-03-31 09:22:28,319 [518] [DEBUG] [app] Ending request: urn:request:a7d3a17d-c282-4f57-a9e4-304ebeee77b2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:28,319 [518] [DEBUG] [app] Ending request: urn:request:a7d3a17d-c282-4f57-a9e4-304ebeee77b2 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:22:28,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:28,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:22:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:22:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:22:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:22:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:28,333 [534] [DEBUG] [app] Starting request: urn:request:76915428-d04a-431e-93b5-3d301d526aff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:28,333 [534] [DEBUG] [app] Starting request: urn:request:76915428-d04a-431e-93b5-3d301d526aff (/c1/_internal_ping) 2020-03-31 09:22:28,334 [534] [DEBUG] [app] Ending request: urn:request:76915428-d04a-431e-93b5-3d301d526aff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:28,334 [534] [DEBUG] [app] Ending request: urn:request:76915428-d04a-431e-93b5-3d301d526aff (/c1/_internal_ping) 2020-03-31 09:22:28,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:22:28,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:28,336 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:22:28,336 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:28,337 [518] [DEBUG] [app] Ending request: urn:request:ede49f1f-847c-4fb3-b880-14e094871053 (/health/instance) 2020-03-31 09:22:28,338 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:28,337 [518] [DEBUG] [app] Ending request: urn:request:ede49f1f-847c-4fb3-b880-14e094871053 (/health/instance) 2020-03-31 09:22:28,338 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:28,338 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:22:28,338 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:22:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:22:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:00.337699+00:00 (in 29.996791 seconds) 2020-03-31 09:22:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:00 UTC)" (scheduled at 2020-03-31 09:22:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:22:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:00.337699+00:00 (in 29.996791 seconds) 2020-03-31 09:22:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:00 UTC)" (scheduled at 2020-03-31 09:22:30.337699+00:00) 2020-03-31 09:22:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 22, 30, 341886), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:22:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 22, 30, 341886), False, 50, 1, 0]) 2020-03-31 09:22:30,522 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:22:30,522 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:22:30,522 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:30,522 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:22:30,522 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:30,522 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:00 UTC)" executed successfully 2020-03-31 09:22:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:22:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:41.440707+00:00 (in 9.999198 seconds) 2020-03-31 09:22:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:41 UTC)" (scheduled at 2020-03-31 09:22:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:22:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:41.440707+00:00 (in 9.999198 seconds) 2020-03-31 09:22:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:41 UTC)" (scheduled at 2020-03-31 09:22:31.440707+00:00) 2020-03-31 09:22:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:22:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:22:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:22:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:22:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:22:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:41 UTC)" executed successfully 2020-03-31 09:22:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:34.323352+00:00 (in 2.824031 seconds) 2020-03-31 09:22:31,500 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:27:31 UTC)" (scheduled at 2020-03-31 09:22:31.498397+00:00) notificationworker stdout | 2020-03-31 09:22:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:34.323352+00:00 (in 2.824031 seconds) 2020-03-31 09:22:31,500 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:27:31 UTC)" (scheduled at 2020-03-31 09:22:31.498397+00:00) 2020-03-31 09:22:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), u'notification/%']) notificationworker stdout | 2020-03-31 09:22:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), u'notification/%']) 2020-03-31 09:22:31,515 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), True, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:22:31,515 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), True, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), 0, u'notification/%']) 2020-03-31 09:22:31,520 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), True, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), u'notification/%']) notificationworker stdout | 2020-03-31 09:22:31,520 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), True, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 22, 31, 500354), u'notification/%']) 2020-03-31 09:22:31,523 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:22:31,523 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:31,526 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:27:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:31,526 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:27:31 UTC)" executed successfully 2020-03-31 09:22:31,528 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:22:31,528 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:31,529 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:22:31,529 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:31,530 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 09:22:31,530 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:22:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:22:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:03 UTC)" (scheduled at 2020-03-31 09:22:33.330112+00:00) 2020-03-31 09:22:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:03.330112+00:00 (in 29.998295 seconds) 2020-03-31 09:22:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:22:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:03 UTC)" (scheduled at 2020-03-31 09:22:33.330112+00:00) 2020-03-31 09:22:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:03.330112+00:00 (in 29.998295 seconds) 2020-03-31 09:22:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 33, 332881), True, datetime.datetime(2020, 3, 31, 9, 22, 33, 332881), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:22:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 33, 332881), True, datetime.datetime(2020, 3, 31, 9, 22, 33, 332881), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:22:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:22:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:22:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:03 UTC)" executed successfully 2020-03-31 09:22:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:22:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:43.539999+00:00 (in 10.172277 seconds) 2020-03-31 09:22:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:33 UTC)" (scheduled at 2020-03-31 09:22:33.366791+00:00) 2020-03-31 09:22:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:22:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:43.539999+00:00 (in 10.172277 seconds) 2020-03-31 09:22:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:33 UTC)" (scheduled at 2020-03-31 09:22:33.366791+00:00) 2020-03-31 09:22:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 33, 368575), True, datetime.datetime(2020, 3, 31, 9, 22, 33, 368575), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:22:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 33, 368575), True, datetime.datetime(2020, 3, 31, 9, 22, 33, 368575), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:22:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:22:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:22:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:33 UTC)" executed successfully 2020-03-31 09:22:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:44.323352+00:00 (in 9.998915 seconds) 2020-03-31 09:22:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:44 UTC)" (scheduled at 2020-03-31 09:22:34.323352+00:00) 2020-03-31 09:22:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:22:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:44.323352+00:00 (in 9.998915 seconds) 2020-03-31 09:22:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:44 UTC)" (scheduled at 2020-03-31 09:22:34.323352+00:00) 2020-03-31 09:22:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 34, 325484), True, datetime.datetime(2020, 3, 31, 9, 22, 34, 325484), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:22:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 34, 325484), True, datetime.datetime(2020, 3, 31, 9, 22, 34, 325484), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:22:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:22:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:44 UTC)" executed successfully 2020-03-31 09:22:34,445 [527] [DEBUG] [app] Starting request: urn:request:34b521ed-03f8-453c-99e3-f541b378cc9d (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:34,445 [527] [DEBUG] [app] Starting request: urn:request:34b521ed-03f8-453c-99e3-f541b378cc9d (/health/instance) 2020-03-31 09:22:34,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:34,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:34,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:34,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:34,452 [548] [DEBUG] [app] Starting request: urn:request:0a2683a7-f8f9-4402-adab-6078e735d0e6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:34,452 [548] [DEBUG] [app] Starting request: urn:request:0a2683a7-f8f9-4402-adab-6078e735d0e6 (/v1/_internal_ping) 2020-03-31 09:22:34,453 [548] [DEBUG] [app] Ending request: urn:request:0a2683a7-f8f9-4402-adab-6078e735d0e6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:34,453 [548] [DEBUG] [app] Ending request: urn:request:0a2683a7-f8f9-4402-adab-6078e735d0e6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:34,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:34,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:34,454 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:34,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:34,454 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:34,456 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:22:34,455 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:22:34,456 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:34,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:34,458 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:34,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:34,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:34,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:34,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:34,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:34,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:34,485 [518] [DEBUG] [app] Starting request: urn:request:bc344054-915e-4dd2-9de0-101aa2dd90dd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:34,485 [518] [DEBUG] [app] Starting request: urn:request:bc344054-915e-4dd2-9de0-101aa2dd90dd (/_internal_ping) 2020-03-31 09:22:34,487 [518] [DEBUG] [app] Ending request: urn:request:bc344054-915e-4dd2-9de0-101aa2dd90dd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:34,487 [518] [DEBUG] [app] Ending request: urn:request:bc344054-915e-4dd2-9de0-101aa2dd90dd (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:22:34,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:34,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:34,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:22:34,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:22:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:22:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:22:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:34,500 [532] [DEBUG] [app] Starting request: urn:request:1acaebda-14c8-4901-bee7-71949cff8959 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:34,500 [532] [DEBUG] [app] Starting request: urn:request:1acaebda-14c8-4901-bee7-71949cff8959 (/c1/_internal_ping) 2020-03-31 09:22:34,500 [532] [DEBUG] [app] Ending request: urn:request:1acaebda-14c8-4901-bee7-71949cff8959 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:34,500 [532] [DEBUG] [app] Ending request: urn:request:1acaebda-14c8-4901-bee7-71949cff8959 (/c1/_internal_ping) 2020-03-31 09:22:34,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:22:34,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:34,502 [527] [DEBUG] [app] Ending request: urn:request:34b521ed-03f8-453c-99e3-f541b378cc9d (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:34,502 [527] [DEBUG] [app] Ending request: urn:request:34b521ed-03f8-453c-99e3-f541b378cc9d (/health/instance) 2020-03-31 09:22:34,502 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:34,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:22:34,502 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:34,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:22:34,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-verbs stdout | 2020-03-31 09:22:34,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:36,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:36,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:36,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:38.010934+00:00 (in 1.082697 seconds) 2020-03-31 09:22:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:46 UTC)" (scheduled at 2020-03-31 09:22:36.924497+00:00) 2020-03-31 09:22:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:22:36,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:38.010934+00:00 (in 1.082697 seconds) 2020-03-31 09:22:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:46 UTC)" (scheduled at 2020-03-31 09:22:36.924497+00:00) 2020-03-31 09:22:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:36,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 36, 928944), True, datetime.datetime(2020, 3, 31, 9, 22, 36, 928944), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:22:36,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 36, 928944), True, datetime.datetime(2020, 3, 31, 9, 22, 36, 928944), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:22:36,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:22:36,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:36,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:36,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:36,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:36,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:46 UTC)" executed successfully 2020-03-31 09:22:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:46.924497+00:00 (in 8.912863 seconds) 2020-03-31 09:22:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:38 UTC)" (scheduled at 2020-03-31 09:22:38.010934+00:00) 2020-03-31 09:22:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:46.924497+00:00 (in 8.912863 seconds) 2020-03-31 09:22:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:38 UTC)" (scheduled at 2020-03-31 09:22:38.010934+00:00) 2020-03-31 09:22:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:38 UTC)" executed successfully 2020-03-31 09:22:38,273 [526] [DEBUG] [app] Starting request: urn:request:4e2ddc1a-f0b1-42fb-8f8b-8984e6d0f845 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:38,273 [526] [DEBUG] [app] Starting request: urn:request:4e2ddc1a-f0b1-42fb-8f8b-8984e6d0f845 (/health/instance) 2020-03-31 09:22:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:38,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:38,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:38,281 [554] [DEBUG] [app] Starting request: urn:request:5c1fbed6-83d2-488c-b83c-8b82f37e627a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:38,281 [554] [DEBUG] [app] Starting request: urn:request:5c1fbed6-83d2-488c-b83c-8b82f37e627a (/v1/_internal_ping) 2020-03-31 09:22:38,282 [554] [DEBUG] [app] Ending request: urn:request:5c1fbed6-83d2-488c-b83c-8b82f37e627a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:38,282 [554] [DEBUG] [app] Ending request: urn:request:5c1fbed6-83d2-488c-b83c-8b82f37e627a (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:38,282 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:38,282 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:22:38,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:22:38,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:38,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:38,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:38,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:38,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:38,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:38,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:38,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:38,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:38,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:38,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:38,316 [526] [DEBUG] [app] Starting request: urn:request:e426e43e-d06a-46a0-9a2e-be59d7534ed7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:38,316 [526] [DEBUG] [app] Starting request: urn:request:e426e43e-d06a-46a0-9a2e-be59d7534ed7 (/_internal_ping) 2020-03-31 09:22:38,316 [526] [DEBUG] [app] Ending request: urn:request:e426e43e-d06a-46a0-9a2e-be59d7534ed7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:38,316 [526] [DEBUG] [app] Ending request: urn:request:e426e43e-d06a-46a0-9a2e-be59d7534ed7 (/_internal_ping) 2020-03-31 09:22:38,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:38,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:38,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:22:38,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:38,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:38,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:22:38,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:22:38,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:22:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:38,330 [535] [DEBUG] [app] Starting request: urn:request:a4ccf5f1-320d-49ed-a972-5ad76c47f622 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:38,330 [535] [DEBUG] [app] Starting request: urn:request:a4ccf5f1-320d-49ed-a972-5ad76c47f622 (/c1/_internal_ping) 2020-03-31 09:22:38,331 [535] [DEBUG] [app] Ending request: urn:request:a4ccf5f1-320d-49ed-a972-5ad76c47f622 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:38,331 [535] [DEBUG] [app] Ending request: urn:request:a4ccf5f1-320d-49ed-a972-5ad76c47f622 (/c1/_internal_ping) 2020-03-31 09:22:38,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:38,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:22:38,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:22:38,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:38,333 [526] [DEBUG] [app] Ending request: urn:request:4e2ddc1a-f0b1-42fb-8f8b-8984e6d0f845 (/health/instance) 2020-03-31 09:22:38,333 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:38,333 [526] [DEBUG] [app] Ending request: urn:request:4e2ddc1a-f0b1-42fb-8f8b-8984e6d0f845 (/health/instance) 2020-03-31 09:22:38,333 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:38,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:22:38,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:22:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:22:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:51.440707+00:00 (in 9.999070 seconds) 2020-03-31 09:22:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:51 UTC)" (scheduled at 2020-03-31 09:22:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:22:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:51.440707+00:00 (in 9.999070 seconds) 2020-03-31 09:22:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:51 UTC)" (scheduled at 2020-03-31 09:22:41.440707+00:00) 2020-03-31 09:22:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:22:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:22:41,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:22:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:22:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:22:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:51 UTC)" executed successfully 2020-03-31 09:22:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:22:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:33.366791+00:00 (in 49.825925 seconds) 2020-03-31 09:22:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:43 UTC)" (scheduled at 2020-03-31 09:22:43.539999+00:00) 2020-03-31 09:22:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:22:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:33.366791+00:00 (in 49.825925 seconds) 2020-03-31 09:22:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:43 UTC)" (scheduled at 2020-03-31 09:22:43.539999+00:00) 2020-03-31 09:22:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:43 UTC)" executed successfully 2020-03-31 09:22:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:49.498966+00:00 (in 5.174540 seconds) 2020-03-31 09:22:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:54 UTC)" (scheduled at 2020-03-31 09:22:44.323352+00:00) 2020-03-31 09:22:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:22:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:49.498966+00:00 (in 5.174540 seconds) 2020-03-31 09:22:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:54 UTC)" (scheduled at 2020-03-31 09:22:44.323352+00:00) 2020-03-31 09:22:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 44, 325387), True, datetime.datetime(2020, 3, 31, 9, 22, 44, 325387), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:22:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 44, 325387), True, datetime.datetime(2020, 3, 31, 9, 22, 44, 325387), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:22:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:22:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:54 UTC)" executed successfully 2020-03-31 09:22:44,446 [527] [DEBUG] [app] Starting request: urn:request:06c62ee5-a61e-4789-bc2a-d7130c0aed39 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:44,446 [527] [DEBUG] [app] Starting request: urn:request:06c62ee5-a61e-4789-bc2a-d7130c0aed39 (/health/instance) 2020-03-31 09:22:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:44,453 [552] [DEBUG] [app] Starting request: urn:request:1ffa4d97-053a-48f4-b577-8a63f44bbe0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:44,453 [552] [DEBUG] [app] Starting request: urn:request:1ffa4d97-053a-48f4-b577-8a63f44bbe0e (/v1/_internal_ping) 2020-03-31 09:22:44,454 [552] [DEBUG] [app] Ending request: urn:request:1ffa4d97-053a-48f4-b577-8a63f44bbe0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:44,454 [552] [DEBUG] [app] Ending request: urn:request:1ffa4d97-053a-48f4-b577-8a63f44bbe0e (/v1/_internal_ping) 2020-03-31 09:22:44,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:22:44,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:44,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:22:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:22:44,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:44,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:44,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:44,459 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:44,459 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:44,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:44,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:44,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:44,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:44,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:44,488 [518] [DEBUG] [app] Starting request: urn:request:97c58719-f7e9-48a7-93c8-07d5fc0edd5d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:44,488 [518] [DEBUG] [app] Starting request: urn:request:97c58719-f7e9-48a7-93c8-07d5fc0edd5d (/_internal_ping) 2020-03-31 09:22:44,489 [518] [DEBUG] [app] Ending request: urn:request:97c58719-f7e9-48a7-93c8-07d5fc0edd5d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:44,489 [518] [DEBUG] [app] Ending request: urn:request:97c58719-f7e9-48a7-93c8-07d5fc0edd5d (/_internal_ping) 2020-03-31 09:22:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:44,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:22:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:22:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:22:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:22:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:22:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:22:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:44,502 [532] [DEBUG] [app] Starting request: urn:request:1debb5c5-e5d9-45ba-847a-61160f894e23 (/c1/_internal_ping) 2020-03-31 09:22:44,503 [532] [DEBUG] [app] Ending request: urn:request:1debb5c5-e5d9-45ba-847a-61160f894e23 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:22:44,502 [532] [DEBUG] [app] Starting request: urn:request:1debb5c5-e5d9-45ba-847a-61160f894e23 (/c1/_internal_ping) 2020-03-31 09:22:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:44,503 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:22:44,503 [532] [DEBUG] [app] Ending request: urn:request:1debb5c5-e5d9-45ba-847a-61160f894e23 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:44,505 [527] [DEBUG] [app] Ending request: urn:request:06c62ee5-a61e-4789-bc2a-d7130c0aed39 (/health/instance) 2020-03-31 09:22:44,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:44,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-verbs stdout | 2020-03-31 09:22:44,503 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:22:44,505 [527] [DEBUG] [app] Ending request: urn:request:06c62ee5-a61e-4789-bc2a-d7130c0aed39 (/health/instance) 2020-03-31 09:22:44,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:44,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:22:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:22:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:15.064514+00:00 (in 29.999288 seconds) 2020-03-31 09:22:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:15 UTC)" (scheduled at 2020-03-31 09:22:45.064514+00:00) gcworker stdout | 2020-03-31 09:22:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:15.064514+00:00 (in 29.999288 seconds) 2020-03-31 09:22:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:15 UTC)" (scheduled at 2020-03-31 09:22:45.064514+00:00) 2020-03-31 09:22:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436965, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:22:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436965, 1209600, 500, 1, 0]) 2020-03-31 09:22:45,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:22:45,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:22:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:45,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:22:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:45,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:15 UTC)" executed successfully 2020-03-31 09:22:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:56.924497+00:00 (in 9.999185 seconds) 2020-03-31 09:22:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:56 UTC)" (scheduled at 2020-03-31 09:22:46.924497+00:00) 2020-03-31 09:22:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:22:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:56.924497+00:00 (in 9.999185 seconds) 2020-03-31 09:22:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:56 UTC)" (scheduled at 2020-03-31 09:22:46.924497+00:00) 2020-03-31 09:22:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 46, 926041), True, datetime.datetime(2020, 3, 31, 9, 22, 46, 926041), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:22:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 46, 926041), True, datetime.datetime(2020, 3, 31, 9, 22, 46, 926041), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:22:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:22:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:22:56 UTC)" executed successfully 2020-03-31 09:22:48,273 [527] [DEBUG] [app] Starting request: urn:request:d53dac1f-a4e3-46c3-91bc-326040c05ecf (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:48,273 [527] [DEBUG] [app] Starting request: urn:request:d53dac1f-a4e3-46c3-91bc-326040c05ecf (/health/instance) 2020-03-31 09:22:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:48,277 [554] [DEBUG] [app] Starting request: urn:request:2ee1ea66-366e-48bf-8add-f842b42f6fb4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:48,277 [554] [DEBUG] [app] Starting request: urn:request:2ee1ea66-366e-48bf-8add-f842b42f6fb4 (/v1/_internal_ping) 2020-03-31 09:22:48,278 [554] [DEBUG] [app] Ending request: urn:request:2ee1ea66-366e-48bf-8add-f842b42f6fb4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:48,278 [554] [DEBUG] [app] Ending request: urn:request:2ee1ea66-366e-48bf-8add-f842b42f6fb4 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:48,279 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:48,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:48,279 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:48,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:48,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:48,281 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:22:48,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:22:48,281 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:48,282 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:48,282 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:48,283 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:48,283 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:48,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:48,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:48,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:48,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:48,310 [527] [DEBUG] [app] Starting request: urn:request:9a389c0d-ba24-4071-8741-ea797122d39f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:48,310 [527] [DEBUG] [app] Starting request: urn:request:9a389c0d-ba24-4071-8741-ea797122d39f (/_internal_ping) 2020-03-31 09:22:48,311 [527] [DEBUG] [app] Ending request: urn:request:9a389c0d-ba24-4071-8741-ea797122d39f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:48,311 [527] [DEBUG] [app] Ending request: urn:request:9a389c0d-ba24-4071-8741-ea797122d39f (/_internal_ping) 2020-03-31 09:22:48,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:22:48,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:48,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:22:48,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:48,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:22:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:22:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:22:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:22:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:48,323 [532] [DEBUG] [app] Starting request: urn:request:7ec0bbc5-4a51-4eff-bbdf-8f8015c3f6b0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:48,323 [532] [DEBUG] [app] Starting request: urn:request:7ec0bbc5-4a51-4eff-bbdf-8f8015c3f6b0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:48,324 [532] [DEBUG] [app] Ending request: urn:request:7ec0bbc5-4a51-4eff-bbdf-8f8015c3f6b0 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:48,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:48,324 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:22:48,324 [532] [DEBUG] [app] Ending request: urn:request:7ec0bbc5-4a51-4eff-bbdf-8f8015c3f6b0 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:48,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:48,325 [527] [DEBUG] [app] Ending request: urn:request:d53dac1f-a4e3-46c3-91bc-326040c05ecf (/health/instance) 2020-03-31 09:22:48,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:22:48,324 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:48,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:22:48,325 [527] [DEBUG] [app] Ending request: urn:request:d53dac1f-a4e3-46c3-91bc-326040c05ecf (/health/instance) 2020-03-31 09:22:48,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:48,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:22:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:54.323352+00:00 (in 4.823345 seconds) 2020-03-31 09:22:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:49 UTC)" (scheduled at 2020-03-31 09:22:49.498966+00:00) 2020-03-31 09:22:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:54.323352+00:00 (in 4.823345 seconds) 2020-03-31 09:22:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:49 UTC)" (scheduled at 2020-03-31 09:22:49.498966+00:00) 2020-03-31 09:22:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:49 UTC)" executed successfully 2020-03-31 09:22:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:22:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:59.560326+00:00 (in 8.448628 seconds) 2020-03-31 09:22:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:51 UTC)" (scheduled at 2020-03-31 09:22:51.110610+00:00) 2020-03-31 09:22:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:22:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:22:59.560326+00:00 (in 8.448628 seconds) 2020-03-31 09:22:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:51 UTC)" (scheduled at 2020-03-31 09:22:51.110610+00:00) 2020-03-31 09:22:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 51, 112686), True, datetime.datetime(2020, 3, 31, 9, 22, 51, 112686), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:22:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 51, 112686), True, datetime.datetime(2020, 3, 31, 9, 22, 51, 112686), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:22:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:22:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:22:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:51 UTC)" executed successfully 2020-03-31 09:22:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:22:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:01 UTC)" (scheduled at 2020-03-31 09:22:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:22:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:01 UTC)" (scheduled at 2020-03-31 09:22:51.440707+00:00) 2020-03-31 09:22:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:22:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:22:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:01.440707+00:00 (in 9.999152 seconds) repositoryactioncounter stdout | 2020-03-31 09:22:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:01.440707+00:00 (in 9.999152 seconds) 2020-03-31 09:22:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:22:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:22:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:22:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:01 UTC)" executed successfully 2020-03-31 09:22:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:22:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:22.113221+00:00 (in 29.999235 seconds) 2020-03-31 09:22:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:22 UTC)" (scheduled at 2020-03-31 09:22:52.113221+00:00) securityworker stdout | 2020-03-31 09:22:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:22.113221+00:00 (in 29.999235 seconds) 2020-03-31 09:22:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:22 UTC)" (scheduled at 2020-03-31 09:22:52.113221+00:00) 2020-03-31 09:22:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:22:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:22:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:22:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:22:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:22:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:22:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:22:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:22 UTC)" executed successfully 2020-03-31 09:22:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:22:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:04.323352+00:00 (in 9.997910 seconds) 2020-03-31 09:22:54,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:04 UTC)" (scheduled at 2020-03-31 09:22:54.323352+00:00) 2020-03-31 09:22:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:22:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:04.323352+00:00 (in 9.997910 seconds) 2020-03-31 09:22:54,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:04 UTC)" (scheduled at 2020-03-31 09:22:54.323352+00:00) 2020-03-31 09:22:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 54, 326295), True, datetime.datetime(2020, 3, 31, 9, 22, 54, 326295), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:22:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 54, 326295), True, datetime.datetime(2020, 3, 31, 9, 22, 54, 326295), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:22:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:22:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:22:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:04 UTC)" executed successfully 2020-03-31 09:22:54,446 [518] [DEBUG] [app] Starting request: urn:request:12a9729b-ac01-4d9a-8337-50761ec561a3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:54,446 [518] [DEBUG] [app] Starting request: urn:request:12a9729b-ac01-4d9a-8337-50761ec561a3 (/health/instance) 2020-03-31 09:22:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:54,453 [549] [DEBUG] [app] Starting request: urn:request:d0ead1c9-c4f2-4b9a-9315-bfca63ff4591 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:54,453 [549] [DEBUG] [app] Starting request: urn:request:d0ead1c9-c4f2-4b9a-9315-bfca63ff4591 (/v1/_internal_ping) 2020-03-31 09:22:54,454 [549] [DEBUG] [app] Ending request: urn:request:d0ead1c9-c4f2-4b9a-9315-bfca63ff4591 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:54,454 [549] [DEBUG] [app] Ending request: urn:request:d0ead1c9-c4f2-4b9a-9315-bfca63ff4591 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:54,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:54,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:22:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:22:54,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:22:54,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:54,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:54,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:54,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:54,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:54,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:54,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:54,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:54,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:54,484 [518] [DEBUG] [app] Starting request: urn:request:b7f9170c-04f8-4a77-a0c1-90fda16e9639 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:54,484 [518] [DEBUG] [app] Starting request: urn:request:b7f9170c-04f8-4a77-a0c1-90fda16e9639 (/_internal_ping) 2020-03-31 09:22:54,485 [518] [DEBUG] [app] Ending request: urn:request:b7f9170c-04f8-4a77-a0c1-90fda16e9639 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:54,485 [518] [DEBUG] [app] Ending request: urn:request:b7f9170c-04f8-4a77-a0c1-90fda16e9639 (/_internal_ping) 2020-03-31 09:22:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:22:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:22:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:22:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:22:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:22:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:22:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:54,500 [534] [DEBUG] [app] Starting request: urn:request:dd748bd2-de70-46c2-ab78-237e7f058d1b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:54,500 [534] [DEBUG] [app] Starting request: urn:request:dd748bd2-de70-46c2-ab78-237e7f058d1b (/c1/_internal_ping) 2020-03-31 09:22:54,501 [534] [DEBUG] [app] Ending request: urn:request:dd748bd2-de70-46c2-ab78-237e7f058d1b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:54,501 [534] [DEBUG] [app] Ending request: urn:request:dd748bd2-de70-46c2-ab78-237e7f058d1b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:54,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:22:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:22:54,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:54,504 [518] [DEBUG] [app] Ending request: urn:request:12a9729b-ac01-4d9a-8337-50761ec561a3 (/health/instance) 2020-03-31 09:22:54,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:54,504 [518] [DEBUG] [app] Ending request: urn:request:12a9729b-ac01-4d9a-8337-50761ec561a3 (/health/instance) 2020-03-31 09:22:54,504 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:22:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:22:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:22:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:22:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:06.924497+00:00 (in 9.999165 seconds) 2020-03-31 09:22:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:06 UTC)" (scheduled at 2020-03-31 09:22:56.924497+00:00) 2020-03-31 09:22:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:22:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:06.924497+00:00 (in 9.999165 seconds) 2020-03-31 09:22:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:06 UTC)" (scheduled at 2020-03-31 09:22:56.924497+00:00) 2020-03-31 09:22:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:22:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 56, 926048), True, datetime.datetime(2020, 3, 31, 9, 22, 56, 926048), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:22:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 22, 56, 926048), True, datetime.datetime(2020, 3, 31, 9, 22, 56, 926048), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:22:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:22:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:22:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:22:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:22:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:06 UTC)" executed successfully 2020-03-31 09:22:58,273 [527] [DEBUG] [app] Starting request: urn:request:d69888cb-2a6d-4e5b-a9e6-c82b8d4ac489 (/health/instance) gunicorn-web stdout | 2020-03-31 09:22:58,273 [527] [DEBUG] [app] Starting request: urn:request:d69888cb-2a6d-4e5b-a9e6-c82b8d4ac489 (/health/instance) 2020-03-31 09:22:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:58,280 [547] [DEBUG] [app] Starting request: urn:request:3b048f7d-5f30-4b41-8334-2704fe6519a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:58,280 [547] [DEBUG] [app] Starting request: urn:request:3b048f7d-5f30-4b41-8334-2704fe6519a0 (/v1/_internal_ping) 2020-03-31 09:22:58,281 [547] [DEBUG] [app] Ending request: urn:request:3b048f7d-5f30-4b41-8334-2704fe6519a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:22:58,281 [547] [DEBUG] [app] Ending request: urn:request:3b048f7d-5f30-4b41-8334-2704fe6519a0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:22:58,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:22:58,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:22:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:22:58,284 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:22:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:22:58,284 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:22:58,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:22:58,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:22:58,286 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:22:58,286 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:22:58,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:22:58,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:22:58,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:22:58,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:22:58,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:58,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:58,316 [527] [DEBUG] [app] Starting request: urn:request:4b5af861-404d-4772-a302-fa8108377396 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:58,316 [527] [DEBUG] [app] Starting request: urn:request:4b5af861-404d-4772-a302-fa8108377396 (/_internal_ping) 2020-03-31 09:22:58,317 [527] [DEBUG] [app] Ending request: urn:request:4b5af861-404d-4772-a302-fa8108377396 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:22:58,317 [527] [DEBUG] [app] Ending request: urn:request:4b5af861-404d-4772-a302-fa8108377396 (/_internal_ping) 2020-03-31 09:22:58,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:22:58,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:22:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:22:58,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:22:58,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:22:58,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:58,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:22:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:22:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:22:58,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:22:58,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:22:58,330 [539] [DEBUG] [app] Starting request: urn:request:8705e43b-dc85-4e78-a853-aa9dded995ac (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:58,330 [539] [DEBUG] [app] Starting request: urn:request:8705e43b-dc85-4e78-a853-aa9dded995ac (/c1/_internal_ping) 2020-03-31 09:22:58,331 [539] [DEBUG] [app] Ending request: urn:request:8705e43b-dc85-4e78-a853-aa9dded995ac (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:22:58,331 [539] [DEBUG] [app] Ending request: urn:request:8705e43b-dc85-4e78-a853-aa9dded995ac (/c1/_internal_ping) 2020-03-31 09:22:58,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:58,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:22:58,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:22:58,333 [527] [DEBUG] [app] Ending request: urn:request:d69888cb-2a6d-4e5b-a9e6-c82b8d4ac489 (/health/instance) 2020-03-31 09:22:58,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:22:58,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:22:58,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:22:58,333 [527] [DEBUG] [app] Ending request: urn:request:d69888cb-2a6d-4e5b-a9e6-c82b8d4ac489 (/health/instance) 2020-03-31 09:22:58,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:22:58,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:22:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:22:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:22:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:22:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:22:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:51.110610+00:00 (in 51.549483 seconds) 2020-03-31 09:22:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:59 UTC)" (scheduled at 2020-03-31 09:22:59.560326+00:00) 2020-03-31 09:22:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:22:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:51.110610+00:00 (in 51.549483 seconds) 2020-03-31 09:22:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:59 UTC)" (scheduled at 2020-03-31 09:22:59.560326+00:00) 2020-03-31 09:22:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:22:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:59 UTC)" executed successfully 2020-03-31 09:23:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:23:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:30.337699+00:00 (in 29.998197 seconds) 2020-03-31 09:23:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:30 UTC)" (scheduled at 2020-03-31 09:23:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:23:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:30.337699+00:00 (in 29.998197 seconds) 2020-03-31 09:23:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:30 UTC)" (scheduled at 2020-03-31 09:23:00.337699+00:00) 2020-03-31 09:23:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 23, 0, 340348), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:23:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 23, 0, 340348), False, 50, 1, 0]) 2020-03-31 09:23:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:23:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:23:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:23:00,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:30 UTC)" executed successfully 2020-03-31 09:23:01,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:23:01,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:11.440707+00:00 (in 9.995309 seconds) 2020-03-31 09:23:01,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:11 UTC)" (scheduled at 2020-03-31 09:23:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:23:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:11.440707+00:00 (in 9.995309 seconds) 2020-03-31 09:23:01,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:11 UTC)" (scheduled at 2020-03-31 09:23:01.440707+00:00) 2020-03-31 09:23:01,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:23:01,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:23:01,467 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:23:01,467 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:23:01,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:01,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:23:01,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:01,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:11 UTC)" executed successfully 2020-03-31 09:23:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:23:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:24.410927+00:00 (in 21.079538 seconds) 2020-03-31 09:23:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:33 UTC)" (scheduled at 2020-03-31 09:23:03.330112+00:00) 2020-03-31 09:23:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:23:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:24.410927+00:00 (in 21.079538 seconds) 2020-03-31 09:23:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:33 UTC)" (scheduled at 2020-03-31 09:23:03.330112+00:00) 2020-03-31 09:23:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 3, 332173), True, datetime.datetime(2020, 3, 31, 9, 23, 3, 332173), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:23:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 3, 332173), True, datetime.datetime(2020, 3, 31, 9, 23, 3, 332173), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:23:03,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:23:03,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:23:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:33 UTC)" executed successfully 2020-03-31 09:23:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:23:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:14.323352+00:00 (in 9.999123 seconds) 2020-03-31 09:23:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:14 UTC)" (scheduled at 2020-03-31 09:23:04.323352+00:00) 2020-03-31 09:23:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:23:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:14.323352+00:00 (in 9.999123 seconds) 2020-03-31 09:23:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:14 UTC)" (scheduled at 2020-03-31 09:23:04.323352+00:00) 2020-03-31 09:23:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 4, 325015), True, datetime.datetime(2020, 3, 31, 9, 23, 4, 325015), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:23:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 4, 325015), True, datetime.datetime(2020, 3, 31, 9, 23, 4, 325015), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:23:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:04,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:23:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:23:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:14 UTC)" executed successfully 2020-03-31 09:23:04,446 [518] [DEBUG] [app] Starting request: urn:request:a6a2bc6a-75c5-4a66-a1a2-d86c680a7611 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:04,446 [518] [DEBUG] [app] Starting request: urn:request:a6a2bc6a-75c5-4a66-a1a2-d86c680a7611 (/health/instance) 2020-03-31 09:23:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:04,454 [549] [DEBUG] [app] Starting request: urn:request:2ee1d8e9-23e5-4ec0-b3b0-7cbaf203460d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:04,454 [549] [DEBUG] [app] Starting request: urn:request:2ee1d8e9-23e5-4ec0-b3b0-7cbaf203460d (/v1/_internal_ping) 2020-03-31 09:23:04,455 [549] [DEBUG] [app] Ending request: urn:request:2ee1d8e9-23e5-4ec0-b3b0-7cbaf203460d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:04,455 [549] [DEBUG] [app] Ending request: urn:request:2ee1d8e9-23e5-4ec0-b3b0-7cbaf203460d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:04,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:23:04,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:23:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:23:04,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:23:04,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:04,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:04,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:04,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:04,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:04,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:04,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:04,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:04,493 [518] [DEBUG] [app] Starting request: urn:request:bedeae52-3e64-437d-be28-68a7baeb711f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:04,493 [518] [DEBUG] [app] Starting request: urn:request:bedeae52-3e64-437d-be28-68a7baeb711f (/_internal_ping) 2020-03-31 09:23:04,494 [518] [DEBUG] [app] Ending request: urn:request:bedeae52-3e64-437d-be28-68a7baeb711f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:04,494 [518] [DEBUG] [app] Ending request: urn:request:bedeae52-3e64-437d-be28-68a7baeb711f (/_internal_ping) 2020-03-31 09:23:04,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:23:04,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:23:04,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:23:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:23:04,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:04,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:23:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:04,507 [532] [DEBUG] [app] Starting request: urn:request:faee4e2f-9191-46c3-b973-c73d8b2ce77f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:04,507 [532] [DEBUG] [app] Starting request: urn:request:faee4e2f-9191-46c3-b973-c73d8b2ce77f (/c1/_internal_ping) 2020-03-31 09:23:04,508 [532] [DEBUG] [app] Ending request: urn:request:faee4e2f-9191-46c3-b973-c73d8b2ce77f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:04,508 [532] [DEBUG] [app] Ending request: urn:request:faee4e2f-9191-46c3-b973-c73d8b2ce77f (/c1/_internal_ping) 2020-03-31 09:23:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:04,509 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:04,510 [518] [DEBUG] [app] Ending request: urn:request:a6a2bc6a-75c5-4a66-a1a2-d86c680a7611 (/health/instance) 2020-03-31 09:23:04,510 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:23:04,509 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:04,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:23:04,510 [518] [DEBUG] [app] Ending request: urn:request:a6a2bc6a-75c5-4a66-a1a2-d86c680a7611 (/health/instance) 2020-03-31 09:23:04,510 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:04,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:23:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:23:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:16.924497+00:00 (in 9.999244 seconds) 2020-03-31 09:23:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:16 UTC)" (scheduled at 2020-03-31 09:23:06.924497+00:00) 2020-03-31 09:23:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:23:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:16.924497+00:00 (in 9.999244 seconds) 2020-03-31 09:23:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:16 UTC)" (scheduled at 2020-03-31 09:23:06.924497+00:00) 2020-03-31 09:23:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 6, 925956), True, datetime.datetime(2020, 3, 31, 9, 23, 6, 925956), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:23:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 6, 925956), True, datetime.datetime(2020, 3, 31, 9, 23, 6, 925956), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:23:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:23:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:23:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:16 UTC)" executed successfully 2020-03-31 09:23:08,273 [527] [DEBUG] [app] Starting request: urn:request:1aae301d-d5d3-4eb1-81a8-7d48d7792b32 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:08,273 [527] [DEBUG] [app] Starting request: urn:request:1aae301d-d5d3-4eb1-81a8-7d48d7792b32 (/health/instance) 2020-03-31 09:23:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:08,281 [538] [DEBUG] [app] Starting request: urn:request:2e76dd63-0a30-4265-9eac-be1ae4acb942 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:08,281 [538] [DEBUG] [app] Starting request: urn:request:2e76dd63-0a30-4265-9eac-be1ae4acb942 (/v1/_internal_ping) 2020-03-31 09:23:08,282 [538] [DEBUG] [app] Ending request: urn:request:2e76dd63-0a30-4265-9eac-be1ae4acb942 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:08,282 [538] [DEBUG] [app] Ending request: urn:request:2e76dd63-0a30-4265-9eac-be1ae4acb942 (/v1/_internal_ping) 2020-03-31 09:23:08,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:23:08,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:08,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:23:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:08,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:23:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:23:08,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:08,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:08,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:08,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:08,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:08,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:08,312 [527] [DEBUG] [app] Starting request: urn:request:15edaa0a-8414-4eeb-a270-ccef1b676c32 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:08,312 [527] [DEBUG] [app] Starting request: urn:request:15edaa0a-8414-4eeb-a270-ccef1b676c32 (/_internal_ping) 2020-03-31 09:23:08,313 [527] [DEBUG] [app] Ending request: urn:request:15edaa0a-8414-4eeb-a270-ccef1b676c32 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:08,313 [527] [DEBUG] [app] Ending request: urn:request:15edaa0a-8414-4eeb-a270-ccef1b676c32 (/_internal_ping) 2020-03-31 09:23:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:08,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:23:08,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:08,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:23:08,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:08,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:08,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:08,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:23:08,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:08,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:08,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:08,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:08,326 [532] [DEBUG] [app] Starting request: urn:request:bb2699ef-4cbf-4fd6-97bb-f50d2b57502d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:08,326 [532] [DEBUG] [app] Starting request: urn:request:bb2699ef-4cbf-4fd6-97bb-f50d2b57502d (/c1/_internal_ping) 2020-03-31 09:23:08,327 [532] [DEBUG] [app] Ending request: urn:request:bb2699ef-4cbf-4fd6-97bb-f50d2b57502d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:08,327 [532] [DEBUG] [app] Ending request: urn:request:bb2699ef-4cbf-4fd6-97bb-f50d2b57502d (/c1/_internal_ping) 2020-03-31 09:23:08,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:08,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:08,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:23:08,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:08,329 [527] [DEBUG] [app] Ending request: urn:request:1aae301d-d5d3-4eb1-81a8-7d48d7792b32 (/health/instance) 2020-03-31 09:23:08,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:08,329 [527] [DEBUG] [app] Ending request: urn:request:1aae301d-d5d3-4eb1-81a8-7d48d7792b32 (/health/instance) 2020-03-31 09:23:08,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:08,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:23:08,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:23:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:23:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:21.440707+00:00 (in 9.999122 seconds) 2020-03-31 09:23:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:21 UTC)" (scheduled at 2020-03-31 09:23:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:23:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:21.440707+00:00 (in 9.999122 seconds) 2020-03-31 09:23:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:21 UTC)" (scheduled at 2020-03-31 09:23:11.440707+00:00) 2020-03-31 09:23:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:23:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:23:11,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:23:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:23:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:23:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:21 UTC)" executed successfully 2020-03-31 09:23:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:23:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:24.323352+00:00 (in 9.999044 seconds) 2020-03-31 09:23:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:24 UTC)" (scheduled at 2020-03-31 09:23:14.323352+00:00) 2020-03-31 09:23:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:23:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:24.323352+00:00 (in 9.999044 seconds) 2020-03-31 09:23:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:24 UTC)" (scheduled at 2020-03-31 09:23:14.323352+00:00) 2020-03-31 09:23:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 14, 325344), True, datetime.datetime(2020, 3, 31, 9, 23, 14, 325344), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:23:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 14, 325344), True, datetime.datetime(2020, 3, 31, 9, 23, 14, 325344), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:23:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:23:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:23:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:24 UTC)" executed successfully 2020-03-31 09:23:14,446 [518] [DEBUG] [app] Starting request: urn:request:aca98ba3-e04e-467a-86cd-35117b98ce2f (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:14,446 [518] [DEBUG] [app] Starting request: urn:request:aca98ba3-e04e-467a-86cd-35117b98ce2f (/health/instance) 2020-03-31 09:23:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:14,454 [555] [DEBUG] [app] Starting request: urn:request:3eaf8038-9dfb-442c-add1-01fc003b1321 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:14,454 [555] [DEBUG] [app] Starting request: urn:request:3eaf8038-9dfb-442c-add1-01fc003b1321 (/v1/_internal_ping) 2020-03-31 09:23:14,455 [555] [DEBUG] [app] Ending request: urn:request:3eaf8038-9dfb-442c-add1-01fc003b1321 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:14,455 [555] [DEBUG] [app] Ending request: urn:request:3eaf8038-9dfb-442c-add1-01fc003b1321 (/v1/_internal_ping) 2020-03-31 09:23:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:23:14,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:23:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:23:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:23:14,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:14,488 [518] [DEBUG] [app] Starting request: urn:request:7b56fabf-7f51-400e-835e-c87071927640 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:14,488 [518] [DEBUG] [app] Starting request: urn:request:7b56fabf-7f51-400e-835e-c87071927640 (/_internal_ping) 2020-03-31 09:23:14,489 [518] [DEBUG] [app] Ending request: urn:request:7b56fabf-7f51-400e-835e-c87071927640 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:14,489 [518] [DEBUG] [app] Ending request: urn:request:7b56fabf-7f51-400e-835e-c87071927640 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:23:14,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:23:14,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:14,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:14,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:23:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:14,503 [539] [DEBUG] [app] Starting request: urn:request:8b8f9990-db6b-46d5-a002-c429a5d10ff6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:14,503 [539] [DEBUG] [app] Starting request: urn:request:8b8f9990-db6b-46d5-a002-c429a5d10ff6 (/c1/_internal_ping) 2020-03-31 09:23:14,505 [539] [DEBUG] [app] Ending request: urn:request:8b8f9990-db6b-46d5-a002-c429a5d10ff6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:14,505 [539] [DEBUG] [app] Ending request: urn:request:8b8f9990-db6b-46d5-a002-c429a5d10ff6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:14,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:23:14,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:14,508 [518] [DEBUG] [app] Ending request: urn:request:aca98ba3-e04e-467a-86cd-35117b98ce2f (/health/instance) 2020-03-31 09:23:14,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:14,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:23:14,508 [518] [DEBUG] [app] Ending request: urn:request:aca98ba3-e04e-467a-86cd-35117b98ce2f (/health/instance) 2020-03-31 09:23:14,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:14,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:23:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:23:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:45.064514+00:00 (in 29.999000 seconds) 2020-03-31 09:23:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:45 UTC)" (scheduled at 2020-03-31 09:23:15.064514+00:00) gcworker stdout | 2020-03-31 09:23:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:45.064514+00:00 (in 29.999000 seconds) 2020-03-31 09:23:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:45 UTC)" (scheduled at 2020-03-31 09:23:15.064514+00:00) 2020-03-31 09:23:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436995, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:23:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584436995, 1209600, 500, 1, 0]) 2020-03-31 09:23:15,089 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:23:15,089 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:23:15,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:15,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:23:15,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:15,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:45 UTC)" executed successfully 2020-03-31 09:23:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:23:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:26.924497+00:00 (in 9.999253 seconds) 2020-03-31 09:23:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:26 UTC)" (scheduled at 2020-03-31 09:23:16.924497+00:00) 2020-03-31 09:23:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:23:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:26.924497+00:00 (in 9.999253 seconds) 2020-03-31 09:23:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:26 UTC)" (scheduled at 2020-03-31 09:23:16.924497+00:00) 2020-03-31 09:23:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 16, 926157), True, datetime.datetime(2020, 3, 31, 9, 23, 16, 926157), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:23:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 16, 926157), True, datetime.datetime(2020, 3, 31, 9, 23, 16, 926157), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:23:16,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:23:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:23:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:16,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:26 UTC)" executed successfully 2020-03-31 09:23:18,273 [527] [DEBUG] [app] Starting request: urn:request:7466910f-dd00-4fe0-b855-0f5ef40ca721 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:18,273 [527] [DEBUG] [app] Starting request: urn:request:7466910f-dd00-4fe0-b855-0f5ef40ca721 (/health/instance) 2020-03-31 09:23:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:18,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:18,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:18,281 [551] [DEBUG] [app] Starting request: urn:request:95d028e2-e273-4069-a686-2e7578839312 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:18,281 [551] [DEBUG] [app] Starting request: urn:request:95d028e2-e273-4069-a686-2e7578839312 (/v1/_internal_ping) 2020-03-31 09:23:18,282 [551] [DEBUG] [app] Ending request: urn:request:95d028e2-e273-4069-a686-2e7578839312 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:18,282 [551] [DEBUG] [app] Ending request: urn:request:95d028e2-e273-4069-a686-2e7578839312 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:18,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:23:18,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:18,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:23:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:23:18,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:18,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:18,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:18,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:18,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:18,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:18,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:18,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:18,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:18,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:18,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:18,311 [527] [DEBUG] [app] Starting request: urn:request:80e15f26-31e1-4fc4-9350-1f8e83182841 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:18,311 [527] [DEBUG] [app] Starting request: urn:request:80e15f26-31e1-4fc4-9350-1f8e83182841 (/_internal_ping) 2020-03-31 09:23:18,312 [527] [DEBUG] [app] Ending request: urn:request:80e15f26-31e1-4fc4-9350-1f8e83182841 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:18,312 [527] [DEBUG] [app] Ending request: urn:request:80e15f26-31e1-4fc4-9350-1f8e83182841 (/_internal_ping) 2020-03-31 09:23:18,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:18,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:23:18,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:18,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:23:18,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:18,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:18,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:18,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:18,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:18,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:23:18,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:18,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:18,327 [534] [DEBUG] [app] Starting request: urn:request:6fd8f064-ab1b-4c8b-8788-099ef89b3f67 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:18,327 [534] [DEBUG] [app] Starting request: urn:request:6fd8f064-ab1b-4c8b-8788-099ef89b3f67 (/c1/_internal_ping) 2020-03-31 09:23:18,328 [534] [DEBUG] [app] Ending request: urn:request:6fd8f064-ab1b-4c8b-8788-099ef89b3f67 (/c1/_internal_ping) 2020-03-31 09:23:18,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:18,328 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:23:18,328 [534] [DEBUG] [app] Ending request: urn:request:6fd8f064-ab1b-4c8b-8788-099ef89b3f67 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:18,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:18,330 [527] [DEBUG] [app] Ending request: urn:request:7466910f-dd00-4fe0-b855-0f5ef40ca721 (/health/instance) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:18,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:18,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-verbs stdout | 2020-03-31 09:23:18,328 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:18,330 [527] [DEBUG] [app] Ending request: urn:request:7466910f-dd00-4fe0-b855-0f5ef40ca721 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:18,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:18,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:23:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:23:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:31.440707+00:00 (in 9.999251 seconds) 2020-03-31 09:23:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:31 UTC)" (scheduled at 2020-03-31 09:23:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:23:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:31.440707+00:00 (in 9.999251 seconds) 2020-03-31 09:23:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:31 UTC)" (scheduled at 2020-03-31 09:23:21.440707+00:00) 2020-03-31 09:23:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:23:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:23:21,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:23:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:23:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:23:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:31 UTC)" executed successfully 2020-03-31 09:23:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:23:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:52.113221+00:00 (in 29.998704 seconds) 2020-03-31 09:23:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:52 UTC)" (scheduled at 2020-03-31 09:23:22.113221+00:00) securityworker stdout | 2020-03-31 09:23:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:52.113221+00:00 (in 29.998704 seconds) 2020-03-31 09:23:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:52 UTC)" (scheduled at 2020-03-31 09:23:22.113221+00:00) 2020-03-31 09:23:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:23:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:23:22,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:23:22,133 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:23:22,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:23:22,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:23:22,136 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:22,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:23:22,136 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:22,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:23:52 UTC)" executed successfully 2020-03-31 09:23:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:23:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:34 UTC)" (scheduled at 2020-03-31 09:23:24.323352+00:00) 2020-03-31 09:23:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:23:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:34 UTC)" (scheduled at 2020-03-31 09:23:24.323352+00:00) 2020-03-31 09:23:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:34.323352+00:00 (in 9.999191 seconds) notificationworker stdout | 2020-03-31 09:23:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:34.323352+00:00 (in 9.999191 seconds) 2020-03-31 09:23:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 24, 324975), True, datetime.datetime(2020, 3, 31, 9, 23, 24, 324975), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:23:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 24, 324975), True, datetime.datetime(2020, 3, 31, 9, 23, 24, 324975), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:23:24,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:23:24,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:24,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:24,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:23:24,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:24,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:34 UTC)" executed successfully 2020-03-31 09:23:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:33.330112+00:00 (in 8.918145 seconds) 2020-03-31 09:23:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:24 UTC)" (scheduled at 2020-03-31 09:23:24.410927+00:00) 2020-03-31 09:23:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:23:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:23:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:33.330112+00:00 (in 8.918145 seconds) 2020-03-31 09:23:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:24 UTC)" (scheduled at 2020-03-31 09:23:24.410927+00:00) 2020-03-31 09:23:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:23:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:24 UTC)" executed successfully 2020-03-31 09:23:24,445 [526] [DEBUG] [app] Starting request: urn:request:dd868c85-a581-427f-a555-d22eb3100c3b (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:24,445 [526] [DEBUG] [app] Starting request: urn:request:dd868c85-a581-427f-a555-d22eb3100c3b (/health/instance) 2020-03-31 09:23:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:24,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:24,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:24,454 [538] [DEBUG] [app] Starting request: urn:request:df4c066b-7045-486e-ac3d-4dda777b6633 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:24,454 [538] [DEBUG] [app] Starting request: urn:request:df4c066b-7045-486e-ac3d-4dda777b6633 (/v1/_internal_ping) 2020-03-31 09:23:24,455 [538] [DEBUG] [app] Ending request: urn:request:df4c066b-7045-486e-ac3d-4dda777b6633 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:24,455 [538] [DEBUG] [app] Ending request: urn:request:df4c066b-7045-486e-ac3d-4dda777b6633 (/v1/_internal_ping) 2020-03-31 09:23:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:24,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:23:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:24,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:24,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:23:24,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:23:24,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:23:24,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:24,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:24,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:24,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:24,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:24,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:24,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:24,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:24,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:24,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:24,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:24,486 [518] [DEBUG] [app] Starting request: urn:request:19efdb51-805d-4be4-ab19-52d27e91f353 (/_internal_ping) 2020-03-31 09:23:24,487 [518] [DEBUG] [app] Ending request: urn:request:19efdb51-805d-4be4-ab19-52d27e91f353 (/_internal_ping) 2020-03-31 09:23:24,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:24,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:23:24,486 [518] [DEBUG] [app] Starting request: urn:request:19efdb51-805d-4be4-ab19-52d27e91f353 (/_internal_ping) 2020-03-31 09:23:24,487 [518] [DEBUG] [app] Ending request: urn:request:19efdb51-805d-4be4-ab19-52d27e91f353 (/_internal_ping) 2020-03-31 09:23:24,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:24,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:23:24,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:24,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:24,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:24,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:24,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:24,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:24,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:24,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:24,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:24,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:24,500 [534] [DEBUG] [app] Starting request: urn:request:3f3eceba-4e91-43bf-b6d5-b2f65faa4ebd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:24,500 [534] [DEBUG] [app] Starting request: urn:request:3f3eceba-4e91-43bf-b6d5-b2f65faa4ebd (/c1/_internal_ping) 2020-03-31 09:23:24,501 [534] [DEBUG] [app] Ending request: urn:request:3f3eceba-4e91-43bf-b6d5-b2f65faa4ebd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:24,501 [534] [DEBUG] [app] Ending request: urn:request:3f3eceba-4e91-43bf-b6d5-b2f65faa4ebd (/c1/_internal_ping) 2020-03-31 09:23:24,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:24,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:24,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:24,503 [526] [DEBUG] [app] Ending request: urn:request:dd868c85-a581-427f-a555-d22eb3100c3b (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:23:24,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:24,504 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:24,504 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:23:24,503 [526] [DEBUG] [app] Ending request: urn:request:dd868c85-a581-427f-a555-d22eb3100c3b (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:24,504 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:24,504 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:23:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:23:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:36.924497+00:00 (in 9.999131 seconds) 2020-03-31 09:23:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:36 UTC)" (scheduled at 2020-03-31 09:23:26.924497+00:00) 2020-03-31 09:23:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:23:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:36.924497+00:00 (in 9.999131 seconds) 2020-03-31 09:23:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:36 UTC)" (scheduled at 2020-03-31 09:23:26.924497+00:00) 2020-03-31 09:23:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 26, 926369), True, datetime.datetime(2020, 3, 31, 9, 23, 26, 926369), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:23:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 26, 926369), True, datetime.datetime(2020, 3, 31, 9, 23, 26, 926369), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:23:26,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:23:26,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:23:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:36 UTC)" executed successfully 2020-03-31 09:23:28,273 [527] [DEBUG] [app] Starting request: urn:request:4797c21f-083c-420e-8e27-1d0f13310b6f (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:28,273 [527] [DEBUG] [app] Starting request: urn:request:4797c21f-083c-420e-8e27-1d0f13310b6f (/health/instance) 2020-03-31 09:23:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:28,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:28,283 [554] [DEBUG] [app] Starting request: urn:request:9d674119-fa6a-4679-9833-151979ef1a4f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:28,283 [554] [DEBUG] [app] Starting request: urn:request:9d674119-fa6a-4679-9833-151979ef1a4f (/v1/_internal_ping) 2020-03-31 09:23:28,284 [554] [DEBUG] [app] Ending request: urn:request:9d674119-fa6a-4679-9833-151979ef1a4f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:28,284 [554] [DEBUG] [app] Ending request: urn:request:9d674119-fa6a-4679-9833-151979ef1a4f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:23:28,285 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:23:28,285 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:28,287 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:23:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:23:28,287 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:28,289 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:28,289 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:28,290 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:28,290 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:28,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:28,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:28,319 [526] [DEBUG] [app] Starting request: urn:request:c76bb62c-5e04-43a7-90c4-6a21da917fff (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:28,319 [526] [DEBUG] [app] Starting request: urn:request:c76bb62c-5e04-43a7-90c4-6a21da917fff (/_internal_ping) 2020-03-31 09:23:28,321 [526] [DEBUG] [app] Ending request: urn:request:c76bb62c-5e04-43a7-90c4-6a21da917fff (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:28,321 [526] [DEBUG] [app] Ending request: urn:request:c76bb62c-5e04-43a7-90c4-6a21da917fff (/_internal_ping) 2020-03-31 09:23:28,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:28,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:23:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:23:28,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:28,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:28,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:28,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:28,336 [539] [DEBUG] [app] Starting request: urn:request:28f1214e-8e85-4eaf-bb4f-60b14fcc371b (/c1/_internal_ping) 2020-03-31 09:23:28,337 [539] [DEBUG] [app] Ending request: urn:request:28f1214e-8e85-4eaf-bb4f-60b14fcc371b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:28,336 [539] [DEBUG] [app] Starting request: urn:request:28f1214e-8e85-4eaf-bb4f-60b14fcc371b (/c1/_internal_ping) 2020-03-31 09:23:28,338 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:28,338 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:23:28,337 [539] [DEBUG] [app] Ending request: urn:request:28f1214e-8e85-4eaf-bb4f-60b14fcc371b (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:28,338 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:28,340 [527] [DEBUG] [app] Ending request: urn:request:4797c21f-083c-420e-8e27-1d0f13310b6f (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:23:28,338 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:28,341 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:28,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:23:28,340 [527] [DEBUG] [app] Ending request: urn:request:4797c21f-083c-420e-8e27-1d0f13310b6f (/health/instance) 10.131.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:23:28,341 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:28,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:23:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:23:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:00.337699+00:00 (in 29.999204 seconds) 2020-03-31 09:23:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:00 UTC)" (scheduled at 2020-03-31 09:23:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:23:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:00.337699+00:00 (in 29.999204 seconds) 2020-03-31 09:23:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:00 UTC)" (scheduled at 2020-03-31 09:23:30.337699+00:00) 2020-03-31 09:23:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 23, 30, 339423), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:23:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 23, 30, 339423), False, 50, 1, 0]) 2020-03-31 09:23:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:23:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:23:30,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:23:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:00 UTC)" executed successfully 2020-03-31 09:23:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:23:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:41.440707+00:00 (in 9.999027 seconds) 2020-03-31 09:23:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:41 UTC)" (scheduled at 2020-03-31 09:23:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:23:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:41.440707+00:00 (in 9.999027 seconds) 2020-03-31 09:23:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:41 UTC)" (scheduled at 2020-03-31 09:23:31.440707+00:00) 2020-03-31 09:23:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:23:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:23:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:23:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:23:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:23:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:41 UTC)" executed successfully 2020-03-31 09:23:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:23:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:03.330112+00:00 (in 29.998403 seconds) 2020-03-31 09:23:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:03 UTC)" (scheduled at 2020-03-31 09:23:33.330112+00:00) 2020-03-31 09:23:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:23:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:03.330112+00:00 (in 29.998403 seconds) 2020-03-31 09:23:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:03 UTC)" (scheduled at 2020-03-31 09:23:33.330112+00:00) 2020-03-31 09:23:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 33, 332766), True, datetime.datetime(2020, 3, 31, 9, 23, 33, 332766), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:23:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 33, 332766), True, datetime.datetime(2020, 3, 31, 9, 23, 33, 332766), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:23:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:33,351 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:23:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:23:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:03 UTC)" executed successfully 2020-03-31 09:23:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:23:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:43.539999+00:00 (in 10.171883 seconds) 2020-03-31 09:23:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:33 UTC)" (scheduled at 2020-03-31 09:23:33.366791+00:00) 2020-03-31 09:23:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:23:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:43.539999+00:00 (in 10.171883 seconds) 2020-03-31 09:23:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:33 UTC)" (scheduled at 2020-03-31 09:23:33.366791+00:00) 2020-03-31 09:23:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 33, 368979), True, datetime.datetime(2020, 3, 31, 9, 23, 33, 368979), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:23:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 33, 368979), True, datetime.datetime(2020, 3, 31, 9, 23, 33, 368979), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:23:33,388 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:23:33,388 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:33,388 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:33,388 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:23:33,388 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:33,388 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:33 UTC)" executed successfully 2020-03-31 09:23:34,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:23:34,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:34,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:44 UTC)" (scheduled at 2020-03-31 09:23:34.323352+00:00) 2020-03-31 09:23:34,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:34,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:44.323352+00:00 (in 9.995457 seconds) notificationworker stdout | 2020-03-31 09:23:34,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:44 UTC)" (scheduled at 2020-03-31 09:23:34.323352+00:00) 2020-03-31 09:23:34,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:34,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:44.323352+00:00 (in 9.995457 seconds) 2020-03-31 09:23:34,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 34, 328955), True, datetime.datetime(2020, 3, 31, 9, 23, 34, 328955), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:23:34,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 34, 328955), True, datetime.datetime(2020, 3, 31, 9, 23, 34, 328955), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:23:34,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:23:34,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:23:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:44 UTC)" executed successfully 2020-03-31 09:23:34,446 [527] [DEBUG] [app] Starting request: urn:request:e4a84d21-d89c-4e2d-b244-2316681cbe1b (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:34,446 [527] [DEBUG] [app] Starting request: urn:request:e4a84d21-d89c-4e2d-b244-2316681cbe1b (/health/instance) 2020-03-31 09:23:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:34,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:34,454 [547] [DEBUG] [app] Starting request: urn:request:f9cbfdd4-d22a-4bc4-b752-f3735a623c31 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:34,454 [547] [DEBUG] [app] Starting request: urn:request:f9cbfdd4-d22a-4bc4-b752-f3735a623c31 (/v1/_internal_ping) 2020-03-31 09:23:34,455 [547] [DEBUG] [app] Ending request: urn:request:f9cbfdd4-d22a-4bc4-b752-f3735a623c31 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:34,455 [547] [DEBUG] [app] Ending request: urn:request:f9cbfdd4-d22a-4bc4-b752-f3735a623c31 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:34,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:23:34,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:23:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:23:34,459 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:23:34,459 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:34,460 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:34,460 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:34,461 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:34,461 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:34,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:34,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:34,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:34,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:34,485 [518] [DEBUG] [app] Starting request: urn:request:7527475a-70c2-4a07-9794-ce92007d8b3c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:34,485 [518] [DEBUG] [app] Starting request: urn:request:7527475a-70c2-4a07-9794-ce92007d8b3c (/_internal_ping) 2020-03-31 09:23:34,486 [518] [DEBUG] [app] Ending request: urn:request:7527475a-70c2-4a07-9794-ce92007d8b3c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:34,486 [518] [DEBUG] [app] Ending request: urn:request:7527475a-70c2-4a07-9794-ce92007d8b3c (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:23:34,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:23:34,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:34,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:34,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:23:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:34,500 [539] [DEBUG] [app] Starting request: urn:request:0cf17616-6a88-4db2-827b-cda52a05ea27 (/c1/_internal_ping) 2020-03-31 09:23:34,501 [539] [DEBUG] [app] Ending request: urn:request:0cf17616-6a88-4db2-827b-cda52a05ea27 (/c1/_internal_ping) 2020-03-31 09:23:34,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:34,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:23:34,500 [539] [DEBUG] [app] Starting request: urn:request:0cf17616-6a88-4db2-827b-cda52a05ea27 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:34,501 [539] [DEBUG] [app] Ending request: urn:request:0cf17616-6a88-4db2-827b-cda52a05ea27 (/c1/_internal_ping) 2020-03-31 09:23:34,503 [527] [DEBUG] [app] Ending request: urn:request:e4a84d21-d89c-4e2d-b244-2316681cbe1b (/health/instance) 2020-03-31 09:23:34,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:34,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:23:34,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:34,503 [527] [DEBUG] [app] Ending request: urn:request:e4a84d21-d89c-4e2d-b244-2316681cbe1b (/health/instance) 2020-03-31 09:23:34,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:23:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:23:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:38.010934+00:00 (in 1.085608 seconds) 2020-03-31 09:23:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:46 UTC)" (scheduled at 2020-03-31 09:23:36.924497+00:00) 2020-03-31 09:23:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:23:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:38.010934+00:00 (in 1.085608 seconds) 2020-03-31 09:23:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:46 UTC)" (scheduled at 2020-03-31 09:23:36.924497+00:00) 2020-03-31 09:23:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 36, 926347), True, datetime.datetime(2020, 3, 31, 9, 23, 36, 926347), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:23:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 36, 926347), True, datetime.datetime(2020, 3, 31, 9, 23, 36, 926347), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:23:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:23:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:23:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:46 UTC)" executed successfully 2020-03-31 09:23:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:23:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:46.924497+00:00 (in 8.912709 seconds) 2020-03-31 09:23:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:38 UTC)" (scheduled at 2020-03-31 09:23:38.010934+00:00) 2020-03-31 09:23:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:23:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:46.924497+00:00 (in 8.912709 seconds) 2020-03-31 09:23:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:38 UTC)" (scheduled at 2020-03-31 09:23:38.010934+00:00) 2020-03-31 09:23:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:23:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:23:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:38 UTC)" executed successfully 2020-03-31 09:23:38,272 [527] [DEBUG] [app] Starting request: urn:request:5dad22a8-2b8e-4f8f-bf96-3b120c768734 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:38,272 [527] [DEBUG] [app] Starting request: urn:request:5dad22a8-2b8e-4f8f-bf96-3b120c768734 (/health/instance) 2020-03-31 09:23:38,275 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:38,275 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:38,277 [538] [DEBUG] [app] Starting request: urn:request:cc8c1e76-1dd1-47ae-b3cc-a32256e7ac13 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:38,277 [538] [DEBUG] [app] Starting request: urn:request:cc8c1e76-1dd1-47ae-b3cc-a32256e7ac13 (/v1/_internal_ping) 2020-03-31 09:23:38,279 [538] [DEBUG] [app] Ending request: urn:request:cc8c1e76-1dd1-47ae-b3cc-a32256e7ac13 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:38,279 [538] [DEBUG] [app] Ending request: urn:request:cc8c1e76-1dd1-47ae-b3cc-a32256e7ac13 (/v1/_internal_ping) 2020-03-31 09:23:38,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:23:38,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:23:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:23:38,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:23:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:23:38,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:38,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:38,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:38,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:38,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:38,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:38,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:38,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:38,304 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:38,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:38,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:38,309 [525] [DEBUG] [app] Starting request: urn:request:8244a2b4-6dd5-4a75-8be7-386c40ba8d33 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:38,309 [525] [DEBUG] [app] Starting request: urn:request:8244a2b4-6dd5-4a75-8be7-386c40ba8d33 (/_internal_ping) 2020-03-31 09:23:38,311 [525] [DEBUG] [app] Ending request: urn:request:8244a2b4-6dd5-4a75-8be7-386c40ba8d33 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:38,311 [525] [DEBUG] [app] Ending request: urn:request:8244a2b4-6dd5-4a75-8be7-386c40ba8d33 (/_internal_ping) 2020-03-31 09:23:38,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:38,311 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:23:38,311 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:38,311 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:23:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:23:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:23:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:23:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:38,323 [534] [DEBUG] [app] Starting request: urn:request:69e25597-6214-48df-b3b1-ea4907a61691 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:38,323 [534] [DEBUG] [app] Starting request: urn:request:69e25597-6214-48df-b3b1-ea4907a61691 (/c1/_internal_ping) 2020-03-31 09:23:38,324 [534] [DEBUG] [app] Ending request: urn:request:69e25597-6214-48df-b3b1-ea4907a61691 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:38,324 [534] [DEBUG] [app] Ending request: urn:request:69e25597-6214-48df-b3b1-ea4907a61691 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:38,324 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:23:38,324 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:38,326 [527] [DEBUG] [app] Ending request: urn:request:5dad22a8-2b8e-4f8f-bf96-3b120c768734 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:38,326 [527] [DEBUG] [app] Ending request: urn:request:5dad22a8-2b8e-4f8f-bf96-3b120c768734 (/health/instance) 2020-03-31 09:23:38,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:38,326 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:38,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:23:38,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:23:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:23:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:51.440707+00:00 (in 9.999119 seconds) 2020-03-31 09:23:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:51 UTC)" (scheduled at 2020-03-31 09:23:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:23:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:51.440707+00:00 (in 9.999119 seconds) 2020-03-31 09:23:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:51 UTC)" (scheduled at 2020-03-31 09:23:41.440707+00:00) 2020-03-31 09:23:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:23:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:23:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:23:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:23:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:23:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:51 UTC)" executed successfully 2020-03-31 09:23:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:23:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:33.366791+00:00 (in 49.825982 seconds) 2020-03-31 09:23:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:43 UTC)" (scheduled at 2020-03-31 09:23:43.539999+00:00) 2020-03-31 09:23:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:23:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:23:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:33.366791+00:00 (in 49.825982 seconds) 2020-03-31 09:23:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:43 UTC)" (scheduled at 2020-03-31 09:23:43.539999+00:00) 2020-03-31 09:23:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:23:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:43 UTC)" executed successfully 2020-03-31 09:23:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:23:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:54 UTC)" (scheduled at 2020-03-31 09:23:44.323352+00:00) 2020-03-31 09:23:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:23:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:54 UTC)" (scheduled at 2020-03-31 09:23:44.323352+00:00) 2020-03-31 09:23:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:49.498966+00:00 (in 5.174257 seconds) notificationworker stdout | 2020-03-31 09:23:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:49.498966+00:00 (in 5.174257 seconds) 2020-03-31 09:23:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 44, 325594), True, datetime.datetime(2020, 3, 31, 9, 23, 44, 325594), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:23:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 44, 325594), True, datetime.datetime(2020, 3, 31, 9, 23, 44, 325594), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:23:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:23:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:23:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:54 UTC)" executed successfully 2020-03-31 09:23:44,446 [518] [DEBUG] [app] Starting request: urn:request:7cffea66-c0e8-4bfc-a4ec-01a5dc2acd53 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:44,446 [518] [DEBUG] [app] Starting request: urn:request:7cffea66-c0e8-4bfc-a4ec-01a5dc2acd53 (/health/instance) 2020-03-31 09:23:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:44,454 [554] [DEBUG] [app] Starting request: urn:request:23b2dc28-2cde-473d-9b03-97af6e7085fa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:44,454 [554] [DEBUG] [app] Starting request: urn:request:23b2dc28-2cde-473d-9b03-97af6e7085fa (/v1/_internal_ping) 2020-03-31 09:23:44,455 [554] [DEBUG] [app] Ending request: urn:request:23b2dc28-2cde-473d-9b03-97af6e7085fa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:44,455 [554] [DEBUG] [app] Ending request: urn:request:23b2dc28-2cde-473d-9b03-97af6e7085fa (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:44,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:23:44,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:44,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:23:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:23:44,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:44,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:44,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:44,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:44,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:44,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:44,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:44,493 [518] [DEBUG] [app] Starting request: urn:request:58252bc4-35a8-4d16-a379-84038b7452ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:44,493 [518] [DEBUG] [app] Starting request: urn:request:58252bc4-35a8-4d16-a379-84038b7452ce (/_internal_ping) 2020-03-31 09:23:44,494 [518] [DEBUG] [app] Ending request: urn:request:58252bc4-35a8-4d16-a379-84038b7452ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:44,494 [518] [DEBUG] [app] Ending request: urn:request:58252bc4-35a8-4d16-a379-84038b7452ce (/_internal_ping) 2020-03-31 09:23:44,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:23:44,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:23:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:44,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:44,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:44,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:44,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:44,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:44,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:44,512 [532] [DEBUG] [app] Starting request: urn:request:6bfcb7c6-0ae6-42a9-ad87-52ac898e2423 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:44,512 [532] [DEBUG] [app] Starting request: urn:request:6bfcb7c6-0ae6-42a9-ad87-52ac898e2423 (/c1/_internal_ping) 2020-03-31 09:23:44,513 [532] [DEBUG] [app] Ending request: urn:request:6bfcb7c6-0ae6-42a9-ad87-52ac898e2423 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:44,513 [532] [DEBUG] [app] Ending request: urn:request:6bfcb7c6-0ae6-42a9-ad87-52ac898e2423 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:44,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:44,513 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:44,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:23:44,513 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:44,515 [518] [DEBUG] [app] Ending request: urn:request:7cffea66-c0e8-4bfc-a4ec-01a5dc2acd53 (/health/instance) 2020-03-31 09:23:44,515 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:44,515 [518] [DEBUG] [app] Ending request: urn:request:7cffea66-c0e8-4bfc-a4ec-01a5dc2acd53 (/health/instance) 2020-03-31 09:23:44,515 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:44,516 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:23:44,516 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:23:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:23:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:15.064514+00:00 (in 29.998953 seconds) 2020-03-31 09:23:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:15 UTC)" (scheduled at 2020-03-31 09:23:45.064514+00:00) gcworker stdout | 2020-03-31 09:23:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:15.064514+00:00 (in 29.998953 seconds) 2020-03-31 09:23:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:15 UTC)" (scheduled at 2020-03-31 09:23:45.064514+00:00) 2020-03-31 09:23:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437025, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:23:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437025, 1209600, 500, 1, 0]) 2020-03-31 09:23:45,086 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:23:45,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:23:45,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:45,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:23:45,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:45,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:15 UTC)" executed successfully 2020-03-31 09:23:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:23:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:56.924497+00:00 (in 9.999107 seconds) 2020-03-31 09:23:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:56 UTC)" (scheduled at 2020-03-31 09:23:46.924497+00:00) 2020-03-31 09:23:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 46, 926168), True, datetime.datetime(2020, 3, 31, 9, 23, 46, 926168), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:23:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:56.924497+00:00 (in 9.999107 seconds) 2020-03-31 09:23:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:56 UTC)" (scheduled at 2020-03-31 09:23:46.924497+00:00) 2020-03-31 09:23:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:23:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 46, 926168), True, datetime.datetime(2020, 3, 31, 9, 23, 46, 926168), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:23:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:23:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:23:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:23:56 UTC)" executed successfully 2020-03-31 09:23:48,272 [527] [DEBUG] [app] Starting request: urn:request:a1d80dfc-ec45-4bf0-8e94-f687a8b066fb (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:48,272 [527] [DEBUG] [app] Starting request: urn:request:a1d80dfc-ec45-4bf0-8e94-f687a8b066fb (/health/instance) 2020-03-31 09:23:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:48,280 [542] [DEBUG] [app] Starting request: urn:request:a9491485-be47-45e7-96c1-a91e6145d674 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:48,280 [542] [DEBUG] [app] Starting request: urn:request:a9491485-be47-45e7-96c1-a91e6145d674 (/v1/_internal_ping) 2020-03-31 09:23:48,281 [542] [DEBUG] [app] Ending request: urn:request:a9491485-be47-45e7-96c1-a91e6145d674 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:48,281 [542] [DEBUG] [app] Ending request: urn:request:a9491485-be47-45e7-96c1-a91e6145d674 (/v1/_internal_ping) 2020-03-31 09:23:48,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:23:48,282 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:23:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:23:48,284 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:23:48,284 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:48,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:48,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:48,286 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:48,286 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:48,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:48,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:48,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:48,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:48,311 [527] [DEBUG] [app] Starting request: urn:request:1c9af730-6b7d-4d50-b63d-50dd6787a9ed (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:48,311 [527] [DEBUG] [app] Starting request: urn:request:1c9af730-6b7d-4d50-b63d-50dd6787a9ed (/_internal_ping) 2020-03-31 09:23:48,312 [527] [DEBUG] [app] Ending request: urn:request:1c9af730-6b7d-4d50-b63d-50dd6787a9ed (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:48,312 [527] [DEBUG] [app] Ending request: urn:request:1c9af730-6b7d-4d50-b63d-50dd6787a9ed (/_internal_ping) 2020-03-31 09:23:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:23:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:23:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:48,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:48,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:48,325 [539] [DEBUG] [app] Starting request: urn:request:e4ba2158-5476-4cae-ad39-b5ba56e66680 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:48,325 [539] [DEBUG] [app] Starting request: urn:request:e4ba2158-5476-4cae-ad39-b5ba56e66680 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:48,326 [539] [DEBUG] [app] Ending request: urn:request:e4ba2158-5476-4cae-ad39-b5ba56e66680 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:48,326 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:23:48,326 [539] [DEBUG] [app] Ending request: urn:request:e4ba2158-5476-4cae-ad39-b5ba56e66680 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:48,328 [527] [DEBUG] [app] Ending request: urn:request:a1d80dfc-ec45-4bf0-8e94-f687a8b066fb (/health/instance) 2020-03-31 09:23:48,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:23:48,326 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:23:48,328 [527] [DEBUG] [app] Ending request: urn:request:a1d80dfc-ec45-4bf0-8e94-f687a8b066fb (/health/instance) 2020-03-31 09:23:48,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:23:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:23:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:49 UTC)" (scheduled at 2020-03-31 09:23:49.498966+00:00) 2020-03-31 09:23:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:23:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:49 UTC)" (scheduled at 2020-03-31 09:23:49.498966+00:00) 2020-03-31 09:23:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:23:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:49 UTC)" executed successfully 2020-03-31 09:23:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:54.323352+00:00 (in 4.821959 seconds) notificationworker stdout | 2020-03-31 09:23:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:49 UTC)" executed successfully 2020-03-31 09:23:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:54.323352+00:00 (in 4.821959 seconds) 2020-03-31 09:23:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:23:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:51 UTC)" (scheduled at 2020-03-31 09:23:51.110610+00:00) 2020-03-31 09:23:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:59.560326+00:00 (in 8.448885 seconds) 2020-03-31 09:23:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:23:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:23:51 UTC)" (scheduled at 2020-03-31 09:23:51.110610+00:00) 2020-03-31 09:23:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:23:59.560326+00:00 (in 8.448885 seconds) 2020-03-31 09:23:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 51, 112739), True, datetime.datetime(2020, 3, 31, 9, 23, 51, 112739), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:23:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 51, 112739), True, datetime.datetime(2020, 3, 31, 9, 23, 51, 112739), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:23:51,134 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:23:51,134 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:51,134 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:23:51,134 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:51,134 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:51 UTC)" executed successfully 2020-03-31 09:23:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:23:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:01.440707+00:00 (in 9.999008 seconds) 2020-03-31 09:23:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:01 UTC)" (scheduled at 2020-03-31 09:23:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:23:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:01.440707+00:00 (in 9.999008 seconds) 2020-03-31 09:23:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:01 UTC)" (scheduled at 2020-03-31 09:23:51.440707+00:00) 2020-03-31 09:23:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:23:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:23:51,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:23:51,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:23:51,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:51,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:23:51,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:51,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:01 UTC)" executed successfully 2020-03-31 09:23:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:23:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:22.113221+00:00 (in 29.997512 seconds) 2020-03-31 09:23:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:22 UTC)" (scheduled at 2020-03-31 09:23:52.113221+00:00) securityworker stdout | 2020-03-31 09:23:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:22.113221+00:00 (in 29.997512 seconds) 2020-03-31 09:23:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:22 UTC)" (scheduled at 2020-03-31 09:23:52.113221+00:00) 2020-03-31 09:23:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:23:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:23:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:23:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:23:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:23:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:23:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:23:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:22 UTC)" executed successfully 2020-03-31 09:23:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:23:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:04.323352+00:00 (in 9.998121 seconds) 2020-03-31 09:23:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:04 UTC)" (scheduled at 2020-03-31 09:23:54.323352+00:00) 2020-03-31 09:23:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:23:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:04.323352+00:00 (in 9.998121 seconds) 2020-03-31 09:23:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:04 UTC)" (scheduled at 2020-03-31 09:23:54.323352+00:00) 2020-03-31 09:23:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 54, 326154), True, datetime.datetime(2020, 3, 31, 9, 23, 54, 326154), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:23:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 54, 326154), True, datetime.datetime(2020, 3, 31, 9, 23, 54, 326154), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:23:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:54,341 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:23:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:23:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:04 UTC)" executed successfully 2020-03-31 09:23:54,446 [518] [DEBUG] [app] Starting request: urn:request:8b4e94c9-07d1-405a-a0c3-37d3644aa696 (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:54,446 [518] [DEBUG] [app] Starting request: urn:request:8b4e94c9-07d1-405a-a0c3-37d3644aa696 (/health/instance) 2020-03-31 09:23:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:54,454 [554] [DEBUG] [app] Starting request: urn:request:fe734e62-50b0-44af-82e1-1bda95bcb46c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:54,454 [554] [DEBUG] [app] Starting request: urn:request:fe734e62-50b0-44af-82e1-1bda95bcb46c (/v1/_internal_ping) 2020-03-31 09:23:54,455 [554] [DEBUG] [app] Ending request: urn:request:fe734e62-50b0-44af-82e1-1bda95bcb46c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:54,455 [554] [DEBUG] [app] Ending request: urn:request:fe734e62-50b0-44af-82e1-1bda95bcb46c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:23:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:23:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:23:54,457 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:23:54,457 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:54,459 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:23:54,459 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:54,461 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:54,461 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:54,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:54,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:54,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:54,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:54,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:54,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:54,483 [518] [DEBUG] [app] Starting request: urn:request:bfdc5eaf-4aaf-4592-b163-65e683ec42bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:54,483 [518] [DEBUG] [app] Starting request: urn:request:bfdc5eaf-4aaf-4592-b163-65e683ec42bc (/_internal_ping) 2020-03-31 09:23:54,484 [518] [DEBUG] [app] Ending request: urn:request:bfdc5eaf-4aaf-4592-b163-65e683ec42bc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:54,484 [518] [DEBUG] [app] Ending request: urn:request:bfdc5eaf-4aaf-4592-b163-65e683ec42bc (/_internal_ping) 2020-03-31 09:23:54,484 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:54,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:23:54,484 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:54,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:23:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:54,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:23:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:54,496 [534] [DEBUG] [app] Starting request: urn:request:bd13c7d3-bc96-4e0a-8ed9-e7b3c15155ba (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:54,496 [534] [DEBUG] [app] Starting request: urn:request:bd13c7d3-bc96-4e0a-8ed9-e7b3c15155ba (/c1/_internal_ping) 2020-03-31 09:23:54,497 [534] [DEBUG] [app] Ending request: urn:request:bd13c7d3-bc96-4e0a-8ed9-e7b3c15155ba (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:54,497 [534] [DEBUG] [app] Ending request: urn:request:bd13c7d3-bc96-4e0a-8ed9-e7b3c15155ba (/c1/_internal_ping) 2020-03-31 09:23:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:54,497 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:23:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:23:54,497 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:54,499 [518] [DEBUG] [app] Ending request: urn:request:8b4e94c9-07d1-405a-a0c3-37d3644aa696 (/health/instance) 2020-03-31 09:23:54,500 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:54,499 [518] [DEBUG] [app] Ending request: urn:request:8b4e94c9-07d1-405a-a0c3-37d3644aa696 (/health/instance) 2020-03-31 09:23:54,500 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:54,500 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:23:54,500 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:23:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:23:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:06.924497+00:00 (in 9.999208 seconds) 2020-03-31 09:23:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:06 UTC)" (scheduled at 2020-03-31 09:23:56.924497+00:00) 2020-03-31 09:23:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:23:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:06.924497+00:00 (in 9.999208 seconds) 2020-03-31 09:23:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:06 UTC)" (scheduled at 2020-03-31 09:23:56.924497+00:00) 2020-03-31 09:23:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:23:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 56, 926075), True, datetime.datetime(2020, 3, 31, 9, 23, 56, 926075), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:23:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 23, 56, 926075), True, datetime.datetime(2020, 3, 31, 9, 23, 56, 926075), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:23:56,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:23:56,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:23:56,944 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:23:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:23:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:23:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:06 UTC)" executed successfully 2020-03-31 09:23:58,273 [518] [DEBUG] [app] Starting request: urn:request:f2dbd308-49d7-49d8-8653-acdbcc7c2dbc (/health/instance) gunicorn-web stdout | 2020-03-31 09:23:58,273 [518] [DEBUG] [app] Starting request: urn:request:f2dbd308-49d7-49d8-8653-acdbcc7c2dbc (/health/instance) 2020-03-31 09:23:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:58,278 [538] [DEBUG] [app] Starting request: urn:request:5859b411-2485-4e3b-975c-1dec2f434e54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:58,278 [538] [DEBUG] [app] Starting request: urn:request:5859b411-2485-4e3b-975c-1dec2f434e54 (/v1/_internal_ping) 2020-03-31 09:23:58,279 [538] [DEBUG] [app] Ending request: urn:request:5859b411-2485-4e3b-975c-1dec2f434e54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:23:58,279 [538] [DEBUG] [app] Ending request: urn:request:5859b411-2485-4e3b-975c-1dec2f434e54 (/v1/_internal_ping) 2020-03-31 09:23:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:23:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:23:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:23:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:23:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:23:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:23:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:23:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:23:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:23:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:23:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:23:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:23:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:23:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:23:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:58,310 [518] [DEBUG] [app] Starting request: urn:request:5edd7550-2ef9-4699-b087-230dbe726cdf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:58,310 [518] [DEBUG] [app] Starting request: urn:request:5edd7550-2ef9-4699-b087-230dbe726cdf (/_internal_ping) 2020-03-31 09:23:58,310 [518] [DEBUG] [app] Ending request: urn:request:5edd7550-2ef9-4699-b087-230dbe726cdf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:58,310 [518] [DEBUG] [app] Ending request: urn:request:5edd7550-2ef9-4699-b087-230dbe726cdf (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:23:58,311 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:58,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:23:58,311 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:58,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:23:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:23:58,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:23:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:23:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:23:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:23:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:23:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:23:58,323 [534] [DEBUG] [app] Starting request: urn:request:5ba4c905-e2b5-4d76-8b77-95e16b45266b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:23:58,323 [534] [DEBUG] [app] Starting request: urn:request:5ba4c905-e2b5-4d76-8b77-95e16b45266b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:58,324 [534] [DEBUG] [app] Ending request: urn:request:5ba4c905-e2b5-4d76-8b77-95e16b45266b (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:23:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:58,324 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:23:58,324 [534] [DEBUG] [app] Ending request: urn:request:5ba4c905-e2b5-4d76-8b77-95e16b45266b (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:23:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:23:58,326 [518] [DEBUG] [app] Ending request: urn:request:f2dbd308-49d7-49d8-8653-acdbcc7c2dbc (/health/instance) 2020-03-31 09:23:58,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:23:58,324 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:23:58,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:23:58,326 [518] [DEBUG] [app] Ending request: urn:request:f2dbd308-49d7-49d8-8653-acdbcc7c2dbc (/health/instance) 2020-03-31 09:23:58,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:23:58,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:23:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:23:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:23:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:23:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:23:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:51.110610+00:00 (in 51.548504 seconds) 2020-03-31 09:23:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:59 UTC)" (scheduled at 2020-03-31 09:23:59.560326+00:00) 2020-03-31 09:23:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:23:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:23:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:51.110610+00:00 (in 51.548504 seconds) 2020-03-31 09:23:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:59 UTC)" (scheduled at 2020-03-31 09:23:59.560326+00:00) 2020-03-31 09:23:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:23:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:24:59 UTC)" executed successfully 2020-03-31 09:24:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:24:00,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:30.337699+00:00 (in 29.997677 seconds) 2020-03-31 09:24:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:30 UTC)" (scheduled at 2020-03-31 09:24:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:24:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:30.337699+00:00 (in 29.997677 seconds) 2020-03-31 09:24:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:30 UTC)" (scheduled at 2020-03-31 09:24:00.337699+00:00) 2020-03-31 09:24:00,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 24, 0, 340863), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:24:00,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 24, 0, 340863), False, 50, 1, 0]) 2020-03-31 09:24:00,360 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:24:00,360 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:24:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:24:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:30 UTC)" executed successfully 2020-03-31 09:24:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:24:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:11.440707+00:00 (in 9.999251 seconds) 2020-03-31 09:24:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:11 UTC)" (scheduled at 2020-03-31 09:24:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:24:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:11.440707+00:00 (in 9.999251 seconds) 2020-03-31 09:24:01,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:11 UTC)" (scheduled at 2020-03-31 09:24:01.440707+00:00) 2020-03-31 09:24:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:24:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:24:01,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:24:01,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:24:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:24:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:11 UTC)" executed successfully 2020-03-31 09:24:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:24:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:23.410371+00:00 (in 20.079135 seconds) 2020-03-31 09:24:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:33 UTC)" (scheduled at 2020-03-31 09:24:03.330112+00:00) 2020-03-31 09:24:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:24:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:23.410371+00:00 (in 20.079135 seconds) 2020-03-31 09:24:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:33 UTC)" (scheduled at 2020-03-31 09:24:03.330112+00:00) 2020-03-31 09:24:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 3, 332169), True, datetime.datetime(2020, 3, 31, 9, 24, 3, 332169), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:24:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 3, 332169), True, datetime.datetime(2020, 3, 31, 9, 24, 3, 332169), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:24:03,366 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:24:03,366 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:03,366 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:03,366 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:24:03,366 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:03,366 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:33 UTC)" executed successfully 2020-03-31 09:24:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:24:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:14 UTC)" (scheduled at 2020-03-31 09:24:04.323352+00:00) 2020-03-31 09:24:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:14.323352+00:00 (in 9.999116 seconds) notificationworker stdout | 2020-03-31 09:24:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:14 UTC)" (scheduled at 2020-03-31 09:24:04.323352+00:00) 2020-03-31 09:24:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:14.323352+00:00 (in 9.999116 seconds) 2020-03-31 09:24:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 4, 324969), True, datetime.datetime(2020, 3, 31, 9, 24, 4, 324969), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:24:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 4, 324969), True, datetime.datetime(2020, 3, 31, 9, 24, 4, 324969), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:24:04,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:24:04,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:24:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:14 UTC)" executed successfully 2020-03-31 09:24:04,446 [527] [DEBUG] [app] Starting request: urn:request:bd2b529b-2b16-4262-96ce-fdf957a2d396 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:04,446 [527] [DEBUG] [app] Starting request: urn:request:bd2b529b-2b16-4262-96ce-fdf957a2d396 (/health/instance) 2020-03-31 09:24:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:04,453 [552] [DEBUG] [app] Starting request: urn:request:ebdd60e6-b6ef-41c9-839e-33bebbf48092 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:04,453 [552] [DEBUG] [app] Starting request: urn:request:ebdd60e6-b6ef-41c9-839e-33bebbf48092 (/v1/_internal_ping) 2020-03-31 09:24:04,454 [552] [DEBUG] [app] Ending request: urn:request:ebdd60e6-b6ef-41c9-839e-33bebbf48092 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:04,454 [552] [DEBUG] [app] Ending request: urn:request:ebdd60e6-b6ef-41c9-839e-33bebbf48092 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:24:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:04,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:04,457 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:04,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:04,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:04,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:04,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:04,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:04,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:04,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:04,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:04,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:04,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:04,488 [527] [DEBUG] [app] Starting request: urn:request:7e057b7d-86f7-4e55-9038-5d3afe913697 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:04,488 [527] [DEBUG] [app] Starting request: urn:request:7e057b7d-86f7-4e55-9038-5d3afe913697 (/_internal_ping) 2020-03-31 09:24:04,489 [527] [DEBUG] [app] Ending request: urn:request:7e057b7d-86f7-4e55-9038-5d3afe913697 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:04,489 [527] [DEBUG] [app] Ending request: urn:request:7e057b7d-86f7-4e55-9038-5d3afe913697 (/_internal_ping) 2020-03-31 09:24:04,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:04,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:24:04,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:04,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:24:04,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:04,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:04,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:04,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:24:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:04,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:04,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:04,505 [539] [DEBUG] [app] Starting request: urn:request:5aac643c-0c7f-4f84-9839-ddaf946acfc8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:04,505 [539] [DEBUG] [app] Starting request: urn:request:5aac643c-0c7f-4f84-9839-ddaf946acfc8 (/c1/_internal_ping) 2020-03-31 09:24:04,506 [539] [DEBUG] [app] Ending request: urn:request:5aac643c-0c7f-4f84-9839-ddaf946acfc8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:04,506 [539] [DEBUG] [app] Ending request: urn:request:5aac643c-0c7f-4f84-9839-ddaf946acfc8 (/c1/_internal_ping) 2020-03-31 09:24:04,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:04,506 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:04,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:04,508 [527] [DEBUG] [app] Ending request: urn:request:bd2b529b-2b16-4262-96ce-fdf957a2d396 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:24:04,506 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:04,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:04,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:24:04,508 [527] [DEBUG] [app] Ending request: urn:request:bd2b529b-2b16-4262-96ce-fdf957a2d396 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:04,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:04,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:24:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:24:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:16.924497+00:00 (in 9.999169 seconds) 2020-03-31 09:24:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:16 UTC)" (scheduled at 2020-03-31 09:24:06.924497+00:00) 2020-03-31 09:24:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:24:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:16.924497+00:00 (in 9.999169 seconds) 2020-03-31 09:24:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:16 UTC)" (scheduled at 2020-03-31 09:24:06.924497+00:00) 2020-03-31 09:24:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 6, 926332), True, datetime.datetime(2020, 3, 31, 9, 24, 6, 926332), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:24:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 6, 926332), True, datetime.datetime(2020, 3, 31, 9, 24, 6, 926332), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:24:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:24:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:24:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:16 UTC)" executed successfully 2020-03-31 09:24:08,273 [527] [DEBUG] [app] Starting request: urn:request:2b55d879-bf2e-4191-bbc3-4ff70a30724b (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:08,273 [527] [DEBUG] [app] Starting request: urn:request:2b55d879-bf2e-4191-bbc3-4ff70a30724b (/health/instance) 2020-03-31 09:24:08,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:08,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:08,278 [538] [DEBUG] [app] Starting request: urn:request:a45ea5c3-1bdb-4136-835e-7a0b1488a50d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:08,278 [538] [DEBUG] [app] Starting request: urn:request:a45ea5c3-1bdb-4136-835e-7a0b1488a50d (/v1/_internal_ping) 2020-03-31 09:24:08,279 [538] [DEBUG] [app] Ending request: urn:request:a45ea5c3-1bdb-4136-835e-7a0b1488a50d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:08,279 [538] [DEBUG] [app] Ending request: urn:request:a45ea5c3-1bdb-4136-835e-7a0b1488a50d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:08,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:08,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:24:08,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:08,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:08,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:24:08,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:24:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:08,300 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:08,300 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:08,320 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:08,320 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:08,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:08,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:08,325 [526] [DEBUG] [app] Starting request: urn:request:30411bae-2cb0-4a2b-aed3-d94a25c72805 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:08,325 [526] [DEBUG] [app] Starting request: urn:request:30411bae-2cb0-4a2b-aed3-d94a25c72805 (/_internal_ping) 2020-03-31 09:24:08,326 [526] [DEBUG] [app] Ending request: urn:request:30411bae-2cb0-4a2b-aed3-d94a25c72805 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:08,326 [526] [DEBUG] [app] Ending request: urn:request:30411bae-2cb0-4a2b-aed3-d94a25c72805 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:24:08,327 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:08,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:08,327 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:08,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:08,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:08,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:08,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:08,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:08,334 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:08,334 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:08,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:24:08,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:24:08,338 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:08,338 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:08,340 [532] [DEBUG] [app] Starting request: urn:request:0dfe07b5-1a06-40f2-aca9-afb04f33f041 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:08,340 [532] [DEBUG] [app] Starting request: urn:request:0dfe07b5-1a06-40f2-aca9-afb04f33f041 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:08,341 [532] [DEBUG] [app] Ending request: urn:request:0dfe07b5-1a06-40f2-aca9-afb04f33f041 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:08,341 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:08,341 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:24:08,341 [532] [DEBUG] [app] Ending request: urn:request:0dfe07b5-1a06-40f2-aca9-afb04f33f041 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:08,341 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:08,343 [527] [DEBUG] [app] Ending request: urn:request:2b55d879-bf2e-4191-bbc3-4ff70a30724b (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:24:08,341 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:08,343 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:08,343 [527] [DEBUG] [app] Ending request: urn:request:2b55d879-bf2e-4191-bbc3-4ff70a30724b (/health/instance) 2020-03-31 09:24:08,344 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:24:08,343 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:08,344 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:24:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:24:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:21.440707+00:00 (in 9.999200 seconds) 2020-03-31 09:24:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:21 UTC)" (scheduled at 2020-03-31 09:24:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:24:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:21.440707+00:00 (in 9.999200 seconds) 2020-03-31 09:24:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:21 UTC)" (scheduled at 2020-03-31 09:24:11.440707+00:00) 2020-03-31 09:24:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:24:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:24:11,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:24:11,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:24:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:24:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:21 UTC)" executed successfully 2020-03-31 09:24:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:24:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:24.323352+00:00 (in 9.998323 seconds) 2020-03-31 09:24:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:24 UTC)" (scheduled at 2020-03-31 09:24:14.323352+00:00) 2020-03-31 09:24:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:24:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:24.323352+00:00 (in 9.998323 seconds) 2020-03-31 09:24:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:24 UTC)" (scheduled at 2020-03-31 09:24:14.323352+00:00) 2020-03-31 09:24:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 14, 326009), True, datetime.datetime(2020, 3, 31, 9, 24, 14, 326009), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:24:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 14, 326009), True, datetime.datetime(2020, 3, 31, 9, 24, 14, 326009), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:24:14,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:24:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:24:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:24 UTC)" executed successfully 2020-03-31 09:24:14,446 [518] [DEBUG] [app] Starting request: urn:request:5b83f067-d574-4cec-9dc8-897912a262d2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:14,446 [518] [DEBUG] [app] Starting request: urn:request:5b83f067-d574-4cec-9dc8-897912a262d2 (/health/instance) 2020-03-31 09:24:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:14,454 [553] [DEBUG] [app] Starting request: urn:request:b9a0a049-3944-4550-b4f0-74edcf32fdb7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:14,454 [553] [DEBUG] [app] Starting request: urn:request:b9a0a049-3944-4550-b4f0-74edcf32fdb7 (/v1/_internal_ping) 2020-03-31 09:24:14,456 [553] [DEBUG] [app] Ending request: urn:request:b9a0a049-3944-4550-b4f0-74edcf32fdb7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:14,456 [553] [DEBUG] [app] Ending request: urn:request:b9a0a049-3944-4550-b4f0-74edcf32fdb7 (/v1/_internal_ping) 2020-03-31 09:24:14,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:24:14,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:14,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:24:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:24:14,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:14,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:14,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:14,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:14,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:14,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:14,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:14,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:14,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:14,487 [518] [DEBUG] [app] Starting request: urn:request:8decb71f-3113-4abf-8937-c624c7207fc7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:14,487 [518] [DEBUG] [app] Starting request: urn:request:8decb71f-3113-4abf-8937-c624c7207fc7 (/_internal_ping) 2020-03-31 09:24:14,488 [518] [DEBUG] [app] Ending request: urn:request:8decb71f-3113-4abf-8937-c624c7207fc7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:14,488 [518] [DEBUG] [app] Ending request: urn:request:8decb71f-3113-4abf-8937-c624c7207fc7 (/_internal_ping) 2020-03-31 09:24:14,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:14,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:14,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:14,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:14,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:14,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:14,501 [539] [DEBUG] [app] Starting request: urn:request:1ef91e53-39a2-471d-beff-86bf517743d3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:14,501 [539] [DEBUG] [app] Starting request: urn:request:1ef91e53-39a2-471d-beff-86bf517743d3 (/c1/_internal_ping) 2020-03-31 09:24:14,502 [539] [DEBUG] [app] Ending request: urn:request:1ef91e53-39a2-471d-beff-86bf517743d3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:14,502 [539] [DEBUG] [app] Ending request: urn:request:1ef91e53-39a2-471d-beff-86bf517743d3 (/c1/_internal_ping) 2020-03-31 09:24:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:14,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:24:14,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:14,505 [518] [DEBUG] [app] Ending request: urn:request:5b83f067-d574-4cec-9dc8-897912a262d2 (/health/instance) 2020-03-31 09:24:14,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:14,505 [518] [DEBUG] [app] Ending request: urn:request:5b83f067-d574-4cec-9dc8-897912a262d2 (/health/instance) 2020-03-31 09:24:14,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:24:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:24:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:24:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:45.064514+00:00 (in 29.999251 seconds) 2020-03-31 09:24:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:45 UTC)" (scheduled at 2020-03-31 09:24:15.064514+00:00) gcworker stdout | 2020-03-31 09:24:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:45.064514+00:00 (in 29.999251 seconds) 2020-03-31 09:24:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:45 UTC)" (scheduled at 2020-03-31 09:24:15.064514+00:00) 2020-03-31 09:24:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437055, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:24:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437055, 1209600, 500, 1, 0]) 2020-03-31 09:24:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:24:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:24:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:24:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:45 UTC)" executed successfully 2020-03-31 09:24:16,929 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:24:16,929 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:16,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:26.924497+00:00 (in 9.994861 seconds) 2020-03-31 09:24:16,930 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:26 UTC)" (scheduled at 2020-03-31 09:24:16.924497+00:00) 2020-03-31 09:24:16,930 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:24:16,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:26.924497+00:00 (in 9.994861 seconds) 2020-03-31 09:24:16,930 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:26 UTC)" (scheduled at 2020-03-31 09:24:16.924497+00:00) 2020-03-31 09:24:16,930 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:16,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 16, 930392), True, datetime.datetime(2020, 3, 31, 9, 24, 16, 930392), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:24:16,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 16, 930392), True, datetime.datetime(2020, 3, 31, 9, 24, 16, 930392), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:24:16,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:24:16,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:16,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:16,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:24:16,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:16,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:26 UTC)" executed successfully 2020-03-31 09:24:18,273 [518] [DEBUG] [app] Starting request: urn:request:06e21492-326e-4c3b-becc-06e9a3492a8e (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:18,273 [518] [DEBUG] [app] Starting request: urn:request:06e21492-326e-4c3b-becc-06e9a3492a8e (/health/instance) 2020-03-31 09:24:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:18,278 [538] [DEBUG] [app] Starting request: urn:request:8e10adc3-bcf7-450e-803e-63fb7da2ad2e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:18,278 [538] [DEBUG] [app] Starting request: urn:request:8e10adc3-bcf7-450e-803e-63fb7da2ad2e (/v1/_internal_ping) 2020-03-31 09:24:18,279 [538] [DEBUG] [app] Ending request: urn:request:8e10adc3-bcf7-450e-803e-63fb7da2ad2e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:18,279 [538] [DEBUG] [app] Ending request: urn:request:8e10adc3-bcf7-450e-803e-63fb7da2ad2e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:18,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:24:18,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:18,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:18,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:18,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:18,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:18,310 [518] [DEBUG] [app] Starting request: urn:request:b19fac54-06d4-4ee6-a145-c33b41f9eb43 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:18,310 [518] [DEBUG] [app] Starting request: urn:request:b19fac54-06d4-4ee6-a145-c33b41f9eb43 (/_internal_ping) 2020-03-31 09:24:18,311 [518] [DEBUG] [app] Ending request: urn:request:b19fac54-06d4-4ee6-a145-c33b41f9eb43 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:18,311 [518] [DEBUG] [app] Ending request: urn:request:b19fac54-06d4-4ee6-a145-c33b41f9eb43 (/_internal_ping) 2020-03-31 09:24:18,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:18,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:24:18,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:18,324 [539] [DEBUG] [app] Starting request: urn:request:2ee755c5-c042-4070-b02f-24ac17046c8e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:18,324 [539] [DEBUG] [app] Starting request: urn:request:2ee755c5-c042-4070-b02f-24ac17046c8e (/c1/_internal_ping) 2020-03-31 09:24:18,324 [539] [DEBUG] [app] Ending request: urn:request:2ee755c5-c042-4070-b02f-24ac17046c8e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:18,324 [539] [DEBUG] [app] Ending request: urn:request:2ee755c5-c042-4070-b02f-24ac17046c8e (/c1/_internal_ping) 2020-03-31 09:24:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:18,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:18,327 [518] [DEBUG] [app] Ending request: urn:request:06e21492-326e-4c3b-becc-06e9a3492a8e (/health/instance) 2020-03-31 09:24:18,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:24:18,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:24:18,327 [518] [DEBUG] [app] Ending request: urn:request:06e21492-326e-4c3b-becc-06e9a3492a8e (/health/instance) 2020-03-31 09:24:18,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:24:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:24:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:31 UTC)" (scheduled at 2020-03-31 09:24:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:24:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:31 UTC)" (scheduled at 2020-03-31 09:24:21.440707+00:00) 2020-03-31 09:24:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:31.440707+00:00 (in 9.998952 seconds) repositoryactioncounter stdout | 2020-03-31 09:24:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:31.440707+00:00 (in 9.998952 seconds) 2020-03-31 09:24:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:24:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:24:21,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:24:21,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:24:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:24:21,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:21,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:31 UTC)" executed successfully 2020-03-31 09:24:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:24:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:52.113221+00:00 (in 29.998956 seconds) 2020-03-31 09:24:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:52 UTC)" (scheduled at 2020-03-31 09:24:22.113221+00:00) securityworker stdout | 2020-03-31 09:24:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:52.113221+00:00 (in 29.998956 seconds) 2020-03-31 09:24:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:52 UTC)" (scheduled at 2020-03-31 09:24:22.113221+00:00) 2020-03-31 09:24:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:24:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:24:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:24:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:24:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:24:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:24:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:24:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:24:52 UTC)" executed successfully 2020-03-31 09:24:23,410 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:24:23,410 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:23 UTC)" (scheduled at 2020-03-31 09:24:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:24:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:23 UTC)" (scheduled at 2020-03-31 09:24:23.410371+00:00) 2020-03-31 09:24:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:24.410927+00:00 (in 0.999802 seconds) 2020-03-31 09:24:23,412 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:24:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:24.410927+00:00 (in 0.999802 seconds) 2020-03-31 09:24:23,412 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), u'security_notification/%']) 2020-03-31 09:24:23,427 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), True, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:24:23,427 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), True, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), 0, u'security_notification/%']) 2020-03-31 09:24:23,431 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), True, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:24:23,431 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), True, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 24, 23, 411827), u'security_notification/%']) 2020-03-31 09:24:23,434 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:24:23,434 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:23,436 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:24:23,436 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:23 UTC)" executed successfully 2020-03-31 09:24:23,438 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:24:23,438 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:23,439 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:24:23,439 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:23,439 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:24:23,439 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:24:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:24:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:34.323352+00:00 (in 9.999121 seconds) 2020-03-31 09:24:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:34 UTC)" (scheduled at 2020-03-31 09:24:24.323352+00:00) 2020-03-31 09:24:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:24:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:34.323352+00:00 (in 9.999121 seconds) 2020-03-31 09:24:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:34 UTC)" (scheduled at 2020-03-31 09:24:24.323352+00:00) 2020-03-31 09:24:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 24, 324932), True, datetime.datetime(2020, 3, 31, 9, 24, 24, 324932), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:24:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 24, 324932), True, datetime.datetime(2020, 3, 31, 9, 24, 24, 324932), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:24:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:24:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:24:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:34 UTC)" executed successfully 2020-03-31 09:24:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:24:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:24,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:33.330112+00:00 (in 8.916984 seconds) 2020-03-31 09:24:24,413 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:24 UTC)" (scheduled at 2020-03-31 09:24:24.410927+00:00) 2020-03-31 09:24:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:24:24,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:33.330112+00:00 (in 8.916984 seconds) 2020-03-31 09:24:24,413 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:24 UTC)" (scheduled at 2020-03-31 09:24:24.410927+00:00) 2020-03-31 09:24:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:24 UTC)" executed successfully 2020-03-31 09:24:24,446 [527] [DEBUG] [app] Starting request: urn:request:27c108a6-43b2-4294-9d22-7fcfd567588e (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:24,446 [527] [DEBUG] [app] Starting request: urn:request:27c108a6-43b2-4294-9d22-7fcfd567588e (/health/instance) 2020-03-31 09:24:24,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:24,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:24,453 [538] [DEBUG] [app] Starting request: urn:request:6483dc59-c327-499c-ac1c-c43e3587ff65 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:24,453 [538] [DEBUG] [app] Starting request: urn:request:6483dc59-c327-499c-ac1c-c43e3587ff65 (/v1/_internal_ping) 2020-03-31 09:24:24,453 [538] [DEBUG] [app] Ending request: urn:request:6483dc59-c327-499c-ac1c-c43e3587ff65 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:24,453 [538] [DEBUG] [app] Ending request: urn:request:6483dc59-c327-499c-ac1c-c43e3587ff65 (/v1/_internal_ping) 2020-03-31 09:24:24,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:24:24,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:24,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:24,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:24,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:24,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:24,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:24,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:24,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:24,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:24,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:24,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:24,485 [527] [DEBUG] [app] Starting request: urn:request:c2632e4a-bdc7-47e6-804a-a1cdb417da6d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:24,485 [527] [DEBUG] [app] Starting request: urn:request:c2632e4a-bdc7-47e6-804a-a1cdb417da6d (/_internal_ping) 2020-03-31 09:24:24,486 [527] [DEBUG] [app] Ending request: urn:request:c2632e4a-bdc7-47e6-804a-a1cdb417da6d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:24,486 [527] [DEBUG] [app] Ending request: urn:request:c2632e4a-bdc7-47e6-804a-a1cdb417da6d (/_internal_ping) 2020-03-31 09:24:24,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:24,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:24,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:24,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:24,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:24,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:24,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:24,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:24,499 [535] [DEBUG] [app] Starting request: urn:request:1f75b2ee-61ac-4db0-b274-5a155820f343 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:24,499 [535] [DEBUG] [app] Starting request: urn:request:1f75b2ee-61ac-4db0-b274-5a155820f343 (/c1/_internal_ping) 2020-03-31 09:24:24,500 [535] [DEBUG] [app] Ending request: urn:request:1f75b2ee-61ac-4db0-b274-5a155820f343 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:24,500 [535] [DEBUG] [app] Ending request: urn:request:1f75b2ee-61ac-4db0-b274-5a155820f343 (/c1/_internal_ping) 2020-03-31 09:24:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:24,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:24,502 [527] [DEBUG] [app] Ending request: urn:request:27c108a6-43b2-4294-9d22-7fcfd567588e (/health/instance) 2020-03-31 09:24:24,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:24:24,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:24,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:24:24,502 [527] [DEBUG] [app] Ending request: urn:request:27c108a6-43b2-4294-9d22-7fcfd567588e (/health/instance) 2020-03-31 09:24:24,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:24,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:24:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:24:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:36.924497+00:00 (in 9.999174 seconds) 2020-03-31 09:24:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:36 UTC)" (scheduled at 2020-03-31 09:24:26.924497+00:00) 2020-03-31 09:24:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:24:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:36.924497+00:00 (in 9.999174 seconds) 2020-03-31 09:24:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:36 UTC)" (scheduled at 2020-03-31 09:24:26.924497+00:00) 2020-03-31 09:24:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 26, 926298), True, datetime.datetime(2020, 3, 31, 9, 24, 26, 926298), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:24:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 26, 926298), True, datetime.datetime(2020, 3, 31, 9, 24, 26, 926298), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:24:26,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:24:26,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:24:26,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:26,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:36 UTC)" executed successfully 2020-03-31 09:24:28,273 [526] [DEBUG] [app] Starting request: urn:request:0d23bb14-e295-4c54-b9ae-84862e8903e5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:28,273 [526] [DEBUG] [app] Starting request: urn:request:0d23bb14-e295-4c54-b9ae-84862e8903e5 (/health/instance) 2020-03-31 09:24:28,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:28,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:28,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:28,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:28,282 [538] [DEBUG] [app] Starting request: urn:request:e908f4e9-bf60-4ee0-bdb8-88d479440cfb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:28,282 [538] [DEBUG] [app] Starting request: urn:request:e908f4e9-bf60-4ee0-bdb8-88d479440cfb (/v1/_internal_ping) 2020-03-31 09:24:28,283 [538] [DEBUG] [app] Ending request: urn:request:e908f4e9-bf60-4ee0-bdb8-88d479440cfb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:28,283 [538] [DEBUG] [app] Ending request: urn:request:e908f4e9-bf60-4ee0-bdb8-88d479440cfb (/v1/_internal_ping) 2020-03-31 09:24:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-registry stdout | 2020-03-31 09:24:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:28,285 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:24:28,286 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:28,285 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:24:28,286 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:24:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:28,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:28,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:28,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:28,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:28,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:28,315 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:28,317 [523] [DEBUG] [app] Starting request: urn:request:a7ec06c5-9068-4992-95c1-de97257ab025 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:28,317 [523] [DEBUG] [app] Starting request: urn:request:a7ec06c5-9068-4992-95c1-de97257ab025 (/_internal_ping) 2020-03-31 09:24:28,318 [523] [DEBUG] [app] Ending request: urn:request:a7ec06c5-9068-4992-95c1-de97257ab025 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:28,318 [523] [DEBUG] [app] Ending request: urn:request:a7ec06c5-9068-4992-95c1-de97257ab025 (/_internal_ping) 2020-03-31 09:24:28,319 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:28,319 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:28,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:28,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:28,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:28,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:28,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:24:28,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:24:28,333 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:28,333 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:28,335 [543] [DEBUG] [app] Starting request: urn:request:93e30587-d0bb-475e-82fb-8f356f81f31c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:28,335 [543] [DEBUG] [app] Starting request: urn:request:93e30587-d0bb-475e-82fb-8f356f81f31c (/c1/_internal_ping) 2020-03-31 09:24:28,336 [543] [DEBUG] [app] Ending request: urn:request:93e30587-d0bb-475e-82fb-8f356f81f31c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:28,336 [543] [DEBUG] [app] Ending request: urn:request:93e30587-d0bb-475e-82fb-8f356f81f31c (/c1/_internal_ping) 2020-03-31 09:24:28,337 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:28,336 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:28,337 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:24:28,336 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:28,338 [526] [DEBUG] [app] Ending request: urn:request:0d23bb14-e295-4c54-b9ae-84862e8903e5 (/health/instance) 2020-03-31 09:24:28,338 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:28,338 [526] [DEBUG] [app] Ending request: urn:request:0d23bb14-e295-4c54-b9ae-84862e8903e5 (/health/instance) 2020-03-31 09:24:28,338 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:28,339 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:24:28,339 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:24:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:24:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:00.337699+00:00 (in 29.998912 seconds) 2020-03-31 09:24:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:00 UTC)" (scheduled at 2020-03-31 09:24:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:24:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:00.337699+00:00 (in 29.998912 seconds) 2020-03-31 09:24:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:00 UTC)" (scheduled at 2020-03-31 09:24:30.337699+00:00) 2020-03-31 09:24:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 24, 30, 339659), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:24:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 24, 30, 339659), False, 50, 1, 0]) 2020-03-31 09:24:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:24:30,356 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:24:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:24:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:24:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:00 UTC)" executed successfully 2020-03-31 09:24:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:24:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:31,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:41.440707+00:00 (in 9.995755 seconds) 2020-03-31 09:24:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:41 UTC)" (scheduled at 2020-03-31 09:24:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:24:31,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:41.440707+00:00 (in 9.995755 seconds) 2020-03-31 09:24:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:41 UTC)" (scheduled at 2020-03-31 09:24:31.440707+00:00) 2020-03-31 09:24:31,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:24:31,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:24:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:24:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:24:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:24:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:41 UTC)" executed successfully 2020-03-31 09:24:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:24:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:03.330112+00:00 (in 29.999116 seconds) 2020-03-31 09:24:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:03 UTC)" (scheduled at 2020-03-31 09:24:33.330112+00:00) 2020-03-31 09:24:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:24:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:03.330112+00:00 (in 29.999116 seconds) 2020-03-31 09:24:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:03 UTC)" (scheduled at 2020-03-31 09:24:33.330112+00:00) 2020-03-31 09:24:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 33, 331949), True, datetime.datetime(2020, 3, 31, 9, 24, 33, 331949), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:24:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 33, 331949), True, datetime.datetime(2020, 3, 31, 9, 24, 33, 331949), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:24:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:24:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:24:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:03 UTC)" executed successfully 2020-03-31 09:24:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:24:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:43.539999+00:00 (in 10.171929 seconds) 2020-03-31 09:24:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:33 UTC)" (scheduled at 2020-03-31 09:24:33.366791+00:00) 2020-03-31 09:24:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:24:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:43.539999+00:00 (in 10.171929 seconds) 2020-03-31 09:24:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:33 UTC)" (scheduled at 2020-03-31 09:24:33.366791+00:00) 2020-03-31 09:24:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 33, 368791), True, datetime.datetime(2020, 3, 31, 9, 24, 33, 368791), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:24:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 33, 368791), True, datetime.datetime(2020, 3, 31, 9, 24, 33, 368791), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:24:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:24:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:24:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:33 UTC)" executed successfully 2020-03-31 09:24:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:24:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:44.323352+00:00 (in 9.999052 seconds) 2020-03-31 09:24:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:44 UTC)" (scheduled at 2020-03-31 09:24:34.323352+00:00) 2020-03-31 09:24:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:24:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:44.323352+00:00 (in 9.999052 seconds) 2020-03-31 09:24:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:44 UTC)" (scheduled at 2020-03-31 09:24:34.323352+00:00) 2020-03-31 09:24:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 34, 325365), True, datetime.datetime(2020, 3, 31, 9, 24, 34, 325365), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:24:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 34, 325365), True, datetime.datetime(2020, 3, 31, 9, 24, 34, 325365), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:24:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:24:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:24:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:44 UTC)" executed successfully 2020-03-31 09:24:34,446 [527] [DEBUG] [app] Starting request: urn:request:dd064b68-2b25-4cf5-8b15-2f2e4ac9a6ec (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:34,446 [527] [DEBUG] [app] Starting request: urn:request:dd064b68-2b25-4cf5-8b15-2f2e4ac9a6ec (/health/instance) 2020-03-31 09:24:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:34,454 [556] [DEBUG] [app] Starting request: urn:request:95a039ca-01cb-4047-bc92-a8153aaa356d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:34,454 [556] [DEBUG] [app] Starting request: urn:request:95a039ca-01cb-4047-bc92-a8153aaa356d (/v1/_internal_ping) 2020-03-31 09:24:34,455 [556] [DEBUG] [app] Ending request: urn:request:95a039ca-01cb-4047-bc92-a8153aaa356d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:34,455 [556] [DEBUG] [app] Ending request: urn:request:95a039ca-01cb-4047-bc92-a8153aaa356d (/v1/_internal_ping) 2020-03-31 09:24:34,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:24:34,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:34,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:24:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:34,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:34,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:34,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:34,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:34,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:34,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:34,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:34,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:34,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:34,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:34,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:34,490 [527] [DEBUG] [app] Starting request: urn:request:19295588-0a73-45cf-9328-b33b10e3a2ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:34,490 [527] [DEBUG] [app] Starting request: urn:request:19295588-0a73-45cf-9328-b33b10e3a2ce (/_internal_ping) 2020-03-31 09:24:34,491 [527] [DEBUG] [app] Ending request: urn:request:19295588-0a73-45cf-9328-b33b10e3a2ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:34,491 [527] [DEBUG] [app] Ending request: urn:request:19295588-0a73-45cf-9328-b33b10e3a2ce (/_internal_ping) 2020-03-31 09:24:34,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:34,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:24:34,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:34,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:24:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:34,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:34,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:34,508 [539] [DEBUG] [app] Starting request: urn:request:16dfaaea-a0c6-4e52-b8d3-83e464409f50 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:34,508 [539] [DEBUG] [app] Starting request: urn:request:16dfaaea-a0c6-4e52-b8d3-83e464409f50 (/c1/_internal_ping) 2020-03-31 09:24:34,508 [539] [DEBUG] [app] Ending request: urn:request:16dfaaea-a0c6-4e52-b8d3-83e464409f50 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:34,508 [539] [DEBUG] [app] Ending request: urn:request:16dfaaea-a0c6-4e52-b8d3-83e464409f50 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:34,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:34,509 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:34,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:24:34,509 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:34,510 [527] [DEBUG] [app] Ending request: urn:request:dd064b68-2b25-4cf5-8b15-2f2e4ac9a6ec (/health/instance) 2020-03-31 09:24:34,511 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:34,510 [527] [DEBUG] [app] Ending request: urn:request:dd064b68-2b25-4cf5-8b15-2f2e4ac9a6ec (/health/instance) 2020-03-31 09:24:34,511 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:34,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:24:34,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:24:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:24:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:38.010934+00:00 (in 1.085254 seconds) 2020-03-31 09:24:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:46 UTC)" (scheduled at 2020-03-31 09:24:36.924497+00:00) 2020-03-31 09:24:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:24:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:38.010934+00:00 (in 1.085254 seconds) 2020-03-31 09:24:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:46 UTC)" (scheduled at 2020-03-31 09:24:36.924497+00:00) 2020-03-31 09:24:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 36, 926449), True, datetime.datetime(2020, 3, 31, 9, 24, 36, 926449), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:24:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 36, 926449), True, datetime.datetime(2020, 3, 31, 9, 24, 36, 926449), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:24:36,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:24:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:24:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:36,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:46 UTC)" executed successfully 2020-03-31 09:24:37,289 [556] [DEBUG] [app] Starting request: urn:request:f9d70dfc-3dc5-49c9-9c77-14ca9a0f912a (/v2/) gunicorn-registry stdout | 2020-03-31 09:24:37,289 [556] [DEBUG] [app] Starting request: urn:request:f9d70dfc-3dc5-49c9-9c77-14ca9a0f912a (/v2/) 2020-03-31 09:24:37,289 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {} 2020-03-31 09:24:37,289 [556] [DEBUG] [auth.registry_jwt_auth] No auth header. gunicorn-registry stdout | 2020-03-31 09:24:37,289 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {} 2020-03-31 09:24:37,289 [556] [DEBUG] [auth.registry_jwt_auth] No auth header. 2020-03-31 09:24:37,290 [556] [DEBUG] [app] Ending request: urn:request:f9d70dfc-3dc5-49c9-9c77-14ca9a0f912a (/v2/) gunicorn-registry stdout | 2020-03-31 09:24:37,290 [556] [DEBUG] [app] Ending request: urn:request:f9d70dfc-3dc5-49c9-9c77-14ca9a0f912a (/v2/) 2020-03-31 09:24:37,291 [556] [INFO] [gunicorn.access] 10.128.2.9 - - [31/Mar/2020:09:24:37 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "Go-http-client/2.0" 10.128.2.9 () - - [31/Mar/2020:09:24:37 +0000] "GET /v2/ HTTP/2.0" 401 4 "-" "Go-http-client/2.0" (0.003 63 0.003) gunicorn-registry stdout | 2020-03-31 09:24:37,291 [556] [INFO] [gunicorn.access] 10.128.2.9 - - [31/Mar/2020:09:24:37 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "Go-http-client/2.0" nginx stdout | 10.128.2.9 () - - [31/Mar/2020:09:24:37 +0000] "GET /v2/ HTTP/2.0" 401 4 "-" "Go-http-client/2.0" (0.003 63 0.003) 2020-03-31 09:24:37,293 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:37,293 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:37,294 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:37,294 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:37,295 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:37,295 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:24:37,602 [538] [DEBUG] [app] Starting request: urn:request:e72569e9-a616-4ef2-9b11-381ace9c69ab (/v2/auth) gunicorn-registry stdout | 2020-03-31 09:24:37,602 [538] [DEBUG] [app] Starting request: urn:request:e72569e9-a616-4ef2-9b11-381ace9c69ab (/v2/auth) 2020-03-31 09:24:37,603 [538] [DEBUG] [auth.basic] Attempt to process basic auth header gunicorn-registry stdout | 2020-03-31 09:24:37,603 [538] [DEBUG] [auth.basic] Attempt to process basic auth header 2020-03-31 09:24:37,609 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', [u'admin', u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:24:37,609 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', [u'admin', u'admin', 1, 0]) 2020-03-31 09:24:37,609 [538] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2020-03-31 09:24:37,609 [538] [DEBUG] [peewee.pool] No connection available in pool. 2020-03-31 09:24:37,623 [538] [DEBUG] [peewee.pool] Created new connection 140043679892496. gunicorn-registry stdout | 2020-03-31 09:24:37,623 [538] [DEBUG] [peewee.pool] Created new connection 140043679892496. 2020-03-31 09:24:37,955 [538] [DEBUG] [auth.credentials] Successfully validated credentials for user admin gunicorn-registry stdout | 2020-03-31 09:24:37,955 [538] [DEBUG] [auth.credentials] Successfully validated credentials for user admin 2020-03-31 09:24:37,956 [538] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [None, None, None, None, , None]) gunicorn-registry stdout | 2020-03-31 09:24:37,956 [538] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [None, None, None, None, , None]) 2020-03-31 09:24:37,956 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:24:37,956 [538] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:24:37,957 [538] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com gunicorn-registry stdout | 2020-03-31 09:24:37,956 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:24:37,956 [538] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:24:37,957 [538] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com 2020-03-31 09:24:37,957 [538] [DEBUG] [endpoints.v2.v2auth] Scope request: [u'repository:testorg/redhat-operators:pull,push'] gunicorn-registry stdout | 2020-03-31 09:24:37,957 [538] [DEBUG] [endpoints.v2.v2auth] Scope request: [u'repository:testorg/redhat-operators:pull,push'] 2020-03-31 09:24:37,958 [538] [DEBUG] [endpoints.v2.v2auth] Match: (u'testorg/redhat-operators', u'testorg/redhat-operators', u'pull,push') gunicorn-registry stdout | 2020-03-31 09:24:37,958 [538] [DEBUG] [endpoints.v2.v2auth] Match: (u'testorg/redhat-operators', u'testorg/redhat-operators', u'pull,push') 2020-03-31 09:24:37,960 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'testorg', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:24:37,960 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'testorg', 1, 0]) 2020-03-31 09:24:37,967 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'testorg', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:24:37,967 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'testorg', u'redhat-operators', 1, 0]) 2020-03-31 09:24:37,974 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac gunicorn-registry stdout | 2020-03-31 09:24:37,974 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username=u'admin', role='admin') 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace=u'admin', role='admin') 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace=u'admin', role='admin') 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] Adding superuser to user: admin gunicorn-registry stdout | 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username=u'admin', role='admin') 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace=u'admin', role='admin') 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace=u'admin', role='admin') 2020-03-31 09:24:37,975 [538] [DEBUG] [auth.permissions] Adding superuser to user: admin 2020-03-31 09:24:37,978 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) gunicorn-registry stdout | 2020-03-31 09:24:37,978 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."organization_id", "t1"."role_id", "t1"."description", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed", "t3"."id", "t3"."name" FROM "team" AS "t1" INNER JOIN "teamrole" AS "t3" ON ("t1"."role_id" = "t3"."id") INNER JOIN "user" AS "t2" ON ("t1"."organization_id" = "t2"."id") INNER JOIN "teammember" AS "t4" ON ("t4"."team_id" = "t1"."id") INNER JOIN "user" AS "t5" ON ("t4"."user_id" = "t5"."id") WHERE (("t5"."id" = %s) AND ("t2"."organization" = %s))', [1, True]) 2020-03-31 09:24:37,983 [538] [DEBUG] [endpoints.v2.v2auth] No permission to create repository testorg/redhat-operators gunicorn-registry stdout | 2020-03-31 09:24:37,983 [538] [DEBUG] [endpoints.v2.v2auth] No permission to create repository testorg/redhat-operators 2020-03-31 09:24:37,984 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac gunicorn-registry stdout | 2020-03-31 09:24:37,984 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:24:37,988 [538] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', [u'redhat-operators', u'testorg', 1, u'redhat-operators', u'testorg', 1]) gunicorn-registry stdout | 2020-03-31 09:24:37,988 [538] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."team_id", "t1"."user_id", "t1"."repository_id", "t1"."role_id", "t2"."id", "t2"."name", "t3"."id", "t3"."namespace_user_id", "t3"."name", "t3"."visibility_id", "t3"."description", "t3"."badge_token", "t3"."kind_id", "t3"."trust_enabled", "t3"."state", "t4"."id", "t4"."uuid", "t4"."username", "t4"."password_hash", "t4"."email", "t4"."verified", "t4"."stripe_id", "t4"."organization", "t4"."robot", "t4"."invoice_email", "t4"."invalid_login_attempts", "t4"."last_invalid_login", "t4"."removed_tag_expiration_s", "t4"."enabled", "t4"."invoice_email_address", "t4"."given_name", "t4"."family_name", "t4"."company", "t4"."location", "t4"."maximum_queued_builds_count", "t4"."creation_date", "t4"."last_accessed" FROM "repositorypermission" AS "t1" INNER JOIN "role" AS "t2" ON ("t1"."role_id" = "t2"."id") INNER JOIN "repository" AS "t3" ON ("t1"."repository_id" = "t3"."id") INNER JOIN "user" AS "t4" ON ("t3"."namespace_user_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t1"."user_id" = "t5"."id") WHERE ((("t3"."name" = %s) AND ("t4"."username" = %s)) AND ("t5"."id" = %s))) UNION (SELECT "t6"."id", "t6"."team_id", "t6"."user_id", "t6"."repository_id", "t6"."role_id", "t7"."id", "t7"."name", "t8"."id", "t8"."namespace_user_id", "t8"."name", "t8"."visibility_id", "t8"."description", "t8"."badge_token", "t8"."kind_id", "t8"."trust_enabled", "t8"."state", "t9"."id", "t9"."uuid", "t9"."username", "t9"."password_hash", "t9"."email", "t9"."verified", "t9"."stripe_id", "t9"."organization", "t9"."robot", "t9"."invoice_email", "t9"."invalid_login_attempts", "t9"."last_invalid_login", "t9"."removed_tag_expiration_s", "t9"."enabled", "t9"."invoice_email_address", "t9"."given_name", "t9"."family_name", "t9"."company", "t9"."location", "t9"."maximum_queued_builds_count", "t9"."creation_date", "t9"."last_accessed" FROM "repositorypermission" AS "t6" INNER JOIN "role" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "repository" AS "t8" ON ("t6"."repository_id" = "t8"."id") INNER JOIN "user" AS "t9" ON ("t8"."namespace_user_id" = "t9"."id") INNER JOIN "team" AS "t10" ON ("t6"."team_id" = "t10"."id") INNER JOIN "teammember" AS "t11" ON ("t11"."team_id" = "t10"."id") INNER JOIN "user" AS "t12" ON ("t12"."id" = "t11"."user_id") WHERE ((("t8"."name" = %s) AND ("t9"."username" = %s)) AND ("t12"."id" = %s)))', [u'redhat-operators', u'testorg', 1, u'redhat-operators', u'testorg', 1]) 2020-03-31 09:24:37,996 [538] [DEBUG] [endpoints.v2.v2auth] No permission to pull repository testorg/redhat-operators gunicorn-registry stdout | 2020-03-31 09:24:37,996 [538] [DEBUG] [endpoints.v2.v2auth] No permission to pull repository testorg/redhat-operators 2020-03-31 09:24:37,998 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:37,998 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:38,005 [538] [DEBUG] [app] Ending request: urn:request:e72569e9-a616-4ef2-9b11-381ace9c69ab (/v2/auth) gunicorn-registry stdout | 2020-03-31 09:24:38,005 [538] [DEBUG] [app] Ending request: urn:request:e72569e9-a616-4ef2-9b11-381ace9c69ab (/v2/auth) 2020-03-31 09:24:38,006 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:38,006 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:24:38,006 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:38,006 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:24:38,006 [538] [INFO] [gunicorn.access] 10.128.2.9 - admin [31/Mar/2020:09:24:38 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/1.1" 200 1082 "-" "Go-http-client/2.0" 10.128.2.9 () - admin [31/Mar/2020:09:24:38 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/2.0" 200 1082 "-" "Go-http-client/2.0" (0.405 134 0.405) gunicorn-registry stdout | 2020-03-31 09:24:38,006 [538] [INFO] [gunicorn.access] 10.128.2.9 - admin [31/Mar/2020:09:24:38 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/1.1" 200 1082 "-" "Go-http-client/2.0" nginx stdout | 10.128.2.9 () - admin [31/Mar/2020:09:24:38 +0000] "GET /v2/auth?account=admin&scope=repository%3Atestorg%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/2.0" 200 1082 "-" "Go-http-client/2.0" (0.405 134 0.405) 2020-03-31 09:24:38,008 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:38,008 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:38,008 [538] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:38,008 [538] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:24:38,010 [556] [DEBUG] [app] Starting request: urn:request:6a471f3d-8214-4dbe-bede-285c782911a4 (/v2/testorg/redhat-operators/blobs/uploads/) gunicorn-registry stdout | 2020-03-31 09:24:38,010 [556] [DEBUG] [app] Starting request: urn:request:6a471f3d-8214-4dbe-bede-285c782911a4 (/v2/testorg/redhat-operators/blobs/uploads/) 2020-03-31 09:24:38,011 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:46.924497+00:00 (in 8.912823 seconds) 2020-03-31 09:24:38,011 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'testorg', 'repo_name': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:24:38,011 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:38 UTC)" (scheduled at 2020-03-31 09:24:38.010934+00:00) 2020-03-31 09:24:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:38 UTC)" executed successfully 2020-03-31 09:24:38,011 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMjc4LCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2Njc4LCJuYmYiOjE1ODU2NDY2NzgsInN1YiI6ImFkbWluIn0.q6Y5-KPT6skWfNN7Y-JREnffP_fDjG0iclbCxxUqlxkXE3nfJDdUi2hbPGnTEXbbpCgITX05rJhougq6gqLkGKExGH4uGldMTG6SQUdz2CmfsFl9QF8ln8_sdWZKMM4UKNc3G8uDntnxWGxXW-xG3KUBtxIrcg_SUk3yo8mo0qoF-0oxPkYc2hc1chT7RGrCVxbo1kszZ9eoEcehpR8sSW7b_B4Ycaf-oBuGrpHUVqG7Vy55AP2KfwegE0VCLSKGUd0TIfNcKoLMDRQ154WLryL9NNKm8fGmLiMJ_IbIZ4icaZNtO3B1sqsUD9IW8ppZ1sE6g3geTZ6w2pcngNobLQ 2020-03-31 09:24:38,012 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMjc4LCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2Njc4LCJuYmYiOjE1ODU2NDY2NzgsInN1YiI6ImFkbWluIn0.q6Y5-KPT6skWfNN7Y-JREnffP_fDjG0iclbCxxUqlxkXE3nfJDdUi2hbPGnTEXbbpCgITX05rJhougq6gqLkGKExGH4uGldMTG6SQUdz2CmfsFl9QF8ln8_sdWZKMM4UKNc3G8uDntnxWGxXW-xG3KUBtxIrcg_SUk3yo8mo0qoF-0oxPkYc2hc1chT7RGrCVxbo1kszZ9eoEcehpR8sSW7b_B4Ycaf-oBuGrpHUVqG7Vy55AP2KfwegE0VCLSKGUd0TIfNcKoLMDRQ154WLryL9NNKm8fGmLiMJ_IbIZ4icaZNtO3B1sqsUD9IW8ppZ1sE6g3geTZ6w2pcngNobLQ storagereplication stdout | 2020-03-31 09:24:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:46.924497+00:00 (in 8.912823 seconds) gunicorn-registry stdout | 2020-03-31 09:24:38,011 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'testorg', 'repo_name': 'redhat-operators'} 2020-03-31 09:24:38,013 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:24:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:38 UTC)" (scheduled at 2020-03-31 09:24:38.010934+00:00) 2020-03-31 09:24:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:38 UTC)" executed successfully 2020-03-31 09:24:38,014 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:38,014 [538] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'login', 'namespace': 'testorg', 'repository': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:24:38,011 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMjc4LCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2Njc4LCJuYmYiOjE1ODU2NDY2NzgsInN1YiI6ImFkbWluIn0.q6Y5-KPT6skWfNN7Y-JREnffP_fDjG0iclbCxxUqlxkXE3nfJDdUi2hbPGnTEXbbpCgITX05rJhougq6gqLkGKExGH4uGldMTG6SQUdz2CmfsFl9QF8ln8_sdWZKMM4UKNc3G8uDntnxWGxXW-xG3KUBtxIrcg_SUk3yo8mo0qoF-0oxPkYc2hc1chT7RGrCVxbo1kszZ9eoEcehpR8sSW7b_B4Ycaf-oBuGrpHUVqG7Vy55AP2KfwegE0VCLSKGUd0TIfNcKoLMDRQ154WLryL9NNKm8fGmLiMJ_IbIZ4icaZNtO3B1sqsUD9IW8ppZ1sE6g3geTZ6w2pcngNobLQ 2020-03-31 09:24:38,012 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6InRlc3RvcmcvcmVkaGF0LW9wZXJhdG9ycyIsImFjdGlvbnMiOltdfV0sImNvbnRleHQiOnsiZW50aXR5X2tpbmQiOiJ1c2VyIiwia2luZCI6InVzZXIiLCJ2ZXJzaW9uIjoyLCJjb20uYXBvc3RpbGxlLnJvb3QiOiIkZGlzYWJsZWQiLCJ1c2VyIjoiYWRtaW4iLCJlbnRpdHlfcmVmZXJlbmNlIjoiODM1M2NhYWUtODVlYy00NzY0LThjMTItYmRhM2ZlYjA3MmFjIiwiY29tLmFwb3N0aWxsZS5yb290cyI6eyJ0ZXN0b3JnL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMjc4LCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2Njc4LCJuYmYiOjE1ODU2NDY2NzgsInN1YiI6ImFkbWluIn0.q6Y5-KPT6skWfNN7Y-JREnffP_fDjG0iclbCxxUqlxkXE3nfJDdUi2hbPGnTEXbbpCgITX05rJhougq6gqLkGKExGH4uGldMTG6SQUdz2CmfsFl9QF8ln8_sdWZKMM4UKNc3G8uDntnxWGxXW-xG3KUBtxIrcg_SUk3yo8mo0qoF-0oxPkYc2hc1chT7RGrCVxbo1kszZ9eoEcehpR8sSW7b_B4Ycaf-oBuGrpHUVqG7Vy55AP2KfwegE0VCLSKGUd0TIfNcKoLMDRQ154WLryL9NNKm8fGmLiMJ_IbIZ4icaZNtO3B1sqsUD9IW8ppZ1sE6g3geTZ6w2pcngNobLQ gunicorn-registry stdout | 2020-03-31 09:24:38,013 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:38,014 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:38,014 [538] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'login', 'namespace': 'testorg', 'repository': 'redhat-operators'} 2020-03-31 09:24:38,016 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 24, 38, 14215), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 24, 38, 14279), u'quay', None, datetime.datetime(2020, 3, 30, 9, 24, 38, 14312), datetime.datetime(2020, 3, 24, 9, 24, 38, 14343), None]) 2020-03-31 09:24:38,016 [556] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2020-03-31 09:24:38,016 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 24, 38, 14215), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 24, 38, 14279), u'quay', None, datetime.datetime(2020, 3, 30, 9, 24, 38, 14312), datetime.datetime(2020, 3, 24, 9, 24, 38, 14343), None]) 2020-03-31 09:24:38,016 [556] [DEBUG] [peewee.pool] No connection available in pool. 2020-03-31 09:24:38,026 [556] [DEBUG] [peewee.pool] Created new connection 140043679892496. gunicorn-registry stdout | 2020-03-31 09:24:38,026 [556] [DEBUG] [peewee.pool] Created new connection 140043679892496. 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'testorg/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:24:38,041 [556] [DEBUG] [endpoints.v2] Checking permission for repo: testorg/redhat-operators gunicorn-registry stdout | 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:24:38,041 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'testorg/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:24:38,041 [556] [DEBUG] [endpoints.v2] Checking permission for repo: testorg/redhat-operators 2020-03-31 09:24:38,042 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"UNAUTHORIZED","detail":{},"message":"access to the requested resource is not authorized"}]} gunicorn-registry stdout | 2020-03-31 09:24:38,042 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"UNAUTHORIZED","detail":{},"message":"access to the requested resource is not authorized"}]} 2020-03-31 09:24:38,043 [556] [DEBUG] [app] Ending request: urn:request:6a471f3d-8214-4dbe-bede-285c782911a4 (/v2/testorg/redhat-operators/blobs/uploads/) gunicorn-registry stdout | 2020-03-31 09:24:38,043 [556] [DEBUG] [app] Ending request: urn:request:6a471f3d-8214-4dbe-bede-285c782911a4 (/v2/testorg/redhat-operators/blobs/uploads/) 2020-03-31 09:24:38,043 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:38,044 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:24:38,043 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:38,044 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:24:38,044 [556] [INFO] [gunicorn.access] 10.128.2.9 - - [31/Mar/2020:09:24:38 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/1.1" 401 112 "-" "Go-http-client/2.0" 10.128.2.9 () - - [31/Mar/2020:09:24:38 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/2.0" 401 112 "-" "Go-http-client/2.0" (0.035 906 0.035) gunicorn-registry stdout | 2020-03-31 09:24:38,044 [556] [INFO] [gunicorn.access] 10.128.2.9 - - [31/Mar/2020:09:24:38 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/1.1" 401 112 "-" "Go-http-client/2.0" nginx stdout | 10.128.2.9 () - - [31/Mar/2020:09:24:38 +0000] "POST /v2/testorg/redhat-operators/blobs/uploads/ HTTP/2.0" 401 112 "-" "Go-http-client/2.0" (0.035 906 0.035) 2020-03-31 09:24:38,046 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:38,046 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:38,047 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:38,047 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:38,048 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:38,048 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:24:38,273 [518] [DEBUG] [app] Starting request: urn:request:35911600-fbdc-40be-af24-621a56ffdc54 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:38,273 [518] [DEBUG] [app] Starting request: urn:request:35911600-fbdc-40be-af24-621a56ffdc54 (/health/instance) 2020-03-31 09:24:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:38,280 [540] [DEBUG] [app] Starting request: urn:request:8c6053d4-11df-41a1-b795-3719f91c6663 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:38,280 [540] [DEBUG] [app] Starting request: urn:request:8c6053d4-11df-41a1-b795-3719f91c6663 (/v1/_internal_ping) 2020-03-31 09:24:38,281 [540] [DEBUG] [app] Ending request: urn:request:8c6053d4-11df-41a1-b795-3719f91c6663 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:38,281 [540] [DEBUG] [app] Ending request: urn:request:8c6053d4-11df-41a1-b795-3719f91c6663 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:38,282 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:24:38,282 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:38,285 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:38,285 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:38,287 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:38,287 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:38,287 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:38,287 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:38,318 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:38,318 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:38,323 [518] [DEBUG] [app] Starting request: urn:request:25c8b851-e692-4ea0-8060-f1b7ac72bfbd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:38,323 [518] [DEBUG] [app] Starting request: urn:request:25c8b851-e692-4ea0-8060-f1b7ac72bfbd (/_internal_ping) 2020-03-31 09:24:38,324 [518] [DEBUG] [app] Ending request: urn:request:25c8b851-e692-4ea0-8060-f1b7ac72bfbd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:38,324 [518] [DEBUG] [app] Ending request: urn:request:25c8b851-e692-4ea0-8060-f1b7ac72bfbd (/_internal_ping) 2020-03-31 09:24:38,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:38,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:38,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:38,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:38,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:38,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:38,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:38,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:38,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:38,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:38,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:38,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:38,336 [532] [DEBUG] [app] Starting request: urn:request:086ba90c-ea92-4664-89d7-ecf6c613ea6a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:38,336 [532] [DEBUG] [app] Starting request: urn:request:086ba90c-ea92-4664-89d7-ecf6c613ea6a (/c1/_internal_ping) 2020-03-31 09:24:38,337 [532] [DEBUG] [app] Ending request: urn:request:086ba90c-ea92-4664-89d7-ecf6c613ea6a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:38,337 [532] [DEBUG] [app] Ending request: urn:request:086ba90c-ea92-4664-89d7-ecf6c613ea6a (/c1/_internal_ping) 2020-03-31 09:24:38,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:38,338 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:38,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:24:38,338 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:38,339 [518] [DEBUG] [app] Ending request: urn:request:35911600-fbdc-40be-af24-621a56ffdc54 (/health/instance) 2020-03-31 09:24:38,340 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:38,339 [518] [DEBUG] [app] Ending request: urn:request:35911600-fbdc-40be-af24-621a56ffdc54 (/health/instance) 2020-03-31 09:24:38,340 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:38,340 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:24:38,340 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:24:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:24:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:51.440707+00:00 (in 9.999283 seconds) 2020-03-31 09:24:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:51 UTC)" (scheduled at 2020-03-31 09:24:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:24:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:51.440707+00:00 (in 9.999283 seconds) 2020-03-31 09:24:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:51 UTC)" (scheduled at 2020-03-31 09:24:41.440707+00:00) 2020-03-31 09:24:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:24:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:24:41,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:24:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:24:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:24:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:51 UTC)" executed successfully 2020-03-31 09:24:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:24:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:14.539408+00:00 (in 30.998614 seconds) 2020-03-31 09:24:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:43 UTC)" (scheduled at 2020-03-31 09:24:43.539999+00:00) 2020-03-31 09:24:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:24:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:14.539408+00:00 (in 30.998614 seconds) 2020-03-31 09:24:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:43 UTC)" (scheduled at 2020-03-31 09:24:43.539999+00:00) 2020-03-31 09:24:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:43 UTC)" executed successfully 2020-03-31 09:24:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:24:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:49.498966+00:00 (in 5.174729 seconds) 2020-03-31 09:24:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:54 UTC)" (scheduled at 2020-03-31 09:24:44.323352+00:00) 2020-03-31 09:24:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:24:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:49.498966+00:00 (in 5.174729 seconds) 2020-03-31 09:24:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:54 UTC)" (scheduled at 2020-03-31 09:24:44.323352+00:00) 2020-03-31 09:24:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 44, 325173), True, datetime.datetime(2020, 3, 31, 9, 24, 44, 325173), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:24:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 44, 325173), True, datetime.datetime(2020, 3, 31, 9, 24, 44, 325173), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:24:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:24:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:24:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:54 UTC)" executed successfully 2020-03-31 09:24:44,446 [518] [DEBUG] [app] Starting request: urn:request:27c782d3-f0db-41af-a27d-b23636fdb4ff (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:44,446 [518] [DEBUG] [app] Starting request: urn:request:27c782d3-f0db-41af-a27d-b23636fdb4ff (/health/instance) 2020-03-31 09:24:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:44,453 [555] [DEBUG] [app] Starting request: urn:request:6053632b-c0f5-4ccc-a982-0c1dab4cd007 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:44,453 [555] [DEBUG] [app] Starting request: urn:request:6053632b-c0f5-4ccc-a982-0c1dab4cd007 (/v1/_internal_ping) 2020-03-31 09:24:44,455 [555] [DEBUG] [app] Ending request: urn:request:6053632b-c0f5-4ccc-a982-0c1dab4cd007 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:44,455 [555] [DEBUG] [app] Ending request: urn:request:6053632b-c0f5-4ccc-a982-0c1dab4cd007 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:44,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:24:44,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:44,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:44,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:44,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:44,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:44,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:44,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:44,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:44,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:44,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:44,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:44,495 [518] [DEBUG] [app] Starting request: urn:request:2cd8687d-0e9d-4aef-91cd-7815faf0586a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:44,495 [518] [DEBUG] [app] Starting request: urn:request:2cd8687d-0e9d-4aef-91cd-7815faf0586a (/_internal_ping) 2020-03-31 09:24:44,495 [518] [DEBUG] [app] Ending request: urn:request:2cd8687d-0e9d-4aef-91cd-7815faf0586a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:44,495 [518] [DEBUG] [app] Ending request: urn:request:2cd8687d-0e9d-4aef-91cd-7815faf0586a (/_internal_ping) 2020-03-31 09:24:44,496 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:44,496 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:44,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:44,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:44,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:44,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:44,509 [539] [DEBUG] [app] Starting request: urn:request:b5d16ea8-f960-4cc2-94ad-e16e934a953e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:44,509 [539] [DEBUG] [app] Starting request: urn:request:b5d16ea8-f960-4cc2-94ad-e16e934a953e (/c1/_internal_ping) 2020-03-31 09:24:44,510 [539] [DEBUG] [app] Ending request: urn:request:b5d16ea8-f960-4cc2-94ad-e16e934a953e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:44,510 [539] [DEBUG] [app] Ending request: urn:request:b5d16ea8-f960-4cc2-94ad-e16e934a953e (/c1/_internal_ping) 2020-03-31 09:24:44,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:44,511 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:44,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:24:44,511 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:44,513 [518] [DEBUG] [app] Ending request: urn:request:27c782d3-f0db-41af-a27d-b23636fdb4ff (/health/instance) 2020-03-31 09:24:44,513 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:44,513 [518] [DEBUG] [app] Ending request: urn:request:27c782d3-f0db-41af-a27d-b23636fdb4ff (/health/instance) 2020-03-31 09:24:44,513 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:44,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:24:44,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:24:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:24:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:15 UTC)" (scheduled at 2020-03-31 09:24:45.064514+00:00) gcworker stdout | 2020-03-31 09:24:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:15 UTC)" (scheduled at 2020-03-31 09:24:45.064514+00:00) 2020-03-31 09:24:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:15.064514+00:00 (in 29.999090 seconds) gcworker stdout | 2020-03-31 09:24:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:15.064514+00:00 (in 29.999090 seconds) 2020-03-31 09:24:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437085, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:24:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437085, 1209600, 500, 1, 0]) 2020-03-31 09:24:45,089 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:24:45,089 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:24:45,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:24:45,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:45,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:15 UTC)" executed successfully 2020-03-31 09:24:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:24:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:56.924497+00:00 (in 9.999221 seconds) 2020-03-31 09:24:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:56 UTC)" (scheduled at 2020-03-31 09:24:46.924497+00:00) 2020-03-31 09:24:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:24:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:56.924497+00:00 (in 9.999221 seconds) 2020-03-31 09:24:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:56 UTC)" (scheduled at 2020-03-31 09:24:46.924497+00:00) 2020-03-31 09:24:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 46, 926002), True, datetime.datetime(2020, 3, 31, 9, 24, 46, 926002), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:24:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 46, 926002), True, datetime.datetime(2020, 3, 31, 9, 24, 46, 926002), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:24:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:24:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:24:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:24:56 UTC)" executed successfully 2020-03-31 09:24:48,272 [525] [DEBUG] [app] Starting request: urn:request:c31e7788-5b13-49ae-850f-268791324546 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:48,272 [525] [DEBUG] [app] Starting request: urn:request:c31e7788-5b13-49ae-850f-268791324546 (/health/instance) 2020-03-31 09:24:48,274 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:48,274 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:48,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:48,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:48,279 [554] [DEBUG] [app] Starting request: urn:request:0c249818-3c3c-4979-a720-9be581ab0b5a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:48,279 [554] [DEBUG] [app] Starting request: urn:request:0c249818-3c3c-4979-a720-9be581ab0b5a (/v1/_internal_ping) 2020-03-31 09:24:48,280 [554] [DEBUG] [app] Ending request: urn:request:0c249818-3c3c-4979-a720-9be581ab0b5a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:48,280 [554] [DEBUG] [app] Ending request: urn:request:0c249818-3c3c-4979-a720-9be581ab0b5a (/v1/_internal_ping) 2020-03-31 09:24:48,281 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:24:48,281 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:48,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:48,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:48,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:48,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:48,284 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:48,284 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:48,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:48,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:48,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:48,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:48,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:48,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:48,307 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:48,307 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:48,310 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:48,310 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:48,312 [525] [DEBUG] [app] Starting request: urn:request:e85b9042-31a5-4c35-beb9-27123cc5b161 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:48,312 [525] [DEBUG] [app] Starting request: urn:request:e85b9042-31a5-4c35-beb9-27123cc5b161 (/_internal_ping) 2020-03-31 09:24:48,312 [525] [DEBUG] [app] Ending request: urn:request:e85b9042-31a5-4c35-beb9-27123cc5b161 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:48,312 [525] [DEBUG] [app] Ending request: urn:request:e85b9042-31a5-4c35-beb9-27123cc5b161 (/_internal_ping) 2020-03-31 09:24:48,313 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:48,313 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:48,313 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:24:48,313 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:48,316 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:48,316 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:48,317 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:24:48,317 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:48,320 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:48,320 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:48,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:48,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:48,324 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:48,324 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:48,326 [543] [DEBUG] [app] Starting request: urn:request:771030df-892b-4add-b24b-a69df6117375 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:48,326 [543] [DEBUG] [app] Starting request: urn:request:771030df-892b-4add-b24b-a69df6117375 (/c1/_internal_ping) 2020-03-31 09:24:48,326 [543] [DEBUG] [app] Ending request: urn:request:771030df-892b-4add-b24b-a69df6117375 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:48,327 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:48,328 [525] [DEBUG] [app] Ending request: urn:request:c31e7788-5b13-49ae-850f-268791324546 (/health/instance) 2020-03-31 09:24:48,329 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:24:48,326 [543] [DEBUG] [app] Ending request: urn:request:771030df-892b-4add-b24b-a69df6117375 (/c1/_internal_ping) 2020-03-31 09:24:48,329 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:24:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:48,327 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:48,328 [525] [DEBUG] [app] Ending request: urn:request:c31e7788-5b13-49ae-850f-268791324546 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:48,329 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:48,329 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:24:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:24:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:54.323352+00:00 (in 4.823498 seconds) 2020-03-31 09:24:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:49 UTC)" (scheduled at 2020-03-31 09:24:49.498966+00:00) 2020-03-31 09:24:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:24:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:54.323352+00:00 (in 4.823498 seconds) 2020-03-31 09:24:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:49 UTC)" (scheduled at 2020-03-31 09:24:49.498966+00:00) 2020-03-31 09:24:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:49 UTC)" executed successfully 2020-03-31 09:24:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:24:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:53.556282+00:00 (in 2.444309 seconds) 2020-03-31 09:24:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:51 UTC)" (scheduled at 2020-03-31 09:24:51.110610+00:00) 2020-03-31 09:24:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:24:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:53.556282+00:00 (in 2.444309 seconds) 2020-03-31 09:24:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:51 UTC)" (scheduled at 2020-03-31 09:24:51.110610+00:00) 2020-03-31 09:24:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 51, 113198), True, datetime.datetime(2020, 3, 31, 9, 24, 51, 113198), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:24:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 51, 113198), True, datetime.datetime(2020, 3, 31, 9, 24, 51, 113198), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:24:51,132 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:24:51,132 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:24:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:51 UTC)" executed successfully 2020-03-31 09:24:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:24:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:01.440707+00:00 (in 9.999241 seconds) 2020-03-31 09:24:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:01 UTC)" (scheduled at 2020-03-31 09:24:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:24:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:01.440707+00:00 (in 9.999241 seconds) 2020-03-31 09:24:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:01 UTC)" (scheduled at 2020-03-31 09:24:51.440707+00:00) 2020-03-31 09:24:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:24:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:24:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:24:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:24:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:24:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:01 UTC)" executed successfully 2020-03-31 09:24:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:24:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:22.113221+00:00 (in 29.999218 seconds) 2020-03-31 09:24:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:22 UTC)" (scheduled at 2020-03-31 09:24:52.113221+00:00) securityworker stdout | 2020-03-31 09:24:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:22.113221+00:00 (in 29.999218 seconds) 2020-03-31 09:24:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:22 UTC)" (scheduled at 2020-03-31 09:24:52.113221+00:00) 2020-03-31 09:24:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:24:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:24:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:24:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:24:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:24:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:24:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:24:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:22 UTC)" executed successfully 2020-03-31 09:24:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:24:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:59.560326+00:00 (in 6.002780 seconds) 2020-03-31 09:24:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:53 UTC)" (scheduled at 2020-03-31 09:24:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:24:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:24:59.560326+00:00 (in 6.002780 seconds) 2020-03-31 09:24:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:53 UTC)" (scheduled at 2020-03-31 09:24:53.556282+00:00) 2020-03-31 09:24:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:24:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), u'namespacegc/%']) 2020-03-31 09:24:53,573 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), True, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:24:53,573 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), True, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), 0, u'namespacegc/%']) 2020-03-31 09:24:53,577 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), True, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:24:53,577 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), True, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 24, 53, 558177), u'namespacegc/%']) 2020-03-31 09:24:53,580 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:24:53,580 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:53,582 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:24:53,582 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:29:53 UTC)" executed successfully 2020-03-31 09:24:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:24:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:53,584 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:24:53,584 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:53,585 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:24:53,585 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:24:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:24:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:04.323352+00:00 (in 9.999170 seconds) 2020-03-31 09:24:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:04 UTC)" (scheduled at 2020-03-31 09:24:54.323352+00:00) 2020-03-31 09:24:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:24:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:04.323352+00:00 (in 9.999170 seconds) 2020-03-31 09:24:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:04 UTC)" (scheduled at 2020-03-31 09:24:54.323352+00:00) 2020-03-31 09:24:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 54, 325145), True, datetime.datetime(2020, 3, 31, 9, 24, 54, 325145), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:24:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 54, 325145), True, datetime.datetime(2020, 3, 31, 9, 24, 54, 325145), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:24:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:24:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:24:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:04 UTC)" executed successfully 2020-03-31 09:24:54,446 [526] [DEBUG] [app] Starting request: urn:request:3dc58d89-c504-423e-a5ce-860e8a2f3116 (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:54,446 [526] [DEBUG] [app] Starting request: urn:request:3dc58d89-c504-423e-a5ce-860e8a2f3116 (/health/instance) 2020-03-31 09:24:54,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:54,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:54,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:54,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:54,454 [551] [DEBUG] [app] Starting request: urn:request:0f14dd7a-754c-41aa-9ebd-34b5d371fd61 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:54,454 [551] [DEBUG] [app] Starting request: urn:request:0f14dd7a-754c-41aa-9ebd-34b5d371fd61 (/v1/_internal_ping) 2020-03-31 09:24:54,455 [551] [DEBUG] [app] Ending request: urn:request:0f14dd7a-754c-41aa-9ebd-34b5d371fd61 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:54,455 [551] [DEBUG] [app] Ending request: urn:request:0f14dd7a-754c-41aa-9ebd-34b5d371fd61 (/v1/_internal_ping) 2020-03-31 09:24:54,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:24:54,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:54,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:54,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:54,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:54,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:24:54,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:24:54,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:54,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:54,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:54,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:54,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:54,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:54,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:54,490 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:54,490 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:54,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:54,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:54,496 [527] [DEBUG] [app] Starting request: urn:request:d29515d7-45a4-4823-9120-baa091fd65ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:54,496 [527] [DEBUG] [app] Starting request: urn:request:d29515d7-45a4-4823-9120-baa091fd65ce (/_internal_ping) 2020-03-31 09:24:54,497 [527] [DEBUG] [app] Ending request: urn:request:d29515d7-45a4-4823-9120-baa091fd65ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:54,497 [527] [DEBUG] [app] Ending request: urn:request:d29515d7-45a4-4823-9120-baa091fd65ce (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:24:54,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:54,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:24:54,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:54,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:54,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:54,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:54,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:24:54,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:24:54,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:54,504 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:54,505 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:54,505 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:54,509 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:54,509 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:54,510 [534] [DEBUG] [app] Starting request: urn:request:1ee08541-49e9-4e8c-ae88-76f44f9084d7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:54,510 [534] [DEBUG] [app] Starting request: urn:request:1ee08541-49e9-4e8c-ae88-76f44f9084d7 (/c1/_internal_ping) 2020-03-31 09:24:54,511 [534] [DEBUG] [app] Ending request: urn:request:1ee08541-49e9-4e8c-ae88-76f44f9084d7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:54,511 [534] [DEBUG] [app] Ending request: urn:request:1ee08541-49e9-4e8c-ae88-76f44f9084d7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:24:54,512 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:54,512 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:54,512 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:24:54,512 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:54,513 [526] [DEBUG] [app] Ending request: urn:request:3dc58d89-c504-423e-a5ce-860e8a2f3116 (/health/instance) 2020-03-31 09:24:54,514 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:24:54,513 [526] [DEBUG] [app] Ending request: urn:request:3dc58d89-c504-423e-a5ce-860e8a2f3116 (/health/instance) 2020-03-31 09:24:54,514 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:54,514 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:24:54,514 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:24:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:24:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:06.924497+00:00 (in 9.999231 seconds) 2020-03-31 09:24:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:06 UTC)" (scheduled at 2020-03-31 09:24:56.924497+00:00) 2020-03-31 09:24:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:24:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:06.924497+00:00 (in 9.999231 seconds) 2020-03-31 09:24:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:06 UTC)" (scheduled at 2020-03-31 09:24:56.924497+00:00) 2020-03-31 09:24:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:24:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 56, 926001), True, datetime.datetime(2020, 3, 31, 9, 24, 56, 926001), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:24:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 24, 56, 926001), True, datetime.datetime(2020, 3, 31, 9, 24, 56, 926001), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:24:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:24:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:24:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:24:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:06 UTC)" executed successfully 2020-03-31 09:24:58,272 [523] [DEBUG] [app] Starting request: urn:request:ac8d1a1d-e0c1-4830-8e8d-972c13e0955e (/health/instance) gunicorn-web stdout | 2020-03-31 09:24:58,272 [523] [DEBUG] [app] Starting request: urn:request:ac8d1a1d-e0c1-4830-8e8d-972c13e0955e (/health/instance) 2020-03-31 09:24:58,275 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:58,275 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:58,279 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:58,279 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:58,281 [555] [DEBUG] [app] Starting request: urn:request:b85e227f-07fb-4408-8a21-bacc969c935f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:58,281 [555] [DEBUG] [app] Starting request: urn:request:b85e227f-07fb-4408-8a21-bacc969c935f (/v1/_internal_ping) 2020-03-31 09:24:58,282 [555] [DEBUG] [app] Ending request: urn:request:b85e227f-07fb-4408-8a21-bacc969c935f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:24:58,282 [555] [DEBUG] [app] Ending request: urn:request:b85e227f-07fb-4408-8a21-bacc969c935f (/v1/_internal_ping) 2020-03-31 09:24:58,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:58,282 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:24:58,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:58,283 [523] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:24:58,282 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:58,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:24:58,283 [523] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:24:58,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:24:58,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:24:58,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:24:58,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:24:58,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:24:58,297 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:24:58,297 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:24:58,317 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:24:58,317 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:24:58,321 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:58,321 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:58,323 [523] [DEBUG] [app] Starting request: urn:request:4b600cbf-71f4-464e-8c85-e94a58bc438a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:58,323 [523] [DEBUG] [app] Starting request: urn:request:4b600cbf-71f4-464e-8c85-e94a58bc438a (/_internal_ping) 2020-03-31 09:24:58,323 [523] [DEBUG] [app] Ending request: urn:request:4b600cbf-71f4-464e-8c85-e94a58bc438a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:24:58,323 [523] [DEBUG] [app] Ending request: urn:request:4b600cbf-71f4-464e-8c85-e94a58bc438a (/_internal_ping) 2020-03-31 09:24:58,324 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:58,324 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:24:58,324 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:24:58,324 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:24:58,327 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:58,327 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:24:58,328 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:24:58,328 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:24:58,331 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:58,331 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:24:58,332 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:24:58,332 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:24:58,335 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:24:58,335 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:24:58,336 [534] [DEBUG] [app] Starting request: urn:request:7f816734-153e-41fd-9140-d475ee9af6fa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:58,336 [534] [DEBUG] [app] Starting request: urn:request:7f816734-153e-41fd-9140-d475ee9af6fa (/c1/_internal_ping) 2020-03-31 09:24:58,337 [534] [DEBUG] [app] Ending request: urn:request:7f816734-153e-41fd-9140-d475ee9af6fa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:24:58,337 [534] [DEBUG] [app] Ending request: urn:request:7f816734-153e-41fd-9140-d475ee9af6fa (/c1/_internal_ping) 2020-03-31 09:24:58,338 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:58,338 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:24:58,338 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:24:58,340 [523] [DEBUG] [app] Ending request: urn:request:ac8d1a1d-e0c1-4830-8e8d-972c13e0955e (/health/instance) 2020-03-31 09:24:58,340 [523] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:24:58,338 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:24:58,340 [523] [DEBUG] [app] Ending request: urn:request:ac8d1a1d-e0c1-4830-8e8d-972c13e0955e (/health/instance) 2020-03-31 09:24:58,340 [523] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:24:58,341 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:24:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:24:58,341 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:24:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:24:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:24:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:24:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:51.110610+00:00 (in 51.549501 seconds) 2020-03-31 09:24:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:59 UTC)" (scheduled at 2020-03-31 09:24:59.560326+00:00) 2020-03-31 09:24:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:24:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:51.110610+00:00 (in 51.549501 seconds) 2020-03-31 09:24:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:59 UTC)" (scheduled at 2020-03-31 09:24:59.560326+00:00) 2020-03-31 09:24:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:24:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:59 UTC)" executed successfully 2020-03-31 09:25:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:25:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:00 UTC)" (scheduled at 2020-03-31 09:25:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:25:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:00 UTC)" (scheduled at 2020-03-31 09:25:00.337699+00:00) 2020-03-31 09:25:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:30.337699+00:00 (in 29.998943 seconds) buildlogsarchiver stdout | 2020-03-31 09:25:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:30.337699+00:00 (in 29.998943 seconds) 2020-03-31 09:25:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 25, 0, 339420), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:25:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 25, 0, 339420), False, 50, 1, 0]) 2020-03-31 09:25:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:25:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:25:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:25:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:30 UTC)" executed successfully 2020-03-31 09:25:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:25:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:11.440707+00:00 (in 9.998893 seconds) 2020-03-31 09:25:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:11 UTC)" (scheduled at 2020-03-31 09:25:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:25:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:11.440707+00:00 (in 9.998893 seconds) 2020-03-31 09:25:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:11 UTC)" (scheduled at 2020-03-31 09:25:01.440707+00:00) 2020-03-31 09:25:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:25:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:25:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:25:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:25:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:25:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:11 UTC)" executed successfully 2020-03-31 09:25:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:25:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:24.410927+00:00 (in 21.079797 seconds) 2020-03-31 09:25:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:33 UTC)" (scheduled at 2020-03-31 09:25:03.330112+00:00) 2020-03-31 09:25:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:25:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:24.410927+00:00 (in 21.079797 seconds) 2020-03-31 09:25:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:33 UTC)" (scheduled at 2020-03-31 09:25:03.330112+00:00) 2020-03-31 09:25:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 3, 332106), True, datetime.datetime(2020, 3, 31, 9, 25, 3, 332106), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:25:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 3, 332106), True, datetime.datetime(2020, 3, 31, 9, 25, 3, 332106), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:25:03,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:25:03,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:25:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:33 UTC)" executed successfully 2020-03-31 09:25:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:25:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:14.323352+00:00 (in 9.998967 seconds) 2020-03-31 09:25:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:14 UTC)" (scheduled at 2020-03-31 09:25:04.323352+00:00) 2020-03-31 09:25:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:25:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:14.323352+00:00 (in 9.998967 seconds) 2020-03-31 09:25:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:14 UTC)" (scheduled at 2020-03-31 09:25:04.323352+00:00) 2020-03-31 09:25:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 4, 325401), True, datetime.datetime(2020, 3, 31, 9, 25, 4, 325401), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:25:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 4, 325401), True, datetime.datetime(2020, 3, 31, 9, 25, 4, 325401), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:25:04,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:04,340 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:25:04,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:04,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:04,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:25:04,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:14 UTC)" executed successfully 2020-03-31 09:25:04,446 [523] [DEBUG] [app] Starting request: urn:request:ee020d56-c3dd-4339-8978-933dc386558a (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:04,446 [523] [DEBUG] [app] Starting request: urn:request:ee020d56-c3dd-4339-8978-933dc386558a (/health/instance) 2020-03-31 09:25:04,448 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:04,448 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:04,453 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:04,453 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:04,454 [544] [DEBUG] [app] Starting request: urn:request:63f32835-2889-4f06-b04f-40f011315fb7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:04,454 [544] [DEBUG] [app] Starting request: urn:request:63f32835-2889-4f06-b04f-40f011315fb7 (/v1/_internal_ping) 2020-03-31 09:25:04,455 [544] [DEBUG] [app] Ending request: urn:request:63f32835-2889-4f06-b04f-40f011315fb7 (/v1/_internal_ping) 2020-03-31 09:25:04,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:25:04,455 [544] [DEBUG] [app] Ending request: urn:request:63f32835-2889-4f06-b04f-40f011315fb7 (/v1/_internal_ping) 2020-03-31 09:25:04,459 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:04,457 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:04,458 [523] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:25:04,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:25:04,459 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:04,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:25:04,457 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:04,458 [523] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:04,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:25:04,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:04,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:04,469 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:04,469 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:04,482 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:04,482 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:04,485 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:04,485 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:04,488 [527] [DEBUG] [app] Starting request: urn:request:ee111583-baef-4fba-a89f-96867b02a2cf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:04,488 [527] [DEBUG] [app] Starting request: urn:request:ee111583-baef-4fba-a89f-96867b02a2cf (/_internal_ping) 2020-03-31 09:25:04,489 [527] [DEBUG] [app] Ending request: urn:request:ee111583-baef-4fba-a89f-96867b02a2cf (/_internal_ping) 2020-03-31 09:25:04,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:04,489 [527] [DEBUG] [app] Ending request: urn:request:ee111583-baef-4fba-a89f-96867b02a2cf (/_internal_ping) 2020-03-31 09:25:04,489 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:04,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:04,489 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:04,493 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:04,493 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:04,493 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:25:04,493 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:25:04,496 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:04,496 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:25:04,497 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:04,497 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:04,500 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:04,500 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:04,502 [534] [DEBUG] [app] Starting request: urn:request:afb7ff18-619a-49fc-b943-0dbfa8ec72f8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:04,502 [534] [DEBUG] [app] Starting request: urn:request:afb7ff18-619a-49fc-b943-0dbfa8ec72f8 (/c1/_internal_ping) 2020-03-31 09:25:04,503 [534] [DEBUG] [app] Ending request: urn:request:afb7ff18-619a-49fc-b943-0dbfa8ec72f8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:04,503 [534] [DEBUG] [app] Ending request: urn:request:afb7ff18-619a-49fc-b943-0dbfa8ec72f8 (/c1/_internal_ping) 2020-03-31 09:25:04,504 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:04,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:04,504 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:25:04,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:04,505 [523] [DEBUG] [app] Ending request: urn:request:ee020d56-c3dd-4339-8978-933dc386558a (/health/instance) 2020-03-31 09:25:04,505 [523] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:04,505 [523] [DEBUG] [app] Ending request: urn:request:ee020d56-c3dd-4339-8978-933dc386558a (/health/instance) 2020-03-31 09:25:04,505 [523] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:04,506 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:25:04,506 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:25:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:25:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:06 UTC)" (scheduled at 2020-03-31 09:25:06.924497+00:00) 2020-03-31 09:25:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:25:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:06 UTC)" (scheduled at 2020-03-31 09:25:06.924497+00:00) 2020-03-31 09:25:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 6, 925864), True, datetime.datetime(2020, 3, 31, 9, 25, 6, 925864), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:25:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 6, 925864), True, datetime.datetime(2020, 3, 31, 9, 25, 6, 925864), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:25:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:16.924497+00:00 (in 9.999309 seconds) storagereplication stdout | 2020-03-31 09:25:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:16.924497+00:00 (in 9.999309 seconds) 2020-03-31 09:25:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:25:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:25:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:16 UTC)" executed successfully 2020-03-31 09:25:08,273 [518] [DEBUG] [app] Starting request: urn:request:d322e5f4-48b8-4e08-b1b4-2ccba05c0c7f (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:08,273 [518] [DEBUG] [app] Starting request: urn:request:d322e5f4-48b8-4e08-b1b4-2ccba05c0c7f (/health/instance) 2020-03-31 09:25:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:08,280 [550] [DEBUG] [app] Starting request: urn:request:ae18870d-5ead-4cff-a411-519e47809201 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:08,280 [550] [DEBUG] [app] Starting request: urn:request:ae18870d-5ead-4cff-a411-519e47809201 (/v1/_internal_ping) 2020-03-31 09:25:08,281 [550] [DEBUG] [app] Ending request: urn:request:ae18870d-5ead-4cff-a411-519e47809201 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:08,281 [550] [DEBUG] [app] Ending request: urn:request:ae18870d-5ead-4cff-a411-519e47809201 (/v1/_internal_ping) 2020-03-31 09:25:08,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:25:08,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:08,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:25:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:25:08,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:08,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:08,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:08,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:08,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:08,300 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:08,300 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:08,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:08,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:08,319 [520] [DEBUG] [app] Starting request: urn:request:ef5ef4ec-4ead-48f2-a956-5f1abb7b6eed (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:08,319 [520] [DEBUG] [app] Starting request: urn:request:ef5ef4ec-4ead-48f2-a956-5f1abb7b6eed (/_internal_ping) 2020-03-31 09:25:08,320 [520] [DEBUG] [app] Ending request: urn:request:ef5ef4ec-4ead-48f2-a956-5f1abb7b6eed (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:08,320 [520] [DEBUG] [app] Ending request: urn:request:ef5ef4ec-4ead-48f2-a956-5f1abb7b6eed (/_internal_ping) 2020-03-31 09:25:08,321 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:25:08,321 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:25:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:08,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:08,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:08,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:25:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:08,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:08,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:08,335 [543] [DEBUG] [app] Starting request: urn:request:d29552f1-a275-4515-9461-a37f35223605 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:08,335 [543] [DEBUG] [app] Starting request: urn:request:d29552f1-a275-4515-9461-a37f35223605 (/c1/_internal_ping) 2020-03-31 09:25:08,335 [543] [DEBUG] [app] Ending request: urn:request:d29552f1-a275-4515-9461-a37f35223605 (/c1/_internal_ping) 2020-03-31 09:25:08,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:08,336 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:25:08,335 [543] [DEBUG] [app] Ending request: urn:request:d29552f1-a275-4515-9461-a37f35223605 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:08,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:08,338 [518] [DEBUG] [app] Ending request: urn:request:d322e5f4-48b8-4e08-b1b4-2ccba05c0c7f (/health/instance) 2020-03-31 09:25:08,338 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:25:08,336 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:08,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:25:08,338 [518] [DEBUG] [app] Ending request: urn:request:d322e5f4-48b8-4e08-b1b4-2ccba05c0c7f (/health/instance) 2020-03-31 09:25:08,338 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:08,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:25:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:25:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:21.440707+00:00 (in 9.999227 seconds) 2020-03-31 09:25:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:21 UTC)" (scheduled at 2020-03-31 09:25:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:25:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:21.440707+00:00 (in 9.999227 seconds) 2020-03-31 09:25:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:21 UTC)" (scheduled at 2020-03-31 09:25:11.440707+00:00) 2020-03-31 09:25:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:25:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:25:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:25:11,461 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:25:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:25:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:25:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:21 UTC)" executed successfully 2020-03-31 09:25:11,762 [556] [DEBUG] [app] Starting request: urn:request:842d4173-890b-4d08-9b21-7af4d8170ef9 (/v2/) gunicorn-registry stdout | 2020-03-31 09:25:11,762 [556] [DEBUG] [app] Starting request: urn:request:842d4173-890b-4d08-9b21-7af4d8170ef9 (/v2/) 2020-03-31 09:25:11,763 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {} 2020-03-31 09:25:11,763 [556] [DEBUG] [auth.registry_jwt_auth] No auth header. gunicorn-registry stdout | 2020-03-31 09:25:11,763 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {} 2020-03-31 09:25:11,763 [556] [DEBUG] [auth.registry_jwt_auth] No auth header. 2020-03-31 09:25:11,764 [556] [DEBUG] [app] Ending request: urn:request:842d4173-890b-4d08-9b21-7af4d8170ef9 (/v2/) gunicorn-registry stdout | 2020-03-31 09:25:11,764 [556] [DEBUG] [app] Ending request: urn:request:842d4173-890b-4d08-9b21-7af4d8170ef9 (/v2/) 2020-03-31 09:25:11,765 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:11 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:11 +0000] "GET /v2/ HTTP/2.0" 401 4 "-" "Go-http-client/2.0" (0.003 63 0.003) gunicorn-registry stdout | 2020-03-31 09:25:11,765 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:11 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:11 +0000] "GET /v2/ HTTP/2.0" 401 4 "-" "Go-http-client/2.0" (0.003 63 0.003) 2020-03-31 09:25:11,766 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:11,766 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:11,768 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:11,768 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:11,769 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:11,769 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:25:12,054 [538] [DEBUG] [app] Starting request: urn:request:d173ca2b-8519-4971-aa95-e6194a64b028 (/v2/auth) gunicorn-registry stdout | 2020-03-31 09:25:12,054 [538] [DEBUG] [app] Starting request: urn:request:d173ca2b-8519-4971-aa95-e6194a64b028 (/v2/auth) 2020-03-31 09:25:12,054 [538] [DEBUG] [auth.basic] Attempt to process basic auth header gunicorn-registry stdout | 2020-03-31 09:25:12,054 [538] [DEBUG] [auth.basic] Attempt to process basic auth header 2020-03-31 09:25:12,056 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', [u'admin', u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,056 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', [u'admin', u'admin', 1, 0]) 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.credentials] Successfully validated credentials for user admin gunicorn-registry stdout | 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.credentials] Successfully validated credentials for user admin 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [None, None, None, None, , None]) 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:25:12,383 [538] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com 2020-03-31 09:25:12,383 [538] [DEBUG] [endpoints.v2.v2auth] Scope request: [u'repository:admin/redhat-operators:pull,push'] gunicorn-registry stdout | 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [None, None, None, None, , None]) 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:12,382 [538] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:25:12,383 [538] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com 2020-03-31 09:25:12,383 [538] [DEBUG] [endpoints.v2.v2auth] Scope request: [u'repository:admin/redhat-operators:pull,push'] 2020-03-31 09:25:12,383 [538] [DEBUG] [endpoints.v2.v2auth] Match: (u'admin/redhat-operators', u'admin/redhat-operators', u'pull,push') gunicorn-registry stdout | 2020-03-31 09:25:12,383 [538] [DEBUG] [endpoints.v2.v2auth] Match: (u'admin/redhat-operators', u'admin/redhat-operators', u'pull,push') 2020-03-31 09:25:12,384 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,384 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) 2020-03-31 09:25:12,389 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,389 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac gunicorn-registry stdout | 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username=u'admin', role='admin') 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace=u'admin', role='admin') 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace=u'admin', role='admin') 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] Adding superuser to user: admin 2020-03-31 09:25:12,394 [538] [DEBUG] [endpoints.v2.v2auth] Creating repository: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username=u'admin', role='admin') 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace=u'admin', role='admin') 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace=u'admin', role='admin') 2020-03-31 09:25:12,394 [538] [DEBUG] [auth.permissions] Adding superuser to user: admin 2020-03-31 09:25:12,394 [538] [DEBUG] [endpoints.v2.v2auth] Creating repository: admin/redhat-operators 2020-03-31 09:25:12,395 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,395 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) 2020-03-31 09:25:12,399 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:12,399 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1"', []) 2020-03-31 09:25:12,401 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:12,401 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) 2020-03-31 09:25:12,404 [538] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [1, u'redhat-operators', 2, None, u'c9d38ea7-b47a-4340-8cdc-7a37cdfc151c', 1, False, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,404 [538] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [1, u'redhat-operators', 2, None, u'c9d38ea7-b47a-4340-8cdc-7a37cdfc151c', 1, False, 0]) 2020-03-31 09:25:12,408 [538] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [1, 0, datetime.date(2020, 3, 30)]) gunicorn-registry stdout | 2020-03-31 09:25:12,408 [538] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [1, 0, datetime.date(2020, 3, 30)]) 2020-03-31 09:25:12,411 [538] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,411 [538] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [1, 0]) 2020-03-31 09:25:12,413 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,413 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) 2020-03-31 09:25:12,415 [538] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [1, 1, 1]) gunicorn-registry stdout | 2020-03-31 09:25:12,415 [538] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [1, 1, 1]) 2020-03-31 09:25:12,421 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,421 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) 2020-03-31 09:25:12,425 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac gunicorn-registry stdout | 2020-03-31 09:25:12,425 [538] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 8353caae-85ec-4764-8c12-bda3feb072ac 2020-03-31 09:25:12,426 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,426 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:12,431 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:12,431 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:12,435 [538] [DEBUG] [app] Ending request: urn:request:d173ca2b-8519-4971-aa95-e6194a64b028 (/v2/auth) gunicorn-registry stdout | 2020-03-31 09:25:12,435 [538] [DEBUG] [app] Ending request: urn:request:d173ca2b-8519-4971-aa95-e6194a64b028 (/v2/auth) 2020-03-31 09:25:12,435 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,436 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:12,435 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,436 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:12,436 [538] [INFO] [gunicorn.access] 10.131.0.7 - admin [31/Mar/2020:09:25:12 +0000] "GET /v2/auth?account=admin&scope=repository%3Aadmin%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/1.1" 200 1094 "-" "Go-http-client/2.0" 10.131.0.7 () - admin [31/Mar/2020:09:25:12 +0000] "GET /v2/auth?account=admin&scope=repository%3Aadmin%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/2.0" 200 1094 "-" "Go-http-client/2.0" (0.384 133 0.384) gunicorn-registry stdout | 2020-03-31 09:25:12,436 [538] [INFO] [gunicorn.access] 10.131.0.7 - admin [31/Mar/2020:09:25:12 +0000] "GET /v2/auth?account=admin&scope=repository%3Aadmin%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/1.1" 200 1094 "-" "Go-http-client/2.0" 2020-03-31 09:25:12,437 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 nginx stdout | 10.131.0.7 () - admin [31/Mar/2020:09:25:12 +0000] "GET /v2/auth?account=admin&scope=repository%3Aadmin%2Fredhat-operators%3Apull%2Cpush&service=quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com HTTP/2.0" 200 1094 "-" "Go-http-client/2.0" (0.384 133 0.384) 2020-03-31 09:25:12,438 [538] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:12,437 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:12,439 [556] [DEBUG] [app] Starting request: urn:request:748b5b70-a3a3-488a-b70f-f44a337800cd (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:12,439 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:12,439 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,439 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:12,438 [538] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:12,439 [556] [DEBUG] [app] Starting request: urn:request:748b5b70-a3a3-488a-b70f-f44a337800cd (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:12,439 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:12,439 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,439 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,441 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:12,441 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:12,443 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:12,443 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:12,444 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:12,442 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:12,443 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:12,443 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:12,444 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:12,445 [538] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'push_start', 'namespace': 'admin', 'repository': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:25:12,445 [538] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'push_start', 'namespace': 'admin', 'repository': 'redhat-operators'} 2020-03-31 09:25:12,447 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,447 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:12,454 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:12,454 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) 2020-03-31 09:25:12,467 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,467 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) 2020-03-31 09:25:12,481 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,481 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:12,489 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:12,489 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) 2020-03-31 09:25:12,501 [556] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'678bb755-8858-455e-ad63-4d6503444bc8', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 12, 500523)]) gunicorn-registry stdout | 2020-03-31 09:25:12,501 [556] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'678bb755-8858-455e-ad63-4d6503444bc8', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 12, 500523)]) 2020-03-31 09:25:12,515 [556] [DEBUG] [app] Ending request: urn:request:748b5b70-a3a3-488a-b70f-f44a337800cd (/v2/admin/redhat-operators/blobs/uploads/) gunicorn-registry stdout | 2020-03-31 09:25:12,515 [556] [DEBUG] [app] Ending request: urn:request:748b5b70-a3a3-488a-b70f-f44a337800cd (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:12,516 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,516 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 10.131.0.7 () - - [31/Mar/2020:09:25:12 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.079 922 0.079) gunicorn-registry stdout | 2020-03-31 09:25:12,516 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,516 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:12,517 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:12 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:12 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.079 922 0.079) gunicorn-registry stdout | 2020-03-31 09:25:12,517 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:12 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:12,519 [538] [DEBUG] [app] Starting request: urn:request:ec889fa4-62c8-42f0-b219-abc85e485c4e (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) 2020-03-31 09:25:12,520 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'678bb755-8858-455e-ad63-4d6503444bc8', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:12,520 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:12,519 [538] [DEBUG] [app] Starting request: urn:request:ec889fa4-62c8-42f0-b219-abc85e485c4e (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) 2020-03-31 09:25:12,520 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'678bb755-8858-455e-ad63-4d6503444bc8', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:12,520 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,520 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:12,520 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:12,520 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:12,520 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,522 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:12,522 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:12,523 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:12,523 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 25, 12, 521055), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 25, 12, 521109), u'quay', None, datetime.datetime(2020, 3, 30, 9, 25, 12, 521136), datetime.datetime(2020, 3, 24, 9, 25, 12, 521157), None]) gunicorn-registry stdout | 2020-03-31 09:25:12,523 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:12,523 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 25, 12, 521055), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 25, 12, 521109), u'quay', None, datetime.datetime(2020, 3, 30, 9, 25, 12, 521136), datetime.datetime(2020, 3, 24, 9, 25, 12, 521157), None]) 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:12,534 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:12,534 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:12,534 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:12,537 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,537 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:12,544 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,544 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) 2020-03-31 09:25:12,548 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,548 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:12,551 [538] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:12,551 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,551 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:12,551 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:12,714 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 2221157 3HcBr14tgg59wfR51vzwlA== x-ms-client-request-id:8602b5c2-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:12 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 blockid:MDAwMDA= comp:block gunicorn-registry stdout | 2020-03-31 09:25:12,714 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 2221157 3HcBr14tgg59wfR51vzwlA== x-ms-client-request-id:8602b5c2-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:12 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 blockid:MDAwMDA= comp:block 2020-03-31 09:25:12,714 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8602b5c2-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '2221157', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8602b5c2-7331-11ea-bc11-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '3HcBr14tgg59wfR51vzwlA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OjR/Y2Y9xulz0cd1QgOg6/uOd5onAnL/KkduHL33RDk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:12,714 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8602b5c2-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '2221157', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8602b5c2-7331-11ea-bc11-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '3HcBr14tgg59wfR51vzwlA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OjR/Y2Y9xulz0cd1QgOg6/uOd5onAnL/KkduHL33RDk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. 2020-03-31 09:25:12,717 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 gunicorn-registry stdout | 2020-03-31 09:25:12,717 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 2020-03-31 09:25:12,895 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:12,895 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 2020-03-31 09:25:12,896 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8602b5c2-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=2f3c10cf-501e-0049-3c3e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '2221157', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8602b5c2-7331-11ea-bc11-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '3HcBr14tgg59wfR51vzwlA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OjR/Y2Y9xulz0cd1QgOg6/uOd5onAnL/KkduHL33RDk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:12,896 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8602b5c2-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=2f3c10cf-501e-0049-3c3e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '2221157', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8602b5c2-7331-11ea-bc11-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '3HcBr14tgg59wfR51vzwlA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OjR/Y2Y9xulz0cd1QgOg6/uOd5onAnL/KkduHL33RDk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. 2020-03-31 09:25:12,897 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,897 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) 2020-03-31 09:25:12,902 [538] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'678bb755-8858-455e-ad63-4d6503444bc8', 2221157, '[[3859691389, 1658786821, 4109600805, 1765523095, 526640288, 61365303, 1797148447, 1679396220], 17769256, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, 9315840, datetime.datetime(2020, 3, 31, 9, 25, 12, 500523), '[[1748105183, 2675024539, 2253341262, 1786249621, 1609115271], 992040, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1]', 'xMSAi/YmMbWi743r+3oX53SSghdQCOYIgrgARzjuFEAfAXOQ2WmeCuWlAI4S+5OFSAeAK5uoIa+jtiT9BraqQl2TIxT9pN5u+0BMxi1z3Ys=', 1]) gunicorn-registry stdout | 2020-03-31 09:25:12,902 [538] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'678bb755-8858-455e-ad63-4d6503444bc8', 2221157, '[[3859691389, 1658786821, 4109600805, 1765523095, 526640288, 61365303, 1797148447, 1679396220], 17769256, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, 9315840, datetime.datetime(2020, 3, 31, 9, 25, 12, 500523), '[[1748105183, 2675024539, 2253341262, 1786249621, 1609115271], 992040, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1]', 'xMSAi/YmMbWi743r+3oX53SSghdQCOYIgrgARzjuFEAfAXOQ2WmeCuWlAI4S+5OFSAeAK5uoIa+jtiT9BraqQl2TIxT9pN5u+0BMxi1z3Ys=', 1]) 2020-03-31 09:25:12,907 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,907 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:12,912 [538] [DEBUG] [app] Ending request: urn:request:ec889fa4-62c8-42f0-b219-abc85e485c4e (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) gunicorn-registry stdout | 2020-03-31 09:25:12,912 [538] [DEBUG] [app] Ending request: urn:request:ec889fa4-62c8-42f0-b219-abc85e485c4e (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) 2020-03-31 09:25:12,913 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,913 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 10.131.0.7 () - - [31/Mar/2020:09:25:12 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8 HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.395 2221227 0.395) gunicorn-registry stdout | 2020-03-31 09:25:12,913 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,913 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:12,913 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:12 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8 HTTP/1.1" 204 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:12 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8 HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.395 2221227 0.395) gunicorn-registry stdout | 2020-03-31 09:25:12,913 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:12 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8 HTTP/1.1" 204 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:12,915 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:12,915 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:12,916 [555] [DEBUG] [app] Starting request: urn:request:09858b8d-5284-4301-b7ab-fbdd5112fd99 (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) 2020-03-31 09:25:12,917 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'678bb755-8858-455e-ad63-4d6503444bc8', 'repo_name': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:25:12,916 [555] [DEBUG] [app] Starting request: urn:request:09858b8d-5284-4301-b7ab-fbdd5112fd99 (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) 2020-03-31 09:25:12,917 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'678bb755-8858-455e-ad63-4d6503444bc8', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:12,917 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,917 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:12,917 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:12,917 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,917 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:12,917 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:12,920 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:12,920 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:12,921 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 25, 12, 918165), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 25, 12, 918247), u'quay', None, datetime.datetime(2020, 3, 30, 9, 25, 12, 918281), datetime.datetime(2020, 3, 24, 9, 25, 12, 918313), None]) gunicorn-registry stdout | 2020-03-31 09:25:12,921 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 25, 12, 918165), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 25, 12, 918247), u'quay', None, datetime.datetime(2020, 3, 30, 9, 25, 12, 918281), datetime.datetime(2020, 3, 24, 9, 25, 12, 918313), None]) 2020-03-31 09:25:12,922 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:12,922 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:12,923 [538] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 10:25:12.923579 gunicorn-registry stdout | 2020-03-31 09:25:12,923 [538] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 10:25:12.923579 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:12,932 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:12,932 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:12,932 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:12,934 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,934 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:12,938 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:12,938 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) 2020-03-31 09:25:12,942 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,942 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) 2020-03-31 09:25:12,945 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,945 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) 2020-03-31 09:25:12,949 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,949 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:12,952 [555] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:12,952 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,952 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:12,952 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:12,953 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,953 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) 2020-03-31 09:25:12,957 [555] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'678bb755-8858-455e-ad63-4d6503444bc8', 2221157, '[[3859691389, 1658786821, 4109600805, 1765523095, 526640288, 61365303, 1797148447, 1679396220], 17769256, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, 9315840, datetime.datetime(2020, 3, 31, 9, 25, 12, 500523), '[[1748105183, 2675024539, 2253341262, 1786249621, 1609115271], 992040, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1]', 'xMSAi/YmMbWi743r+3oX53SSghdQCOYIgrgARzjuFEAfAXOQ2WmeCuWlAI4S+5OFSAeAK5uoIa+jtiT9BraqQl2TIxT9pN5u+0BMxi1z3Ys=', 1]) gunicorn-registry stdout | 2020-03-31 09:25:12,957 [555] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'678bb755-8858-455e-ad63-4d6503444bc8', 2221157, '[[3859691389, 1658786821, 4109600805, 1765523095, 526640288, 61365303, 1797148447, 1679396220], 17769256, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, 9315840, datetime.datetime(2020, 3, 31, 9, 25, 12, 500523), '[[1748105183, 2675024539, 2253341262, 1786249621, 1609115271], 992040, 0, "0e9aQRA09K/ud63+pU38ZHuyPdn+5bb/NwAA//95JlEMACaOAA==", 1]', 'xMSAi/YmMbWi743r+3oX53SSghdQCOYIgrgARzjuFEAfAXOQ2WmeCuWlAI4S+5OFSAeAK5uoIa+jtiT9BraqQl2TIxT9pN5u+0BMxi1z3Ys=', 1]) 2020-03-31 09:25:12,963 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:12,963 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:12,965 [555] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:12,965 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:12,966 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:12,966 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:12,966 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:862944c6-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:12 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:12,967 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '862944c6-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:+YTjYnm/ZSs39NFp+Ap9XtAyo5Y1TjjKAbHWHwaqdLM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:12,966 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:862944c6-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:12 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:12,967 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '862944c6-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:+YTjYnm/ZSs39NFp+Ap9XtAyo5Y1TjjKAbHWHwaqdLM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. 2020-03-31 09:25:12,969 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 gunicorn-registry stdout | 2020-03-31 09:25:12,969 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 2020-03-31 09:25:12,971 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:12,971 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:12,973 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 gunicorn-registry stdout | 2020-03-31 09:25:12,973 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:12,974 [555] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.00019812583923339844}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} gunicorn-registry stdout | 2020-03-31 09:25:12,974 [555] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.00019812583923339844}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} 2020-03-31 09:25:13,088 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 gunicorn-registry stdout | 2020-03-31 09:25:13,088 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:13,088 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c31b-a01e-005d-0a3e-07d5eb000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '862944c6-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:+YTjYnm/ZSs39NFp+Ap9XtAyo5Y1TjjKAbHWHwaqdLM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,088 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c31b-a01e-005d-0a3e-07d5eb000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '862944c6-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:+YTjYnm/ZSs39NFp+Ap9XtAyo5Y1TjjKAbHWHwaqdLM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:12 GMT'}. 2020-03-31 09:25:13,097 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c31b-a01e-005d-0a3e-07d5eb000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. gunicorn-registry stdout | 2020-03-31 09:25:13,097 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c31b-a01e-005d-0a3e-07d5eb000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:13,097 [555] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c31b-a01e-005d-0a3e-07d5eb000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. gunicorn-registry stdout | 2020-03-31 09:25:13,097 [555] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=862944c6-7331-11ea-ba5e-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c31b-a01e-005d-0a3e-07d5eb000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:13,098 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:863d6a6e-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 comp:blocklist 2020-03-31 09:25:13,098 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863d6a6e-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '863d6a6e-7331-11ea-ba5e-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3D6WGeXnJcUtC4ODX/blzGcGNG/vrqCo/J4JZIBsvlw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,098 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:863d6a6e-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 comp:blocklist 2020-03-31 09:25:13,098 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863d6a6e-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '863d6a6e-7331-11ea-ba5e-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3D6WGeXnJcUtC4ODX/blzGcGNG/vrqCo/J4JZIBsvlw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,109 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:13,109 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?comp=blocklist HTTP/1.1" 201 0 2020-03-31 09:25:13,110 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863d6a6e-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c322-a01e-005d-0f3e-07d5eb000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '863d6a6e-7331-11ea-ba5e-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3D6WGeXnJcUtC4ODX/blzGcGNG/vrqCo/J4JZIBsvlw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,110 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863d6a6e-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c322-a01e-005d-0f3e-07d5eb000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '863d6a6e-7331-11ea-ba5e-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3D6WGeXnJcUtC4ODX/blzGcGNG/vrqCo/J4JZIBsvlw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,111 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:863f6db4-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 comp:properties gunicorn-registry stdout | 2020-03-31 09:25:13,111 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:863f6db4-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 comp:properties 2020-03-31 09:25:13,111 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863f6db4-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '863f6db4-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cqZZh5B25WFl+qF4SwAqiqSyvqHUezW2fWkglT3f/qc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,111 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863f6db4-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '863f6db4-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cqZZh5B25WFl+qF4SwAqiqSyvqHUezW2fWkglT3f/qc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,119 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?comp=properties HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:13,119 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?comp=properties HTTP/1.1" 200 0 2020-03-31 09:25:13,120 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863f6db4-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c325-a01e-005d-123e-07d5eb000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '863f6db4-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cqZZh5B25WFl+qF4SwAqiqSyvqHUezW2fWkglT3f/qc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,120 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=863f6db4-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c325-a01e-005d-123e-07d5eb000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '863f6db4-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cqZZh5B25WFl+qF4SwAqiqSyvqHUezW2fWkglT3f/qc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,121 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:8640da28-7331-11ea-ba5e-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?sr=b&sp=r&sig=oHo8KXGquFyd3c2M%2BcyNSt5NRC3JxNnNmghV8PpMNV0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A13Z x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:25:13,121 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8640da28-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?sr=b&sp=r&sig=oHo8KXGquFyd3c2M%2BcyNSt5NRC3JxNnNmghV8PpMNV0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A13Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Pg0st+TpKQ1DTQoe9KYxFvRlkb6gcCeNzpW5Nq7KK34=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8640da28-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,121 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:8640da28-7331-11ea-ba5e-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?sr=b&sp=r&sig=oHo8KXGquFyd3c2M%2BcyNSt5NRC3JxNnNmghV8PpMNV0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A13Z x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:25:13,121 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8640da28-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?sr=b&sp=r&sig=oHo8KXGquFyd3c2M%2BcyNSt5NRC3JxNnNmghV8PpMNV0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A13Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Pg0st+TpKQ1DTQoe9KYxFvRlkb6gcCeNzpW5Nq7KK34=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8640da28-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,260 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:13,260 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 2020-03-31 09:25:13,261 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8640da28-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c326-a01e-005d-133e-07d5eb000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?sr=b&sp=r&sig=oHo8KXGquFyd3c2M%2BcyNSt5NRC3JxNnNmghV8PpMNV0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A13Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Pg0st+TpKQ1DTQoe9KYxFvRlkb6gcCeNzpW5Nq7KK34=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8640da28-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,261 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8640da28-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c326-a01e-005d-133e-07d5eb000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8?sr=b&sp=r&sig=oHo8KXGquFyd3c2M%2BcyNSt5NRC3JxNnNmghV8PpMNV0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A13Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Pg0st+TpKQ1DTQoe9KYxFvRlkb6gcCeNzpW5Nq7KK34=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8640da28-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,261 [555] [DEBUG] [storage.azurestorage] Deleting chunked upload 678bb755-8858-455e-ad63-4d6503444bc8 at path datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 gunicorn-registry stdout | 2020-03-31 09:25:13,261 [555] [DEBUG] [storage.azurestorage] Deleting chunked upload 678bb755-8858-455e-ad63-4d6503444bc8 at path datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 2020-03-31 09:25:13,262 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:86565f06-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 2020-03-31 09:25:13,262 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86565f06-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86565f06-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:LAsord3ymqL/vwcUZQux0RkHgM+0GlWO5i9B+zw/rYc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,262 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:86565f06-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 2020-03-31 09:25:13,262 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86565f06-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86565f06-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:LAsord3ymqL/vwcUZQux0RkHgM+0GlWO5i9B+zw/rYc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,270 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:13,270 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/678bb755-8858-455e-ad63-4d6503444bc8 HTTP/1.1" 202 0 2020-03-31 09:25:13,270 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86565f06-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c334-a01e-005d-1f3e-07d5eb000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86565f06-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:LAsord3ymqL/vwcUZQux0RkHgM+0GlWO5i9B+zw/rYc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,270 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86565f06-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:12 GMT, Server-Request-ID=6033c334-a01e-005d-1f3e-07d5eb000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86565f06-7331-11ea-ba5e-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:LAsord3ymqL/vwcUZQux0RkHgM+0GlWO5i9B+zw/rYc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,271 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,271 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'678bb755-8858-455e-ad63-4d6503444bc8', 1, 0]) 2020-03-31 09:25:13,277 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:13,277 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) 2020-03-31 09:25:13,280 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,280 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 1, 0]) 2020-03-31 09:25:13,283 [555] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'229ce905-1640-4594-aae8-fb66fcc5e51f', 2221157, 9315840, False, True, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438']) gunicorn-registry stdout | 2020-03-31 09:25:13,283 [555] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'229ce905-1640-4594-aae8-fb66fcc5e51f', 2221157, 9315840, False, True, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438']) 2020-03-31 09:25:13,286 [555] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [1, 10]) gunicorn-registry stdout | 2020-03-31 09:25:13,286 [555] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [1, 10]) 2020-03-31 09:25:13,288 [555] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'b32fd55c-5018-402a-9b45-5f6a883e89ad', 1, u'/', 1, False, -1]) gunicorn-registry stdout | 2020-03-31 09:25:13,288 [555] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'b32fd55c-5018-402a-9b45-5f6a883e89ad', 1, u'/', 1, False, -1]) 2020-03-31 09:25:13,292 [555] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'05e5010e-f7ce-4ef6-8ce4-e9960ea393d4', 1, 1, 1585646713, 1585650313, True, False]) gunicorn-registry stdout | 2020-03-31 09:25:13,292 [555] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'05e5010e-f7ce-4ef6-8ce4-e9960ea393d4', 1, 1, 1585646713, 1585650313, True, False]) 2020-03-31 09:25:13,294 [555] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [1]) gunicorn-registry stdout | 2020-03-31 09:25:13,294 [555] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [1]) 2020-03-31 09:25:13,296 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,296 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:13,298 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,298 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 1, 1, 0]) 2020-03-31 09:25:13,301 [555] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [1, 524288, 'xMSAi/YmMbWi743r+3oX53SSghdQCOYIgrgARzjuFEAfAXOQ2WmeCuWlAI4S+5OFSAeAK5uoIa+jtiT9BraqQl2TIxT9pN5u+0BMxi1z3YuJ9ZQcBzkFjp8+gvi3N6MXAX7+EA==']) gunicorn-registry stdout | 2020-03-31 09:25:13,301 [555] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [1, 524288, 'xMSAi/YmMbWi743r+3oX53SSghdQCOYIgrgARzjuFEAfAXOQ2WmeCuWlAI4S+5OFSAeAK5uoIa+jtiT9BraqQl2TIxT9pN5u+0BMxi1z3YuJ9ZQcBzkFjp8+gvi3N6MXAX7+EA==']) 2020-03-31 09:25:13,308 [555] [DEBUG] [app] Ending request: urn:request:09858b8d-5284-4301-b7ab-fbdd5112fd99 (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) gunicorn-registry stdout | 2020-03-31 09:25:13,308 [555] [DEBUG] [app] Ending request: urn:request:09858b8d-5284-4301-b7ab-fbdd5112fd99 (/v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8) 2020-03-31 09:25:13,308 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,308 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,308 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,308 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,309 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8?digest=sha256%3A070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8?digest=sha256%3A070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.394 126 0.394) gunicorn-registry stdout | 2020-03-31 09:25:13,309 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8?digest=sha256%3A070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/678bb755-8858-455e-ad63-4d6503444bc8?digest=sha256%3A070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.394 126 0.394) 2020-03-31 09:25:13,311 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,311 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,312 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:13,312 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,313 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:13,314 [538] [DEBUG] [app] Starting request: urn:request:c23f36d4-2abf-4da7-91f7-ab26074fde46 (/v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438) gunicorn-registry stdout | 2020-03-31 09:25:13,313 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:13,314 [538] [DEBUG] [app] Starting request: urn:request:c23f36d4-2abf-4da7-91f7-ab26074fde46 (/v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438) 2020-03-31 09:25:13,314 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,314 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,314 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:13,314 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,314 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,314 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,317 [538] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:13,317 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,317 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,317 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,318 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,318 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,318 [538] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2 gunicorn-registry stdout | 2020-03-31 09:25:13,317 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,317 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,318 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,318 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,318 [538] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2 2020-03-31 09:25:13,320 [538] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2; calling loader gunicorn-registry stdout | 2020-03-31 09:25:13,320 [538] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2; calling loader 2020-03-31 09:25:13,322 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,322 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,327 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,327 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', False, 1, 0]) 2020-03-31 09:25:13,333 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,333 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', False, 1, 0]) 2020-03-31 09:25:13,340 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) gunicorn-registry stdout | 2020-03-31 09:25:13,340 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) 2020-03-31 09:25:13,345 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:13,345 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) 2020-03-31 09:25:13,349 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) gunicorn-registry stdout | 2020-03-31 09:25:13,349 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) 2020-03-31 09:25:13,352 [538] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438'}, 'compressed_size': 2221157, 'db_id': 1, 'uuid': u'229ce905-1640-4594-aae8-fb66fcc5e51f', 'uploading': False, 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'uncompressed_size': 9315840} gunicorn-registry stdout | 2020-03-31 09:25:13,352 [538] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438'}, 'compressed_size': 2221157, 'db_id': 1, 'uuid': u'229ce905-1640-4594-aae8-fb66fcc5e51f', 'uploading': False, 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'uncompressed_size': 9315840} 2020-03-31 09:25:13,352 [538] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438'}, 'compressed_size': 2221157, 'db_id': 1, 'uuid': u'229ce905-1640-4594-aae8-fb66fcc5e51f', 'uploading': False, 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'uncompressed_size': 9315840}: 60s 2020-03-31 09:25:13,353 [538] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438'}, 'compressed_size': 2221157, 'db_id': 1, 'uuid': u'229ce905-1640-4594-aae8-fb66fcc5e51f', 'uploading': False, 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'uncompressed_size': 9315840}: 60s gunicorn-registry stdout | 2020-03-31 09:25:13,352 [538] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438'}, 'compressed_size': 2221157, 'db_id': 1, 'uuid': u'229ce905-1640-4594-aae8-fb66fcc5e51f', 'uploading': False, 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'uncompressed_size': 9315840}: 60s 2020-03-31 09:25:13,353 [538] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438'}, 'compressed_size': 2221157, 'db_id': 1, 'uuid': u'229ce905-1640-4594-aae8-fb66fcc5e51f', 'uploading': False, 'digest': u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 'uncompressed_size': 9315840}: 60s 2020-03-31 09:25:13,354 [538] [DEBUG] [app] Ending request: urn:request:c23f36d4-2abf-4da7-91f7-ab26074fde46 (/v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438) gunicorn-registry stdout | 2020-03-31 09:25:13,354 [538] [DEBUG] [app] Ending request: urn:request:c23f36d4-2abf-4da7-91f7-ab26074fde46 (/v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438) 2020-03-31 09:25:13,354 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,354 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,354 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.042 86 0.042) gunicorn-registry stdout | 2020-03-31 09:25:13,354 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,354 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,354 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.042 86 0.042) 2020-03-31 09:25:13,357 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,357 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,358 [556] [DEBUG] [app] Starting request: urn:request:420ca77b-8079-4da1-b7bb-d94726596535 (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) gunicorn-registry stdout | 2020-03-31 09:25:13,358 [556] [DEBUG] [app] Starting request: urn:request:420ca77b-8079-4da1-b7bb-d94726596535 (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) 2020-03-31 09:25:13,358 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,358 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,358 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,358 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:13,358 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,358 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,358 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,358 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,359 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:13,359 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,361 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,361 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,361 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,361 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 gunicorn-registry stdout | 2020-03-31 09:25:13,361 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 2020-03-31 09:25:13,363 [556] [DEBUG] [app] Starting request: urn:request:7f63084a-4384-43fb-853f-be290a123f9c (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) 2020-03-31 09:25:13,363 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'repo_name': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:25:13,363 [556] [DEBUG] [app] Starting request: urn:request:7f63084a-4384-43fb-853f-be290a123f9c (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) 2020-03-31 09:25:13,363 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,363 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,363 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:13,363 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,363 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,365 [556] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:13,365 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,366 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,366 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,366 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,366 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,366 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 gunicorn-registry stdout | 2020-03-31 09:25:13,366 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,366 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,366 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,366 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,366 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 2020-03-31 09:25:13,367 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2; calling loader gunicorn-registry stdout | 2020-03-31 09:25:13,367 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2; calling loader 2020-03-31 09:25:13,369 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,369 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,374 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,374 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) 2020-03-31 09:25:13,380 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,380 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) 2020-03-31 09:25:13,384 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2: None gunicorn-registry stdout | 2020-03-31 09:25:13,384 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2: None 2020-03-31 09:25:13,384 [556] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2: None 2020-03-31 09:25:13,384 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]} gunicorn-registry stdout | 2020-03-31 09:25:13,384 [556] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2: None 2020-03-31 09:25:13,384 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]} 2020-03-31 09:25:13,385 [556] [DEBUG] [app] Ending request: urn:request:420ca77b-8079-4da1-b7bb-d94726596535 (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) 2020-03-31 09:25:13,386 [556] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:13,385 [556] [DEBUG] [app] Ending request: urn:request:420ca77b-8079-4da1-b7bb-d94726596535 (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) 2020-03-31 09:25:13,386 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,386 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,386 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 404 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/2.0" 404 0 "-" "Go-http-client/2.0" (0.030 81 0.030) gunicorn-registry stdout | 2020-03-31 09:25:13,386 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,386 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 404 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/2.0" 404 0 "-" "Go-http-client/2.0" (0.030 81 0.030) 2020-03-31 09:25:13,389 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,389 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,389 [555] [DEBUG] [app] Starting request: urn:request:0693ed06-d83f-4ccf-ba33-9b8e459b4918 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:13,389 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,389 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2; calling loader 2020-03-31 09:25:13,389 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,389 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:13,389 [555] [DEBUG] [app] Starting request: urn:request:0693ed06-d83f-4ccf-ba33-9b8e459b4918 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:13,389 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,389 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2; calling loader 2020-03-31 09:25:13,389 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,389 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,393 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,393 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,395 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,395 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,395 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,397 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,397 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,398 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,398 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) 2020-03-31 09:25:13,402 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,402 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:13,403 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,403 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) 2020-03-31 09:25:13,406 [555] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 13, 405460)]) gunicorn-registry stdout | 2020-03-31 09:25:13,406 [555] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 13, 405460)]) 2020-03-31 09:25:13,407 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2: None 2020-03-31 09:25:13,407 [556] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2: None gunicorn-registry stdout | 2020-03-31 09:25:13,407 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2: None 2020-03-31 09:25:13,407 [556] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2: None 2020-03-31 09:25:13,408 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]} gunicorn-registry stdout | 2020-03-31 09:25:13,408 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]} 2020-03-31 09:25:13,409 [556] [DEBUG] [app] Ending request: urn:request:7f63084a-4384-43fb-853f-be290a123f9c (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) 2020-03-31 09:25:13,409 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,410 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,409 [556] [DEBUG] [app] Ending request: urn:request:7f63084a-4384-43fb-853f-be290a123f9c (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) 2020-03-31 09:25:13,409 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,410 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,410 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 404 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/2.0" 404 0 "-" "Go-http-client/2.0" (0.054 80 0.054) gunicorn-registry stdout | 2020-03-31 09:25:13,410 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 404 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/2.0" 404 0 "-" "Go-http-client/2.0" (0.054 80 0.054) 2020-03-31 09:25:13,411 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,412 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:13,411 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,413 [555] [DEBUG] [app] Ending request: urn:request:0693ed06-d83f-4ccf-ba33-9b8e459b4918 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:13,414 [538] [DEBUG] [app] Starting request: urn:request:b3a7558c-9fb6-40a0-9adc-fe804697b763 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:13,414 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,414 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,414 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,414 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,414 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,412 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:25:13,415 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:13,415 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.028 9 0.028) gunicorn-registry stdout | 2020-03-31 09:25:13,413 [555] [DEBUG] [app] Ending request: urn:request:0693ed06-d83f-4ccf-ba33-9b8e459b4918 (/v2/admin/redhat-operators/blobs/uploads/) gunicorn-registry stdout | 2020-03-31 09:25:13,414 [538] [DEBUG] [app] Starting request: urn:request:b3a7558c-9fb6-40a0-9adc-fe804697b763 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:13,414 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,414 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,414 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,414 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,417 [556] [DEBUG] [app] Starting request: urn:request:494ad480-63d8-4700-8749-10ad036137fe (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) 2020-03-31 09:25:13,418 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,418 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,418 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,418 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:13,414 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,415 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:13,415 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,419 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,420 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.028 9 0.028) 2020-03-31 09:25:13,420 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,420 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,420 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,420 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,420 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin gunicorn-registry stdout | 2020-03-31 09:25:13,417 [556] [DEBUG] [app] Starting request: urn:request:494ad480-63d8-4700-8749-10ad036137fe (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) 2020-03-31 09:25:13,418 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,418 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,418 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,418 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,420 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,421 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,421 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,421 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:13,419 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,420 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin gunicorn-registry stdout | 2020-03-31 09:25:13,420 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,420 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,420 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,420 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,420 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,422 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:13,423 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,420 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,421 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,421 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,421 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:13,422 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:13,423 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,424 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,424 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,428 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,428 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:13,429 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,429 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) 2020-03-31 09:25:13,431 [538] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 13, 431219)]) gunicorn-registry stdout | 2020-03-31 09:25:13,431 [538] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 13, 431219)]) 2020-03-31 09:25:13,432 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,432 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:13,435 [556] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:13,435 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,435 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,435 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,436 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:13,436 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,437 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:13,437 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:25:13,438 [538] [DEBUG] [app] Ending request: urn:request:b3a7558c-9fb6-40a0-9adc-fe804697b763 (/v2/admin/redhat-operators/blobs/uploads/) gunicorn-registry stdout | 2020-03-31 09:25:13,438 [538] [DEBUG] [app] Ending request: urn:request:b3a7558c-9fb6-40a0-9adc-fe804697b763 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:13,439 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,439 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,439 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,439 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,439 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.028 9 0.028) gunicorn-registry stdout | 2020-03-31 09:25:13,439 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.028 9 0.028) 2020-03-31 09:25:13,442 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,442 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,443 [555] [DEBUG] [app] Starting request: urn:request:b1275116-7a7a-4b20-8d9f-426fc1d6ea96 (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) 2020-03-31 09:25:13,443 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,443 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 'repo_name': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:25:13,443 [555] [DEBUG] [app] Starting request: urn:request:b1275116-7a7a-4b20-8d9f-426fc1d6ea96 (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) 2020-03-31 09:25:13,443 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,443 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,443 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,444 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,444 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:13,443 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,444 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,444 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,447 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,447 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,447 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,450 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,450 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,455 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,455 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) 2020-03-31 09:25:13,459 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,459 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:13,462 [555] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:13,462 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,462 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,462 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,758 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 3405480 yLIWbYP030V+dEfpSsZqAQ== x-ms-client-request-id:86a21a90-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc blockid:MDAwMDA= comp:block gunicorn-registry stdout | 2020-03-31 09:25:13,758 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 3405480 yLIWbYP030V+dEfpSsZqAQ== x-ms-client-request-id:86a21a90-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc blockid:MDAwMDA= comp:block 2020-03-31 09:25:13,758 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86a21a90-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '3405480', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86a21a90-7331-11ea-b9ed-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'yLIWbYP030V+dEfpSsZqAQ==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uIdBrUTBn7rEumqZTFURpzlnBQb8VSHnINhXYUZLF0E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,758 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86a21a90-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '3405480', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86a21a90-7331-11ea-b9ed-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'yLIWbYP030V+dEfpSsZqAQ==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uIdBrUTBn7rEumqZTFURpzlnBQb8VSHnINhXYUZLF0E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,762 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 gunicorn-registry stdout | 2020-03-31 09:25:13,762 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 2020-03-31 09:25:13,865 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 2020-03-31 09:25:13,866 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86a21a90-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d1d-b01e-006e-213e-078a40000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '3405480', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86a21a90-7331-11ea-b9ed-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'yLIWbYP030V+dEfpSsZqAQ==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uIdBrUTBn7rEumqZTFURpzlnBQb8VSHnINhXYUZLF0E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,865 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 2020-03-31 09:25:13,868 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,866 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86a21a90-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d1d-b01e-006e-213e-078a40000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '3405480', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86a21a90-7331-11ea-b9ed-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'yLIWbYP030V+dEfpSsZqAQ==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uIdBrUTBn7rEumqZTFURpzlnBQb8VSHnINhXYUZLF0E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,868 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) 2020-03-31 09:25:13,874 [556] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 3405480, '[[2214649058, 2114237976, 1683316375, 2093077682, 2404037178, 2342092931, 3606656618, 2574221943], 27243840, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, 9158656, datetime.datetime(2020, 3, 31, 9, 25, 13, 405460), '[[705806375, 481301370, 734879828, 318090407, 1941970536], 2078016, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1]', 'foGiDKWuuYWy+SpBLhHWdaxCYw/RY7bi+x22Rp6K6bcHHtanlJ5brNf3GlPdbJ3NNZS0QUILwHiwuLl+Ey3+af0S6k38UC6jSDHvLiMvIOrElTDsnIDNPBHCyRLM0euXtOpWQbIY2f45iCxAvDlkOgUz/Q1csuGq', 2]) gunicorn-registry stdout | 2020-03-31 09:25:13,874 [556] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 3405480, '[[2214649058, 2114237976, 1683316375, 2093077682, 2404037178, 2342092931, 3606656618, 2574221943], 27243840, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, 9158656, datetime.datetime(2020, 3, 31, 9, 25, 13, 405460), '[[705806375, 481301370, 734879828, 318090407, 1941970536], 2078016, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1]', 'foGiDKWuuYWy+SpBLhHWdaxCYw/RY7bi+x22Rp6K6bcHHtanlJ5brNf3GlPdbJ3NNZS0QUILwHiwuLl+Ey3+af0S6k38UC6jSDHvLiMvIOrElTDsnIDNPBHCyRLM0euXtOpWQbIY2f45iCxAvDlkOgUz/Q1csuGq', 2]) 2020-03-31 09:25:13,882 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,882 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:13,886 [556] [DEBUG] [app] Ending request: urn:request:494ad480-63d8-4700-8749-10ad036137fe (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) gunicorn-registry stdout | 2020-03-31 09:25:13,886 [556] [DEBUG] [app] Ending request: urn:request:494ad480-63d8-4700-8749-10ad036137fe (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) 2020-03-31 09:25:13,887 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,887 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,887 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,887 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,888 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc HTTP/1.1" 204 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.472 3405543 0.472) gunicorn-registry stdout | 2020-03-31 09:25:13,888 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:13 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc HTTP/1.1" 204 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:13 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.472 3405543 0.472) 2020-03-31 09:25:13,893 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,893 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,893 [538] [DEBUG] [app] Starting request: urn:request:adeebbf1-50ba-4b27-9315-43cffc64511d (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) 2020-03-31 09:25:13,894 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,894 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,894 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,896 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:13,897 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,897 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,897 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,898 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,898 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,899 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,893 [538] [DEBUG] [app] Starting request: urn:request:adeebbf1-50ba-4b27-9315-43cffc64511d (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) 2020-03-31 09:25:13,894 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:13,894 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,894 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:13,901 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,896 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:13,897 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:13,897 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:13,897 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:13,898 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:13,898 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:13,899 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,901 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:13,905 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,906 [556] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 10:25:13.906015 2020-03-31 09:25:13,906 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,905 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:13,906 [556] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 10:25:13.906015 gunicorn-registry stdout | 2020-03-31 09:25:13,906 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) 2020-03-31 09:25:13,911 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,911 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:13,913 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,914 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,913 [538] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:13,914 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,915 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,915 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) 2020-03-31 09:25:13,919 [538] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 3405480, '[[2214649058, 2114237976, 1683316375, 2093077682, 2404037178, 2342092931, 3606656618, 2574221943], 27243840, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, 9158656, datetime.datetime(2020, 3, 31, 9, 25, 13, 405460), '[[705806375, 481301370, 734879828, 318090407, 1941970536], 2078016, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1]', 'foGiDKWuuYWy+SpBLhHWdaxCYw/RY7bi+x22Rp6K6bcHHtanlJ5brNf3GlPdbJ3NNZS0QUILwHiwuLl+Ey3+af0S6k38UC6jSDHvLiMvIOrElTDsnIDNPBHCyRLM0euXtOpWQbIY2f45iCxAvDlkOgUz/Q1csuGq', 2]) gunicorn-registry stdout | 2020-03-31 09:25:13,919 [538] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 3405480, '[[2214649058, 2114237976, 1683316375, 2093077682, 2404037178, 2342092931, 3606656618, 2574221943], 27243840, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, 9158656, datetime.datetime(2020, 3, 31, 9, 25, 13, 405460), '[[705806375, 481301370, 734879828, 318090407, 1941970536], 2078016, 0, "q2eaZtqXrr1rmoL+3//97//+93//+z/wf/9vAAAA//+ew9WmAMCLAA==", 1]', 'foGiDKWuuYWy+SpBLhHWdaxCYw/RY7bi+x22Rp6K6bcHHtanlJ5brNf3GlPdbJ3NNZS0QUILwHiwuLl+Ey3+af0S6k38UC6jSDHvLiMvIOrElTDsnIDNPBHCyRLM0euXtOpWQbIY2f45iCxAvDlkOgUz/Q1csuGq', 2]) 2020-03-31 09:25:13,925 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:13,929 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,929 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:13,930 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:86bc4460-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:13,931 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86bc4460-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hDxiidS5NC+lbCEVF+TufKS7w7WeUQ323FCkaR6gumU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,925 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:13,929 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:13,929 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:13,930 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:86bc4460-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:13,931 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86bc4460-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hDxiidS5NC+lbCEVF+TufKS7w7WeUQ323FCkaR6gumU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,940 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:13,945 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:13,947 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1109-501e-0049-6e3e-079d84000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86bc4460-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hDxiidS5NC+lbCEVF+TufKS7w7WeUQ323FCkaR6gumU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,949 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1109-501e-0049-6e3e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:13,949 [538] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1109-501e-0049-6e3e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:13,951 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:86bf9200-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc comp:blocklist 2020-03-31 09:25:13,951 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bf9200-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86bf9200-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:HJoZGjtMgarvFBfV09LJV9GPsn9f4M6rMB0M5QOsgJU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,940 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:13,945 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:13,967 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:13,969 [538] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.00014090538024902344}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} 2020-03-31 09:25:13,978 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:13,947 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1109-501e-0049-6e3e-079d84000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86bc4460-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hDxiidS5NC+lbCEVF+TufKS7w7WeUQ323FCkaR6gumU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,979 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bf9200-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c110d-501e-0049-723e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86bf9200-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:HJoZGjtMgarvFBfV09LJV9GPsn9f4M6rMB0M5QOsgJU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,949 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1109-501e-0049-6e3e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:13,949 [538] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=86bc4460-7331-11ea-bc11-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1109-501e-0049-6e3e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. gunicorn-registry stdout | 2020-03-31 09:25:13,951 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:86bf9200-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc comp:blocklist 2020-03-31 09:25:13,951 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bf9200-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86bf9200-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:HJoZGjtMgarvFBfV09LJV9GPsn9f4M6rMB0M5QOsgJU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,967 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 gunicorn-registry stdout | 2020-03-31 09:25:13,969 [538] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.00014090538024902344}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} gunicorn-registry stdout | 2020-03-31 09:25:13,978 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:13,979 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86bf9200-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c110d-501e-0049-723e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86bf9200-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:HJoZGjtMgarvFBfV09LJV9GPsn9f4M6rMB0M5QOsgJU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:13,985 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:86c4c84c-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc comp:properties 2020-03-31 09:25:13,986 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c4c84c-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86c4c84c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:leKOukKyghWVDA2Lh3zrvmZeFUYcu1qpxX9EG4RADAU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:13,985 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:86c4c84c-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:13 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc comp:properties gunicorn-registry stdout | 2020-03-31 09:25:13,986 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c4c84c-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86c4c84c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:leKOukKyghWVDA2Lh3zrvmZeFUYcu1qpxX9EG4RADAU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:14,009 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?comp=properties HTTP/1.1" 200 0 2020-03-31 09:25:14,010 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c4c84c-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1111-501e-0049-763e-079d84000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86c4c84c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:leKOukKyghWVDA2Lh3zrvmZeFUYcu1qpxX9EG4RADAU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,009 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?comp=properties HTTP/1.1" 200 0 2020-03-31 09:25:14,010 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c4c84c-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1111-501e-0049-763e-079d84000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86c4c84c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:leKOukKyghWVDA2Lh3zrvmZeFUYcu1qpxX9EG4RADAU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:13 GMT'}. 2020-03-31 09:25:14,014 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:86c92144-7331-11ea-bc11-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?sr=b&sp=r&sig=GxnclsONxhqnQcPFrudru7P5IbbLB4tJX%2BtgcF3W0%2Bc%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:25:14,014 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c92144-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?sr=b&sp=r&sig=GxnclsONxhqnQcPFrudru7P5IbbLB4tJX%2BtgcF3W0%2Bc%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:K65bnhGKAkHbbSsjvmcksPG1ZxIhBjg8g0FDqt1tHrI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86c92144-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,014 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:86c92144-7331-11ea-bc11-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?sr=b&sp=r&sig=GxnclsONxhqnQcPFrudru7P5IbbLB4tJX%2BtgcF3W0%2Bc%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:25:14,014 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c92144-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?sr=b&sp=r&sig=GxnclsONxhqnQcPFrudru7P5IbbLB4tJX%2BtgcF3W0%2Bc%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:K65bnhGKAkHbbSsjvmcksPG1ZxIhBjg8g0FDqt1tHrI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86c92144-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,057 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:14,057 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 2020-03-31 09:25:14,061 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c92144-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1112-501e-0049-773e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?sr=b&sp=r&sig=GxnclsONxhqnQcPFrudru7P5IbbLB4tJX%2BtgcF3W0%2Bc%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:K65bnhGKAkHbbSsjvmcksPG1ZxIhBjg8g0FDqt1tHrI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86c92144-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,061 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86c92144-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1112-501e-0049-773e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?sr=b&sp=r&sig=GxnclsONxhqnQcPFrudru7P5IbbLB4tJX%2BtgcF3W0%2Bc%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:K65bnhGKAkHbbSsjvmcksPG1ZxIhBjg8g0FDqt1tHrI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86c92144-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,065 [538] [DEBUG] [storage.azurestorage] Deleting chunked upload 04ed9c74-b528-43a1-a2ec-4d53eb5554bc at path datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc gunicorn-registry stdout | 2020-03-31 09:25:14,065 [538] [DEBUG] [storage.azurestorage] Deleting chunked upload 04ed9c74-b528-43a1-a2ec-4d53eb5554bc at path datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc 2020-03-31 09:25:14,070 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:86d111e2-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc 2020-03-31 09:25:14,070 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86d111e2-7331-11ea-bc11-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86d111e2-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:yt2fjj6GF9VT27XuOllVL8nXHAOnmwxn9VIicHctLTU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,070 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:86d111e2-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc 2020-03-31 09:25:14,070 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86d111e2-7331-11ea-bc11-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86d111e2-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:yt2fjj6GF9VT27XuOllVL8nXHAOnmwxn9VIicHctLTU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,082 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:14,082 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc HTTP/1.1" 202 0 2020-03-31 09:25:14,086 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86d111e2-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1118-501e-0049-7d3e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86d111e2-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:yt2fjj6GF9VT27XuOllVL8nXHAOnmwxn9VIicHctLTU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,086 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86d111e2-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1118-501e-0049-7d3e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86d111e2-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:yt2fjj6GF9VT27XuOllVL8nXHAOnmwxn9VIicHctLTU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,092 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,092 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'04ed9c74-b528-43a1-a2ec-4d53eb5554bc', 1, 0]) 2020-03-31 09:25:14,097 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,097 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 1, 0]) 2020-03-31 09:25:14,104 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 3405480, 9158656, False, True, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29']) gunicorn-registry stdout | 2020-03-31 09:25:14,104 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 3405480, 9158656, False, True, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29']) 2020-03-31 09:25:14,107 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [2, 10]) gunicorn-registry stdout | 2020-03-31 09:25:14,107 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [2, 10]) 2020-03-31 09:25:14,111 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'dd3b0723-1ea2-44b7-ad13-8d2b207b61b3', 1, u'/', 2, False, -1]) gunicorn-registry stdout | 2020-03-31 09:25:14,111 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'dd3b0723-1ea2-44b7-ad13-8d2b207b61b3', 1, u'/', 2, False, -1]) 2020-03-31 09:25:14,114 [538] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'169254b7-5465-479f-9474-dd66ff214255', 2, 1, 1585646714, 1585650314, True, False]) gunicorn-registry stdout | 2020-03-31 09:25:14,114 [538] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'169254b7-5465-479f-9474-dd66ff214255', 2, 1, 1585646714, 1585650314, True, False]) 2020-03-31 09:25:14,118 [538] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [2]) gunicorn-registry stdout | 2020-03-31 09:25:14,118 [538] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [2]) 2020-03-31 09:25:14,121 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,121 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) 2020-03-31 09:25:14,123 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 2, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,123 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 2, 1, 0]) 2020-03-31 09:25:14,126 [538] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [2, 524288, 'foGiDKWuuYWy+SpBLhHWdaxCYw/RY7bi+x22Rp6K6bcHHtanlJ5brNf3GlPdbJ3NNZS0QUILwHiwuLl+Ey3+af0S6k38UC6jSDHvLiMvIOrElTDsnIDNPBHCyRLM0euXtOpWQbIY2f45iCxAvDlkOgUz/Q1csuGqVZ0jVJ8tQb31zQLO/SXAcgA4KbY=']) gunicorn-registry stdout | 2020-03-31 09:25:14,126 [538] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [2, 524288, 'foGiDKWuuYWy+SpBLhHWdaxCYw/RY7bi+x22Rp6K6bcHHtanlJ5brNf3GlPdbJ3NNZS0QUILwHiwuLl+Ey3+af0S6k38UC6jSDHvLiMvIOrElTDsnIDNPBHCyRLM0euXtOpWQbIY2f45iCxAvDlkOgUz/Q1csuGqVZ0jVJ8tQb31zQLO/SXAcgA4KbY=']) 2020-03-31 09:25:14,133 [538] [DEBUG] [app] Ending request: urn:request:adeebbf1-50ba-4b27-9315-43cffc64511d (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) 2020-03-31 09:25:14,134 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,135 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?digest=sha256%3Ae4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.246 121 0.246) 2020-03-31 09:25:14,137 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?digest=sha256%3Ae4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" gunicorn-registry stdout | 2020-03-31 09:25:14,133 [538] [DEBUG] [app] Ending request: urn:request:adeebbf1-50ba-4b27-9315-43cffc64511d (/v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc) 2020-03-31 09:25:14,143 [556] [DEBUG] [app] Starting request: urn:request:b2e102d5-0e29-4b65-a94d-725005c0ce7f (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) 2020-03-31 09:25:14,143 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,145 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,146 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 9506905 Y+ixV9OyhKULe0Qsc+sxHA== x-ms-client-request-id:86dd3eae-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 blockid:MDAwMDA= comp:block 2020-03-31 09:25:14,146 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86dd3eae-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '9506905', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86dd3eae-7331-11ea-ba5e-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'Y+ixV9OyhKULe0Qsc+sxHA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:9av5EXgc4HPs4ioVzGH9gPLunBjJLQwxVDwNfx3r3VM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,144 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,147 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:14,134 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,135 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?digest=sha256%3Ae4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.246 121 0.246) gunicorn-registry stdout | 2020-03-31 09:25:14,137 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/04ed9c74-b528-43a1-a2ec-4d53eb5554bc?digest=sha256%3Ae4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,151 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,152 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 gunicorn-registry stdout | 2020-03-31 09:25:14,143 [556] [DEBUG] [app] Starting request: urn:request:b2e102d5-0e29-4b65-a94d-725005c0ce7f (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) 2020-03-31 09:25:14,143 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,153 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,154 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,145 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,146 [555] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 9506905 Y+ixV9OyhKULe0Qsc+sxHA== x-ms-client-request-id:86dd3eae-7331-11ea-ba5e-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 blockid:MDAwMDA= comp:block 2020-03-31 09:25:14,146 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86dd3eae-7331-11ea-ba5e-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '9506905', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86dd3eae-7331-11ea-ba5e-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'Y+ixV9OyhKULe0Qsc+sxHA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:9av5EXgc4HPs4ioVzGH9gPLunBjJLQwxVDwNfx3r3VM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,144 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,147 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,156 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2; calling loader gunicorn-registry stdout | 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,151 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,151 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,152 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 gunicorn-registry stdout | 2020-03-31 09:25:14,153 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,154 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,156 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2; calling loader 2020-03-31 09:25:14,159 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,159 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,165 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,165 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) 2020-03-31 09:25:14,170 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,170 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', False, 1, 0]) 2020-03-31 09:25:14,176 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) gunicorn-registry stdout | 2020-03-31 09:25:14,176 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) 2020-03-31 09:25:14,181 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) gunicorn-registry stdout | 2020-03-31 09:25:14,181 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) 2020-03-31 09:25:14,185 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29'}, 'compressed_size': 3405480, 'db_id': 2, 'uuid': u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 'uploading': False, 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'uncompressed_size': 9158656} gunicorn-registry stdout | 2020-03-31 09:25:14,185 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29'}, 'compressed_size': 3405480, 'db_id': 2, 'uuid': u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 'uploading': False, 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'uncompressed_size': 9158656} 2020-03-31 09:25:14,185 [556] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29'}, 'compressed_size': 3405480, 'db_id': 2, 'uuid': u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 'uploading': False, 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'uncompressed_size': 9158656}: 60s 2020-03-31 09:25:14,186 [556] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29'}, 'compressed_size': 3405480, 'db_id': 2, 'uuid': u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 'uploading': False, 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'uncompressed_size': 9158656}: 60s gunicorn-registry stdout | 2020-03-31 09:25:14,185 [556] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29'}, 'compressed_size': 3405480, 'db_id': 2, 'uuid': u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 'uploading': False, 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'uncompressed_size': 9158656}: 60s 2020-03-31 09:25:14,186 [556] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29'}, 'compressed_size': 3405480, 'db_id': 2, 'uuid': u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8', 'uploading': False, 'digest': u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 'uncompressed_size': 9158656}: 60s 2020-03-31 09:25:14,188 [556] [DEBUG] [app] Ending request: urn:request:b2e102d5-0e29-4b65-a94d-725005c0ce7f (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) 2020-03-31 09:25:14,190 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,190 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,190 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.051 6 0.051) 2020-03-31 09:25:14,195 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,188 [556] [DEBUG] [app] Ending request: urn:request:b2e102d5-0e29-4b65-a94d-725005c0ce7f (/v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29) gunicorn-registry stdout | 2020-03-31 09:25:14,190 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,190 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,190 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.051 6 0.051) gunicorn-registry stdout | 2020-03-31 09:25:14,195 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,199 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,199 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,201 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,201 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,249 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:14,249 [555] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 2020-03-31 09:25:14,250 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86dd3eae-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=6033c356-a01e-005d-3f3e-07d5eb000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '9506905', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86dd3eae-7331-11ea-ba5e-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'Y+ixV9OyhKULe0Qsc+sxHA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:9av5EXgc4HPs4ioVzGH9gPLunBjJLQwxVDwNfx3r3VM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,250 [555] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86dd3eae-7331-11ea-ba5e-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=6033c356-a01e-005d-3f3e-07d5eb000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '9506905', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86dd3eae-7331-11ea-ba5e-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': 'Y+ixV9OyhKULe0Qsc+sxHA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:9av5EXgc4HPs4ioVzGH9gPLunBjJLQwxVDwNfx3r3VM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,251 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,251 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) 2020-03-31 09:25:14,256 [555] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 9506905, '[[2552739968, 2710085537, 1375792233, 1622564603, 742735765, 2572181374, 1643169938, 1318722110], 76055240, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, 25587712, datetime.datetime(2020, 3, 31, 9, 25, 13, 431219), '[[3014677442, 2909707954, 2437205147, 3713134934, 3458891136], 557768, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1]', 'd8nYTKaCMVjsVdVlNz+WaUx5js8yJd4Y+l9nxrAfNWz2pnzU5UqzoF8oJKgGfxP3pAHYNRDOwyH6rkWDAftzkQtwnNeIOT1tj3NR0JUdUXi38hqAEQvtuudpY0V/tEZUPZHjGGK9j3fkBUrDV/wGKdSmOA5cq/Pt2ejgRtzr3athBiyf+ZP3pxp1ChQJeggQ13JZjMFz9ywNSjOoGo+jxQD2LWxO1IVKms4nRW5jZ6XZFlAf4V+3PZVvkxztVJZzrVyQR8YpSR+wygXv6/ap2XTjkNtEaiNfzVFxv9v3H1H8hOmQE1HGDUNnew9CR34NTrVWGXVunHd8wgQdcVwPYu6bagCF2lSlMD+PxuXnsQJdziOCrXybG+YIgXtBRIvMwiO2oIgyDTqS0j3JqWmA8uFBg06F/nTXNSjJ9STykBaTLlKlCM//cpzA4h6qQm3fghT7pywOFmC/qbL/fCjfgBKGkuIeVQ5W', 3]) gunicorn-registry stdout | 2020-03-31 09:25:14,256 [555] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 9506905, '[[2552739968, 2710085537, 1375792233, 1622564603, 742735765, 2572181374, 1643169938, 1318722110], 76055240, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, 25587712, datetime.datetime(2020, 3, 31, 9, 25, 13, 431219), '[[3014677442, 2909707954, 2437205147, 3713134934, 3458891136], 557768, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1]', 'd8nYTKaCMVjsVdVlNz+WaUx5js8yJd4Y+l9nxrAfNWz2pnzU5UqzoF8oJKgGfxP3pAHYNRDOwyH6rkWDAftzkQtwnNeIOT1tj3NR0JUdUXi38hqAEQvtuudpY0V/tEZUPZHjGGK9j3fkBUrDV/wGKdSmOA5cq/Pt2ejgRtzr3athBiyf+ZP3pxp1ChQJeggQ13JZjMFz9ywNSjOoGo+jxQD2LWxO1IVKms4nRW5jZ6XZFlAf4V+3PZVvkxztVJZzrVyQR8YpSR+wygXv6/ap2XTjkNtEaiNfzVFxv9v3H1H8hOmQE1HGDUNnew9CR34NTrVWGXVunHd8wgQdcVwPYu6bagCF2lSlMD+PxuXnsQJdziOCrXybG+YIgXtBRIvMwiO2oIgyDTqS0j3JqWmA8uFBg06F/nTXNSjJ9STykBaTLlKlCM//cpzA4h6qQm3fghT7pywOFmC/qbL/fCjfgBKGkuIeVQ5W', 3]) 2020-03-31 09:25:14,261 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,261 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:14,265 [555] [DEBUG] [app] Ending request: urn:request:b1275116-7a7a-4b20-8d9f-426fc1d6ea96 (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) gunicorn-registry stdout | 2020-03-31 09:25:14,265 [555] [DEBUG] [app] Ending request: urn:request:b1275116-7a7a-4b20-8d9f-426fc1d6ea96 (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) 2020-03-31 09:25:14,266 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,266 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:14,266 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,266 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,267 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 HTTP/1.1" 204 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.825 9506968 0.825) gunicorn-registry stdout | 2020-03-31 09:25:14,267 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 HTTP/1.1" 204 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:14,269 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.825 9506968 0.825) 2020-03-31 09:25:14,270 [556] [DEBUG] [app] Starting request: urn:request:e97efb30-bdd7-4ae4-8807-687045d3849e (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) gunicorn-registry stdout | 2020-03-31 09:25:14,269 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,270 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,270 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,271 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:14,270 [556] [DEBUG] [app] Starting request: urn:request:e97efb30-bdd7-4ae4-8807-687045d3849e (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) gunicorn-registry stdout | 2020-03-31 09:25:14,270 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,270 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,271 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,274 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 gunicorn-registry stdout | 2020-03-31 09:25:14,274 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:14,274 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,274 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,274 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,275 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,275 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,275 [555] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["9506905", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.7880020141601562}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 9506905}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "response_time", "LabelValues": ["v2.upload_chunk"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.8210799694061279}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "response_code", "LabelValues": ["v2.upload_chunk", "204"], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} gunicorn-registry stdout | 2020-03-31 09:25:14,274 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,274 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,274 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,275 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,275 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,275 [555] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["9506905", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.7880020141601562}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 9506905}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "response_time", "LabelValues": ["v2.upload_chunk"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.8210799694061279}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "response_code", "LabelValues": ["v2.upload_chunk", "204"], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} 2020-03-31 09:25:14,277 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,277 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,282 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,282 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) 2020-03-31 09:25:14,287 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,287 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:14,290 [556] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:14,290 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,290 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:14,290 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,292 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,292 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) 2020-03-31 09:25:14,295 [556] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 9506905, '[[2552739968, 2710085537, 1375792233, 1622564603, 742735765, 2572181374, 1643169938, 1318722110], 76055240, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, 25587712, datetime.datetime(2020, 3, 31, 9, 25, 13, 431219), '[[3014677442, 2909707954, 2437205147, 3713134934, 3458891136], 557768, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1]', 'd8nYTKaCMVjsVdVlNz+WaUx5js8yJd4Y+l9nxrAfNWz2pnzU5UqzoF8oJKgGfxP3pAHYNRDOwyH6rkWDAftzkQtwnNeIOT1tj3NR0JUdUXi38hqAEQvtuudpY0V/tEZUPZHjGGK9j3fkBUrDV/wGKdSmOA5cq/Pt2ejgRtzr3athBiyf+ZP3pxp1ChQJeggQ13JZjMFz9ywNSjOoGo+jxQD2LWxO1IVKms4nRW5jZ6XZFlAf4V+3PZVvkxztVJZzrVyQR8YpSR+wygXv6/ap2XTjkNtEaiNfzVFxv9v3H1H8hOmQE1HGDUNnew9CR34NTrVWGXVunHd8wgQdcVwPYu6bagCF2lSlMD+PxuXnsQJdziOCrXybG+YIgXtBRIvMwiO2oIgyDTqS0j3JqWmA8uFBg06F/nTXNSjJ9STykBaTLlKlCM//cpzA4h6qQm3fghT7pywOFmC/qbL/fCjfgBKGkuIeVQ5W', 3]) gunicorn-registry stdout | 2020-03-31 09:25:14,295 [556] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 9506905, '[[2552739968, 2710085537, 1375792233, 1622564603, 742735765, 2572181374, 1643169938, 1318722110], 76055240, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, 25587712, datetime.datetime(2020, 3, 31, 9, 25, 13, 431219), '[[3014677442, 2909707954, 2437205147, 3713134934, 3458891136], 557768, 0, "+/3X/rV/7V/737T/EwAA//8QhLFCAHCGAQ==", 1]', 'd8nYTKaCMVjsVdVlNz+WaUx5js8yJd4Y+l9nxrAfNWz2pnzU5UqzoF8oJKgGfxP3pAHYNRDOwyH6rkWDAftzkQtwnNeIOT1tj3NR0JUdUXi38hqAEQvtuudpY0V/tEZUPZHjGGK9j3fkBUrDV/wGKdSmOA5cq/Pt2ejgRtzr3athBiyf+ZP3pxp1ChQJeggQ13JZjMFz9ywNSjOoGo+jxQD2LWxO1IVKms4nRW5jZ6XZFlAf4V+3PZVvkxztVJZzrVyQR8YpSR+wygXv6/ap2XTjkNtEaiNfzVFxv9v3H1H8hOmQE1HGDUNnew9CR34NTrVWGXVunHd8wgQdcVwPYu6bagCF2lSlMD+PxuXnsQJdziOCrXybG+YIgXtBRIvMwiO2oIgyDTqS0j3JqWmA8uFBg06F/nTXNSjJ9STykBaTLlKlCM//cpzA4h6qQm3fghT7pywOFmC/qbL/fCjfgBKGkuIeVQ5W', 3]) 2020-03-31 09:25:14,301 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,301 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:14,304 [556] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:14,304 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,304 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,305 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:86f5886a-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:25:14,305 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86f5886a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:dWGzAQfeaC7hZH1AsCyR8DtdcfjyskGAJWvmBrOdZcQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,304 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,305 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:86f5886a-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:25:14,305 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86f5886a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:dWGzAQfeaC7hZH1AsCyR8DtdcfjyskGAJWvmBrOdZcQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,309 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,309 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,310 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 gunicorn-registry stdout | 2020-03-31 09:25:14,310 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:14,311 [556] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.0001270771026611328}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} gunicorn-registry stdout | 2020-03-31 09:25:14,311 [556] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.0001270771026611328}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} 2020-03-31 09:25:14,312 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 gunicorn-registry stdout | 2020-03-31 09:25:14,312 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:14,313 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3e-b01e-006e-3f3e-078a40000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86f5886a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:dWGzAQfeaC7hZH1AsCyR8DtdcfjyskGAJWvmBrOdZcQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,313 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3e-b01e-006e-3f3e-078a40000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '86f5886a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:dWGzAQfeaC7hZH1AsCyR8DtdcfjyskGAJWvmBrOdZcQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,313 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3e-b01e-006e-3f3e-078a40000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:14,313 [556] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3e-b01e-006e-3f3e-078a40000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. gunicorn-registry stdout | 2020-03-31 09:25:14,313 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3e-b01e-006e-3f3e-078a40000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:14,313 [556] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=86f5886a-7331-11ea-b9ed-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3e-b01e-006e-3f3e-078a40000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:14,314 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:86f6eb1a-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 comp:blocklist 2020-03-31 09:25:14,314 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f6eb1a-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86f6eb1a-7331-11ea-b9ed-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:b0V6J9OkAA3KTRnsTajcIt17xkrIE+Z5t69AC9KiPXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,314 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:86f6eb1a-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 comp:blocklist 2020-03-31 09:25:14,314 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f6eb1a-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86f6eb1a-7331-11ea-b9ed-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:b0V6J9OkAA3KTRnsTajcIt17xkrIE+Z5t69AC9KiPXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,323 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:14,323 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?comp=blocklist HTTP/1.1" 201 0 2020-03-31 09:25:14,324 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f6eb1a-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3f-b01e-006e-403e-078a40000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86f6eb1a-7331-11ea-b9ed-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:b0V6J9OkAA3KTRnsTajcIt17xkrIE+Z5t69AC9KiPXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gunicorn-registry stdout | 2020-03-31 09:25:14,324 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f6eb1a-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d3f-b01e-006e-403e-078a40000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '86f6eb1a-7331-11ea-b9ed-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:b0V6J9OkAA3KTRnsTajcIt17xkrIE+Z5t69AC9KiPXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:24.323352+00:00 (in 9.998881 seconds) 2020-03-31 09:25:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:24 UTC)" (scheduled at 2020-03-31 09:25:14.323352+00:00) 2020-03-31 09:25:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 14, 325503), True, datetime.datetime(2020, 3, 31, 9, 25, 14, 325503), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:25:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:14,326 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:86f8b166-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 comp:properties 2020-03-31 09:25:14,326 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f8b166-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86f8b166-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gKcBJTUpUQemArKiB8YrFVuQGP8nP/MZK7SUFM3AhMU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. notificationworker stdout | 2020-03-31 09:25:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:24.323352+00:00 (in 9.998881 seconds) 2020-03-31 09:25:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:24 UTC)" (scheduled at 2020-03-31 09:25:14.323352+00:00) 2020-03-31 09:25:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 14, 325503), True, datetime.datetime(2020, 3, 31, 9, 25, 14, 325503), 0, u'notification/%', 50, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,326 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:86f8b166-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 comp:properties 2020-03-31 09:25:14,326 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f8b166-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86f8b166-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gKcBJTUpUQemArKiB8YrFVuQGP8nP/MZK7SUFM3AhMU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,333 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?comp=properties HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,333 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?comp=properties HTTP/1.1" 200 0 2020-03-31 09:25:14,334 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f8b166-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d41-b01e-006e-423e-078a40000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86f8b166-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gKcBJTUpUQemArKiB8YrFVuQGP8nP/MZK7SUFM3AhMU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,334 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86f8b166-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d41-b01e-006e-423e-078a40000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '86f8b166-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gKcBJTUpUQemArKiB8YrFVuQGP8nP/MZK7SUFM3AhMU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,335 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:86fa1a1a-7331-11ea-b9ed-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?sr=b&sp=r&sig=vfiVxDt1hah1K6KNKeIctUUIC4aakvxD/Lvp1Uuh7KQ%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:25:14,335 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86fa1a1a-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?sr=b&sp=r&sig=vfiVxDt1hah1K6KNKeIctUUIC4aakvxD/Lvp1Uuh7KQ%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SC5YA9fkPrqE1gjVlbC3F8j5BHklAW7Pc9yyiu0NDks=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86fa1a1a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,335 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:86fa1a1a-7331-11ea-b9ed-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?sr=b&sp=r&sig=vfiVxDt1hah1K6KNKeIctUUIC4aakvxD/Lvp1Uuh7KQ%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:25:14,335 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86fa1a1a-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?sr=b&sp=r&sig=vfiVxDt1hah1K6KNKeIctUUIC4aakvxD/Lvp1Uuh7KQ%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SC5YA9fkPrqE1gjVlbC3F8j5BHklAW7Pc9yyiu0NDks=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86fa1a1a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:25:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:25:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:24 UTC)" executed successfully 2020-03-31 09:25:14,394 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:14,394 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 2020-03-31 09:25:14,395 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86fa1a1a-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d42-b01e-006e-433e-078a40000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?sr=b&sp=r&sig=vfiVxDt1hah1K6KNKeIctUUIC4aakvxD/Lvp1Uuh7KQ%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SC5YA9fkPrqE1gjVlbC3F8j5BHklAW7Pc9yyiu0NDks=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86fa1a1a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,395 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=86fa1a1a-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d42-b01e-006e-433e-078a40000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?sr=b&sp=r&sig=vfiVxDt1hah1K6KNKeIctUUIC4aakvxD/Lvp1Uuh7KQ%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SC5YA9fkPrqE1gjVlbC3F8j5BHklAW7Pc9yyiu0NDks=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '86fa1a1a-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,395 [556] [DEBUG] [storage.azurestorage] Deleting chunked upload 74f0b059-61d1-4efa-8df8-3acaf8ef1807 at path datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 2020-03-31 09:25:14,396 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:87036624-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 2020-03-31 09:25:14,396 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=87036624-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '87036624-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/wdGQjl9SSciTHRD3xYB06Y4A2r1yErGyfwaIzHPo2c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,395 [556] [DEBUG] [storage.azurestorage] Deleting chunked upload 74f0b059-61d1-4efa-8df8-3acaf8ef1807 at path datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 2020-03-31 09:25:14,396 [556] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:87036624-7331-11ea-b9ed-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 2020-03-31 09:25:14,396 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=87036624-7331-11ea-b9ed-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '87036624-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/wdGQjl9SSciTHRD3xYB06Y4A2r1yErGyfwaIzHPo2c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,404 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:14,404 [556] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807 HTTP/1.1" 202 0 2020-03-31 09:25:14,404 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=87036624-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d45-b01e-006e-463e-078a40000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '87036624-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/wdGQjl9SSciTHRD3xYB06Y4A2r1yErGyfwaIzHPo2c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,404 [556] [INFO] [azure.storage.common.storageclient] Client-Request-ID=87036624-7331-11ea-b9ed-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=e4831d45-b01e-006e-463e-078a40000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '87036624-7331-11ea-b9ed-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/wdGQjl9SSciTHRD3xYB06Y4A2r1yErGyfwaIzHPo2c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,405 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,405 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'74f0b059-61d1-4efa-8df8-3acaf8ef1807', 1, 0]) 2020-03-31 09:25:14,409 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,409 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 1, 0]) 2020-03-31 09:25:14,411 [556] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'd910da7a-e575-40de-8965-a8fbd51c3b85', 9506905, 25587712, False, True, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4']) gunicorn-registry stdout | 2020-03-31 09:25:14,411 [556] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'd910da7a-e575-40de-8965-a8fbd51c3b85', 9506905, 25587712, False, True, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4']) 2020-03-31 09:25:14,413 [556] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [3, 10]) gunicorn-registry stdout | 2020-03-31 09:25:14,413 [556] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [3, 10]) 2020-03-31 09:25:14,416 [556] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'd2bc9bf1-53b4-4871-925c-c0f25454067a', 1, u'/', 3, False, -1]) gunicorn-registry stdout | 2020-03-31 09:25:14,416 [556] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'd2bc9bf1-53b4-4871-925c-c0f25454067a', 1, u'/', 3, False, -1]) 2020-03-31 09:25:14,418 [556] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'ca467acc-240a-4463-af57-1d691bbb6386', 3, 1, 1585646714, 1585650314, True, False]) gunicorn-registry stdout | 2020-03-31 09:25:14,418 [556] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'ca467acc-240a-4463-af57-1d691bbb6386', 3, 1, 1585646714, 1585650314, True, False]) 2020-03-31 09:25:14,421 [556] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [3]) gunicorn-registry stdout | 2020-03-31 09:25:14,421 [556] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [3]) 2020-03-31 09:25:14,422 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,422 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [3, 1, 0]) 2020-03-31 09:25:14,425 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 3, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,425 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 3, 1, 0]) 2020-03-31 09:25:14,427 [556] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [3, 524288, 'd8nYTKaCMVjsVdVlNz+WaUx5js8yJd4Y+l9nxrAfNWz2pnzU5UqzoF8oJKgGfxP3pAHYNRDOwyH6rkWDAftzkQtwnNeIOT1tj3NR0JUdUXi38hqAEQvtuudpY0V/tEZUPZHjGGK9j3fkBUrDV/wGKdSmOA5cq/Pt2ejgRtzr3athBiyf+ZP3pxp1ChQJeggQ13JZjMFz9ywNSjOoGo+jxQD2LWxO1IVKms4nRW5jZ6XZFlAf4V+3PZVvkxztVJZzrVyQR8YpSR+wygXv6/ap2XTjkNtEaiNfzVFxv9v3H1H8hOmQE1HGDUNnew9CR34NTrVWGXVunHd8wgQdcVwPYu6bagCF2lSlMD+PxuXnsQJdziOCrXybG+YIgXtBRIvMwiO2oIgyDTqS0j3JqWmA8uFBg06F/nTXNSjJ9STykBaTLlKlCM//cpzA4h6qQm3fghT7pywOFmC/qbL/fCjfgBKGkuIeVQ5WtKEi/6Veq2GKxe/OGqBfhzI0U+E=']) gunicorn-registry stdout | 2020-03-31 09:25:14,427 [556] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [3, 524288, 'd8nYTKaCMVjsVdVlNz+WaUx5js8yJd4Y+l9nxrAfNWz2pnzU5UqzoF8oJKgGfxP3pAHYNRDOwyH6rkWDAftzkQtwnNeIOT1tj3NR0JUdUXi38hqAEQvtuudpY0V/tEZUPZHjGGK9j3fkBUrDV/wGKdSmOA5cq/Pt2ejgRtzr3athBiyf+ZP3pxp1ChQJeggQ13JZjMFz9ywNSjOoGo+jxQD2LWxO1IVKms4nRW5jZ6XZFlAf4V+3PZVvkxztVJZzrVyQR8YpSR+wygXv6/ap2XTjkNtEaiNfzVFxv9v3H1H8hOmQE1HGDUNnew9CR34NTrVWGXVunHd8wgQdcVwPYu6bagCF2lSlMD+PxuXnsQJdziOCrXybG+YIgXtBRIvMwiO2oIgyDTqS0j3JqWmA8uFBg06F/nTXNSjJ9STykBaTLlKlCM//cpzA4h6qQm3fghT7pywOFmC/qbL/fCjfgBKGkuIeVQ5WtKEi/6Veq2GKxe/OGqBfhzI0U+E=']) 2020-03-31 09:25:14,437 [556] [DEBUG] [app] Ending request: urn:request:e97efb30-bdd7-4ae4-8807-687045d3849e (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) gunicorn-registry stdout | 2020-03-31 09:25:14,437 [556] [DEBUG] [app] Ending request: urn:request:e97efb30-bdd7-4ae4-8807-687045d3849e (/v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807) 2020-03-31 09:25:14,437 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,437 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:14,437 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,437 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,438 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?digest=sha256%3A224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?digest=sha256%3A224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.170 121 0.170) gunicorn-registry stdout | 2020-03-31 09:25:14,438 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?digest=sha256%3A224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/74f0b059-61d1-4efa-8df8-3acaf8ef1807?digest=sha256%3A224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.170 121 0.170) 2020-03-31 09:25:14,440 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,440 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,441 [538] [DEBUG] [app] Starting request: urn:request:58b68a07-c866-44ef-829c-19c6598c648e (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) gunicorn-registry stdout | 2020-03-31 09:25:14,441 [538] [DEBUG] [app] Starting request: urn:request:58b68a07-c866-44ef-829c-19c6598c648e (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) 2020-03-31 09:25:14,441 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,441 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,442 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:14,441 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,441 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,442 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,442 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,443 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,442 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,443 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} gunicorn-registry stdout | 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,445 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,445 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,445 [538] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 2020-03-31 09:25:14,446 [538] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2; calling loader gunicorn-registry stdout | 2020-03-31 09:25:14,445 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,445 [538] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 2020-03-31 09:25:14,446 [538] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2; calling loader 2020-03-31 09:25:14,447 [518] [DEBUG] [app] Starting request: urn:request:ba52704a-0daf-431d-a57e-d3fe52ee1f1a (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:14,447 [518] [DEBUG] [app] Starting request: urn:request:ba52704a-0daf-431d-a57e-d3fe52ee1f1a (/health/instance) 2020-03-31 09:25:14,448 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,448 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:14,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:14,453 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,453 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) 2020-03-31 09:25:14,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:14,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:14,456 [554] [DEBUG] [app] Starting request: urn:request:dfaae170-e955-4a17-957b-7d6f55df6bd5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:14,456 [554] [DEBUG] [app] Starting request: urn:request:dfaae170-e955-4a17-957b-7d6f55df6bd5 (/v1/_internal_ping) 2020-03-31 09:25:14,457 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) 2020-03-31 09:25:14,458 [554] [DEBUG] [app] Ending request: urn:request:dfaae170-e955-4a17-957b-7d6f55df6bd5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:14,457 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', False, 1, 0]) 2020-03-31 09:25:14,458 [554] [DEBUG] [app] Ending request: urn:request:dfaae170-e955-4a17-957b-7d6f55df6bd5 (/v1/_internal_ping) 2020-03-31 09:25:14,458 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:14,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:25:14,458 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:14,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:14,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:14,461 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) gunicorn-web stdout | 2020-03-31 09:25:14,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:14,461 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,461 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) gunicorn-registry stdout | 2020-03-31 09:25:14,461 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,463 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,463 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,464 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,464 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,465 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) gunicorn-registry stdout | 2020-03-31 09:25:14,465 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) 2020-03-31 09:25:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:14,469 [538] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4'}, 'compressed_size': 9506905, 'db_id': 3, 'uuid': u'd910da7a-e575-40de-8965-a8fbd51c3b85', 'uploading': False, 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'uncompressed_size': 25587712} 2020-03-31 09:25:14,469 [538] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4'}, 'compressed_size': 9506905, 'db_id': 3, 'uuid': u'd910da7a-e575-40de-8965-a8fbd51c3b85', 'uploading': False, 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'uncompressed_size': 25587712}: 60s 2020-03-31 09:25:14,469 [538] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4'}, 'compressed_size': 9506905, 'db_id': 3, 'uuid': u'd910da7a-e575-40de-8965-a8fbd51c3b85', 'uploading': False, 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'uncompressed_size': 25587712}: 60s gunicorn-registry stdout | 2020-03-31 09:25:14,469 [538] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4'}, 'compressed_size': 9506905, 'db_id': 3, 'uuid': u'd910da7a-e575-40de-8965-a8fbd51c3b85', 'uploading': False, 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'uncompressed_size': 25587712} 2020-03-31 09:25:14,469 [538] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4'}, 'compressed_size': 9506905, 'db_id': 3, 'uuid': u'd910da7a-e575-40de-8965-a8fbd51c3b85', 'uploading': False, 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'uncompressed_size': 25587712}: 60s 2020-03-31 09:25:14,469 [538] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4'}, 'compressed_size': 9506905, 'db_id': 3, 'uuid': u'd910da7a-e575-40de-8965-a8fbd51c3b85', 'uploading': False, 'digest': u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 'uncompressed_size': 25587712}: 60s 2020-03-31 09:25:14,470 [538] [DEBUG] [app] Ending request: urn:request:58b68a07-c866-44ef-829c-19c6598c648e (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) 2020-03-31 09:25:14,471 [538] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:14,470 [538] [DEBUG] [app] Ending request: urn:request:58b68a07-c866-44ef-829c-19c6598c648e (/v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4) 2020-03-31 09:25:14,471 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,471 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,471 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.032 6 0.032) gunicorn-registry stdout | 2020-03-31 09:25:14,471 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,471 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.032 6 0.032) 2020-03-31 09:25:14,474 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,474 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,475 [556] [DEBUG] [app] Starting request: urn:request:925773c5-bed1-4b2d-bbdc-b48ec1454c6a (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) 2020-03-31 09:25:14,475 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,475 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'repo_name': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:25:14,475 [556] [DEBUG] [app] Starting request: urn:request:925773c5-bed1-4b2d-bbdc-b48ec1454c6a (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) 2020-03-31 09:25:14,475 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,475 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,475 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,475 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,476 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,475 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,475 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,476 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,479 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,480 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 2020-03-31 09:25:14,480 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2; calling loader gunicorn-registry stdout | 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,479 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,479 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,480 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 2020-03-31 09:25:14,480 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2; calling loader 2020-03-31 09:25:14,482 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,482 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,487 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,487 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) 2020-03-31 09:25:14,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:14,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:14,492 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,492 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) 2020-03-31 09:25:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:14,498 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2: None 2020-03-31 09:25:14,498 [556] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2: None 2020-03-31 09:25:14,498 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]} 2020-03-31 09:25:14,498 [521] [DEBUG] [app] Starting request: urn:request:7d1c005c-500b-4f50-b294-0861519726d4 (/_internal_ping) 2020-03-31 09:25:14,500 [521] [DEBUG] [app] Ending request: urn:request:7d1c005c-500b-4f50-b294-0861519726d4 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:14,500 [556] [DEBUG] [app] Ending request: urn:request:925773c5-bed1-4b2d-bbdc-b48ec1454c6a (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) 2020-03-31 09:25:14,501 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,501 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,500 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:14,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:25:14,498 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2: None 2020-03-31 09:25:14,498 [556] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2: None 2020-03-31 09:25:14,498 [556] [DEBUG] [endpoints.v2] sending response: {"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]} 2020-03-31 09:25:14,501 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 404 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/2.0" 404 0 "-" "Go-http-client/2.0" (0.028 81 0.028) gunicorn-web stdout | 2020-03-31 09:25:14,498 [521] [DEBUG] [app] Starting request: urn:request:7d1c005c-500b-4f50-b294-0861519726d4 (/_internal_ping) 2020-03-31 09:25:14,504 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:25:14,500 [521] [DEBUG] [app] Ending request: urn:request:7d1c005c-500b-4f50-b294-0861519726d4 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:25:14,504 [555] [DEBUG] [app] Starting request: urn:request:80b75342-13b6-4fcc-8b95-c1d6c620c723 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:14,505 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,505 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,505 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-registry stdout | 2020-03-31 09:25:14,500 [556] [DEBUG] [app] Ending request: urn:request:925773c5-bed1-4b2d-bbdc-b48ec1454c6a (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) 2020-03-31 09:25:14,501 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,501 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:25:14,500 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:14,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:14,506 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,501 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 404 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:14,507 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,508 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/2.0" 404 0 "-" "Go-http-client/2.0" (0.028 81 0.028) gunicorn-registry stdout | 2020-03-31 09:25:14,504 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-registry stdout | 2020-03-31 09:25:14,504 [555] [DEBUG] [app] Starting request: urn:request:80b75342-13b6-4fcc-8b95-c1d6c620c723 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:14,505 [555] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,505 [555] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,505 [555] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,510 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:25:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:14,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-registry stdout | 2020-03-31 09:25:14,506 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,507 [556] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,508 [555] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,508 [555] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-web stdout | 2020-03-31 09:25:14,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:14,514 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,510 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,514 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:25:14,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:14,516 [532] [DEBUG] [app] Starting request: urn:request:7d63f12f-e9f2-44cf-a8c1-6f95a0710f5d (/c1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:14,514 [555] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:14,517 [532] [DEBUG] [app] Ending request: urn:request:7d63f12f-e9f2-44cf-a8c1-6f95a0710f5d (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:14,514 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:14,517 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:25:14,516 [532] [DEBUG] [app] Starting request: urn:request:7d63f12f-e9f2-44cf-a8c1-6f95a0710f5d (/c1/_internal_ping) 2020-03-31 09:25:14,518 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:25:14,517 [532] [DEBUG] [app] Ending request: urn:request:7d63f12f-e9f2-44cf-a8c1-6f95a0710f5d (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:14,520 [518] [DEBUG] [app] Ending request: urn:request:ba52704a-0daf-431d-a57e-d3fe52ee1f1a (/health/instance) 2020-03-31 09:25:14,520 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:25:14,517 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:14,521 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) gunicorn-web stdout | 2020-03-31 09:25:14,518 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:14,522 [555] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 14, 517774)]) gunicorn-web stdout | 2020-03-31 09:25:14,520 [518] [DEBUG] [app] Ending request: urn:request:ba52704a-0daf-431d-a57e-d3fe52ee1f1a (/health/instance) 2020-03-31 09:25:14,520 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:14,521 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) gunicorn-registry stdout | 2020-03-31 09:25:14,522 [555] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [1, u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 0, '[[1779033703, 3144134277, 1013904242, 2773480762, 1359893119, 2600822924, 528734635, 1541459225], 0, 0, "", 1, 32]', 10, '{"content-type": null, "blocks": []}', 0, datetime.datetime(2020, 3, 31, 9, 25, 14, 517774)]) 2020-03-31 09:25:14,529 [555] [DEBUG] [app] Ending request: urn:request:80b75342-13b6-4fcc-8b95-c1d6c620c723 (/v2/admin/redhat-operators/blobs/uploads/) gunicorn-registry stdout | 2020-03-31 09:25:14,529 [555] [DEBUG] [app] Ending request: urn:request:80b75342-13b6-4fcc-8b95-c1d6c620c723 (/v2/admin/redhat-operators/blobs/uploads/) 2020-03-31 09:25:14,530 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,530 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.028 9 0.028) gunicorn-registry stdout | 2020-03-31 09:25:14,530 [555] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,530 [555] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,530 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/2.0" 202 0 "-" "Go-http-client/2.0" (0.028 9 0.028) gunicorn-registry stdout | 2020-03-31 09:25:14,530 [555] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "POST /v2/admin/redhat-operators/blobs/uploads/ HTTP/1.1" 202 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:14,533 [552] [DEBUG] [app] Starting request: urn:request:b2af898d-0c57-4c4e-bce6-edd1fd13c5a4 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) 2020-03-31 09:25:14,533 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,533 [552] [DEBUG] [app] Starting request: urn:request:b2af898d-0c57-4c4e-bce6-edd1fd13c5a4 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) 2020-03-31 09:25:14,533 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,534 [552] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 'repo_name': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:25:14,534 [552] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,534 [552] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:14,534 [552] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,535 [552] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,535 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,536 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,535 [552] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,535 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,536 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,538 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 25, 14, 536622), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 25, 14, 536665), u'quay', None, datetime.datetime(2020, 3, 30, 9, 25, 14, 536688), datetime.datetime(2020, 3, 24, 9, 25, 14, 536707), None]) gunicorn-registry stdout | 2020-03-31 09:25:14,538 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 25, 14, 536622), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 25, 14, 536665), u'quay', None, datetime.datetime(2020, 3, 30, 9, 25, 14, 536688), datetime.datetime(2020, 3, 24, 9, 25, 14, 536707), None]) 2020-03-31 09:25:14,538 [552] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2020-03-31 09:25:14,538 [552] [DEBUG] [peewee.pool] No connection available in pool. 2020-03-31 09:25:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:25:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:33.366791+00:00 (in 18.826027 seconds) 2020-03-31 09:25:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:30:14 UTC)" (scheduled at 2020-03-31 09:25:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:25:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:33.366791+00:00 (in 18.826027 seconds) 2020-03-31 09:25:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:30:14 UTC)" (scheduled at 2020-03-31 09:25:14.539408+00:00) 2020-03-31 09:25:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:25:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), u'exportactionlogs/%']) 2020-03-31 09:25:14,549 [552] [DEBUG] [peewee.pool] Created new connection 140043679893456. gunicorn-registry stdout | 2020-03-31 09:25:14,549 [552] [DEBUG] [peewee.pool] Created new connection 140043679893456. 2020-03-31 09:25:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), True, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:25:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), True, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), 0, u'exportactionlogs/%']) 2020-03-31 09:25:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), True, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:25:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), True, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 25, 14, 541576), u'exportactionlogs/%']) 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,564 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,564 [552] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators exportactionlogsworker stdout | 2020-03-31 09:25:14,564 [166] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,564 [552] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,564 [552] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,567 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:25:14,567 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,568 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:30:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:25:14,568 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:30:14 UTC)" executed successfully 2020-03-31 09:25:14,569 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,568 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:25:14,569 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,570 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,568 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:25:14,570 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:25:14,574 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:14,574 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) 2020-03-31 09:25:14,577 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,577 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'public', 1, 0]) 2020-03-31 09:25:14,581 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,581 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) 2020-03-31 09:25:14,584 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,584 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:14,587 [552] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:14,587 [552] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,587 [552] [DEBUG] [peewee.pool] Returning 140043679893456 to pool. gunicorn-registry stdout | 2020-03-31 09:25:14,587 [552] [DEBUG] [peewee.pool] Returning 140043679893456 to pool. 2020-03-31 09:25:14,588 [552] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 1832 +IbCEZAc5HdX87ehFlDTJA== x-ms-client-request-id:8720cafc-7331-11ea-a1d0-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 blockid:MDAwMDA= comp:block gunicorn-registry stdout | 2020-03-31 09:25:14,588 [552] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 1832 +IbCEZAc5HdX87ehFlDTJA== x-ms-client-request-id:8720cafc-7331-11ea-a1d0-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 blockid:MDAwMDA= comp:block 2020-03-31 09:25:14,589 [552] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8720cafc-7331-11ea-a1d0-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '1832', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8720cafc-7331-11ea-a1d0-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '+IbCEZAc5HdX87ehFlDTJA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aeE50d51duURCdqx01vlIVI9KhRvhkaeWHj5BJn/PKc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,589 [552] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8720cafc-7331-11ea-a1d0-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'comp': 'block', 'blockid': u'MDAwMDA=', 'timeout': None}, Headers={'Content-Length': '1832', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8720cafc-7331-11ea-a1d0-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '+IbCEZAc5HdX87ehFlDTJA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aeE50d51duURCdqx01vlIVI9KhRvhkaeWHj5BJn/PKc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,591 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 gunicorn-registry stdout | 2020-03-31 09:25:14,591 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 2020-03-31 09:25:14,645 [552] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:14,645 [552] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?comp=block&blockid=MDAwMDA%3D HTTP/1.1" 201 0 2020-03-31 09:25:14,645 [552] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8720cafc-7331-11ea-a1d0-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=6e253aa8-e01e-005c-433e-078a37000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '1832', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8720cafc-7331-11ea-a1d0-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '+IbCEZAc5HdX87ehFlDTJA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aeE50d51duURCdqx01vlIVI9KhRvhkaeWHj5BJn/PKc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,645 [552] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8720cafc-7331-11ea-a1d0-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=6e253aa8-e01e-005c-433e-078a37000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '1832', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8720cafc-7331-11ea-a1d0-0a580a830015', 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'x-ms-version': '2017-07-29', 'Content-MD5': '+IbCEZAc5HdX87ehFlDTJA==', 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aeE50d51duURCdqx01vlIVI9KhRvhkaeWHj5BJn/PKc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,646 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,646 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) 2020-03-31 09:25:14,650 [552] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1832, '[[3847690121, 1241206689, 2121916224, 3634044946, 3041919519, 3554765961, 1922902366, 2780293177], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, None, datetime.datetime(2020, 3, 31, 9, 25, 14, 517774), '[[1211979719, 2842443439, 172209987, 2864127822, 4278068783], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1]', '', 4]) gunicorn-registry stdout | 2020-03-31 09:25:14,650 [552] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1832, '[[3847690121, 1241206689, 2121916224, 3634044946, 3041919519, 3554765961, 1922902366, 2780293177], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 1, None, datetime.datetime(2020, 3, 31, 9, 25, 14, 517774), '[[1211979719, 2842443439, 172209987, 2864127822, 4278068783], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1]', '', 4]) 2020-03-31 09:25:14,658 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,658 [552] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:14,661 [552] [DEBUG] [app] Ending request: urn:request:b2af898d-0c57-4c4e-bce6-edd1fd13c5a4 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) gunicorn-registry stdout | 2020-03-31 09:25:14,661 [552] [DEBUG] [app] Ending request: urn:request:b2af898d-0c57-4c4e-bce6-edd1fd13c5a4 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) 2020-03-31 09:25:14,662 [552] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,663 [552] [DEBUG] [peewee.pool] Returning 140043679893456 to pool. 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.132 1895 0.132) gunicorn-registry stdout | 2020-03-31 09:25:14,662 [552] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,663 [552] [DEBUG] [peewee.pool] Returning 140043679893456 to pool. 2020-03-31 09:25:14,663 [552] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 HTTP/1.1" 204 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 HTTP/2.0" 204 0 "-" "Go-http-client/2.0" (0.132 1895 0.132) gunicorn-registry stdout | 2020-03-31 09:25:14,663 [552] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PATCH /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 HTTP/1.1" 204 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:14,666 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,666 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,667 [538] [DEBUG] [app] Starting request: urn:request:f873e880-50bc-413d-94d1-585aa754b477 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) 2020-03-31 09:25:14,667 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,667 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,667 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:14,667 [538] [DEBUG] [app] Starting request: urn:request:f873e880-50bc-413d-94d1-585aa754b477 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) 2020-03-31 09:25:14,667 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'upload_uuid': u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,667 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,667 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,668 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 gunicorn-registry stdout | 2020-03-31 09:25:14,668 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,670 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,670 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,670 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,671 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,671 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,672 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,672 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,673 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,673 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,674 [552] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 10:25:14.674343 gunicorn-registry stdout | 2020-03-31 09:25:14,674 [552] [DEBUG] [util.metrics.prometheus] Register returned 200 for 37 metrics; setting next to 2020-03-31 10:25:14.674343 2020-03-31 09:25:14,677 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,677 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) 2020-03-31 09:25:14,680 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,680 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:14,682 [538] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:14,682 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,682 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:14,682 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,684 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,684 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) 2020-03-31 09:25:14,687 [538] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1832, '[[3847690121, 1241206689, 2121916224, 3634044946, 3041919519, 3554765961, 1922902366, 2780293177], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, None, datetime.datetime(2020, 3, 31, 9, 25, 14, 517774), '[[1211979719, 2842443439, 172209987, 2864127822, 4278068783], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1]', '', 4]) gunicorn-registry stdout | 2020-03-31 09:25:14,687 [538] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [1, u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1832, '[[3847690121, 1241206689, 2121916224, 3634044946, 3041919519, 3554765961, 1922902366, 2780293177], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1, 32]', 10, '{"content-type": "application/octet-stream", "blocks": ["00000"]}', 2, None, datetime.datetime(2020, 3, 31, 9, 25, 14, 517774), '[[1211979719, 2842443439, 172209987, 2864127822, 4278068783], 14656, 0, "YlwiXSIsImVtcHR5X2xheWVyIjp0cnVlfV0sIm9zIjoibGludXgifQ==", 1]', '', 4]) 2020-03-31 09:25:14,693 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,693 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [10, 1, 0]) 2020-03-31 09:25:14,695 [538] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2020-03-31 09:25:14,695 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,695 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,695 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:873122c6-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 gunicorn-registry stdout | 2020-03-31 09:25:14,695 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,695 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:873122c6-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:25:14,696 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '873122c6-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:GaWnlWffSW/aw5SVBsI4+r6sx7RYnwz9ukMJsSimze0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,696 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '873122c6-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:GaWnlWffSW/aw5SVBsI4+r6sx7RYnwz9ukMJsSimze0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,700 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:14,700 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,703 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 gunicorn-registry stdout | 2020-03-31 09:25:14,703 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 35 2020-03-31 09:25:14,704 [538] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.00011706352233886719}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} 2020-03-31 09:25:14,705 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:14,705 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1142-501e-0049-223e-079d84000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '873122c6-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:GaWnlWffSW/aw5SVBsI4+r6sx7RYnwz9ukMJsSimze0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,704 [538] [DEBUG] [util.metrics.prometheus] Failed sending to prometheus: 500: [0] inconsistent label cardinality : {"Data": {"Subsystem": "", "Name": "chunk_upload_time", "LabelValues": ["0", "azure-us"], "Type": "Histogram", "Namespace": "quay", "Method": "Observe", "Value": 0.00011706352233886719}, "Call": "put"}, {"Data": {"Subsystem": "", "Name": "registry_push_byte_count", "LabelValues": [], "Type": "Counter", "Namespace": "quay", "Method": "Inc", "Value": 0}, "Call": "put"} 2020-03-31 09:25:14,705 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:14,706 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1142-501e-0049-223e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:14,706 [538] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1142-501e-0049-223e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. gunicorn-registry stdout | 2020-03-31 09:25:14,705 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1142-501e-0049-223e-079d84000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '873122c6-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:GaWnlWffSW/aw5SVBsI4+r6sx7RYnwz9ukMJsSimze0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,707 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:8732d12a-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 comp:blocklist 2020-03-31 09:25:14,707 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8732d12a-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8732d12a-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:kEQ32UlEGnSobCPsvi70UGew3C6/585PRJU+z2sWYeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,706 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1142-501e-0049-223e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:14,706 [538] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=873122c6-7331-11ea-bc11-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1142-501e-0049-223e-079d84000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. gunicorn-registry stdout | 2020-03-31 09:25:14,707 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 87 x-ms-client-request-id:8732d12a-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 comp:blocklist 2020-03-31 09:25:14,707 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8732d12a-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'comp': 'blocklist', 'timeout': None}, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8732d12a-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:kEQ32UlEGnSobCPsvi70UGew3C6/585PRJU+z2sWYeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,717 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:14,717 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?comp=blocklist HTTP/1.1" 201 0 2020-03-31 09:25:14,718 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8732d12a-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1143-501e-0049-233e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8732d12a-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:kEQ32UlEGnSobCPsvi70UGew3C6/585PRJU+z2sWYeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,718 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8732d12a-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1143-501e-0049-233e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '87', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '8732d12a-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:kEQ32UlEGnSobCPsvi70UGew3C6/585PRJU+z2sWYeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,719 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:8734bb34-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 comp:properties 2020-03-31 09:25:14,719 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8734bb34-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '8734bb34-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FCJ0/3GNsjg6V7yoI97s6at6AEqLO6FY8CxoigTep3c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,719 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-blob-content-type:application/octet-stream x-ms-client-request-id:8734bb34-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 comp:properties 2020-03-31 09:25:14,719 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8734bb34-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'comp': 'properties', 'timeout': None}, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '8734bb34-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FCJ0/3GNsjg6V7yoI97s6at6AEqLO6FY8CxoigTep3c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,727 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?comp=properties HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,727 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?comp=properties HTTP/1.1" 200 0 2020-03-31 09:25:14,728 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8734bb34-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1144-501e-0049-243e-079d84000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '8734bb34-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FCJ0/3GNsjg6V7yoI97s6at6AEqLO6FY8CxoigTep3c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,728 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8734bb34-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:13 GMT, Server-Request-ID=2f3c1144-501e-0049-243e-079d84000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-blob-content-md5': None, 'x-ms-version': '2017-07-29', 'x-ms-blob-content-language': None, 'x-ms-blob-cache-control': None, 'x-ms-client-request-id': '8734bb34-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-blob-content-type': 'application/octet-stream', 'If-Modified-Since': None, 'x-ms-blob-content-disposition': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-blob-content-encoding': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FCJ0/3GNsjg6V7yoI97s6at6AEqLO6FY8CxoigTep3c=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,729 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:8736528c-7331-11ea-bc11-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?sr=b&sp=r&sig=E0WFdsv2dug23CDVj00tWqyQjh8Xc8QEv7IBBVW/Ix0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:25:14,730 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8736528c-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?sr=b&sp=r&sig=E0WFdsv2dug23CDVj00tWqyQjh8Xc8QEv7IBBVW/Ix0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:KGfJHy9NJAsxQPr3w4bANpfT+QbZqqFB2PbqquhG+cY=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8736528c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,729 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:8736528c-7331-11ea-bc11-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?sr=b&sp=r&sig=E0WFdsv2dug23CDVj00tWqyQjh8Xc8QEv7IBBVW/Ix0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:25:14,730 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8736528c-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?sr=b&sp=r&sig=E0WFdsv2dug23CDVj00tWqyQjh8Xc8QEv7IBBVW/Ix0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:KGfJHy9NJAsxQPr3w4bANpfT+QbZqqFB2PbqquhG+cY=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8736528c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,832 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:14,832 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 2020-03-31 09:25:14,833 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8736528c-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1145-501e-0049-253e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?sr=b&sp=r&sig=E0WFdsv2dug23CDVj00tWqyQjh8Xc8QEv7IBBVW/Ix0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:KGfJHy9NJAsxQPr3w4bANpfT+QbZqqFB2PbqquhG+cY=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8736528c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,833 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8736528c-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1145-501e-0049-253e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?sr=b&sp=r&sig=E0WFdsv2dug23CDVj00tWqyQjh8Xc8QEv7IBBVW/Ix0%3D&sv=2017-07-29&se=2020-03-31T09%3A30%3A14Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:KGfJHy9NJAsxQPr3w4bANpfT+QbZqqFB2PbqquhG+cY=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8736528c-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,834 [538] [DEBUG] [storage.azurestorage] Deleting chunked upload ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 at path datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 gunicorn-registry stdout | 2020-03-31 09:25:14,834 [538] [DEBUG] [storage.azurestorage] Deleting chunked upload ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 at path datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 2020-03-31 09:25:14,834 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:8746499e-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 2020-03-31 09:25:14,834 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8746499e-7331-11ea-bc11-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8746499e-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ea2ZITb+zak1pAvkuzcho41A9Ib8E0kUjqdrchNxqsc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,834 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=DELETE x-ms-client-request-id:8746499e-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 2020-03-31 09:25:14,834 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8746499e-7331-11ea-bc11-0a580a830015 Outgoing request: Method=DELETE, Path=/quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8746499e-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ea2ZITb+zak1pAvkuzcho41A9Ib8E0kUjqdrchNxqsc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,843 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 HTTP/1.1" 202 0 gunicorn-registry stdout | 2020-03-31 09:25:14,843 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "DELETE /quayregistry/datastorage/registry/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3 HTTP/1.1" 202 0 2020-03-31 09:25:14,844 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8746499e-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1154-501e-0049-323e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8746499e-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ea2ZITb+zak1pAvkuzcho41A9Ib8E0kUjqdrchNxqsc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,844 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8746499e-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1154-501e-0049-323e-079d84000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8746499e-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'x-ms-delete-snapshots': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ea2ZITb+zak1pAvkuzcho41A9Ib8E0kUjqdrchNxqsc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,846 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,846 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes" FROM "blobupload" AS "t1" WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', [u'ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3', 1, 0]) 2020-03-31 09:25:14,849 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,849 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', [u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 1, 0]) 2020-03-31 09:25:14,851 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'7a10c766-d963-4234-89c4-998a563408cc', 1832, None, False, True, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97']) gunicorn-registry stdout | 2020-03-31 09:25:14,851 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'7a10c766-d963-4234-89c4-998a563408cc', 1832, None, False, True, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97']) 2020-03-31 09:25:14,853 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [4, 10]) gunicorn-registry stdout | 2020-03-31 09:25:14,853 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [4, 10]) 2020-03-31 09:25:14,855 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'bb3f54d6-659e-4c70-afeb-6cc56b6d2d54', 1, u'/', 4, False, -1]) gunicorn-registry stdout | 2020-03-31 09:25:14,855 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "security_indexed", "security_indexed_engine") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'bb3f54d6-659e-4c70-afeb-6cc56b6d2d54', 1, u'/', 4, False, -1]) 2020-03-31 09:25:14,856 [538] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'40bdaa5d-f4f7-44c9-b308-03341c5172a1', 4, 1, 1585646714, 1585650314, True, False]) gunicorn-registry stdout | 2020-03-31 09:25:14,856 [538] [DEBUG] [peewee] ('INSERT INTO "repositorytag" ("name", "image_id", "repository_id", "lifetime_start_ts", "lifetime_end_ts", "hidden", "reversion") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "repositorytag"."id"', [u'40bdaa5d-f4f7-44c9-b308-03341c5172a1', 4, 1, 1585646714, 1585650314, True, False]) 2020-03-31 09:25:14,858 [538] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [4]) gunicorn-registry stdout | 2020-03-31 09:25:14,858 [538] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [4]) 2020-03-31 09:25:14,860 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,860 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [4, 1, 0]) 2020-03-31 09:25:14,862 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 4, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,862 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."piece_length", "t1"."pieces" FROM "torrentinfo" AS "t1" WHERE (("t1"."piece_length" = %s) AND ("t1"."storage_id" = %s)) LIMIT %s OFFSET %s', [524288, 4, 1, 0]) 2020-03-31 09:25:14,864 [538] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [4, 524288, '3pdmFn29WHJBoqXwpO8Oq3vHtRA=']) gunicorn-registry stdout | 2020-03-31 09:25:14,864 [538] [DEBUG] [peewee] ('INSERT INTO "torrentinfo" ("storage_id", "piece_length", "pieces") VALUES (%s, %s, %s) RETURNING "torrentinfo"."id"', [4, 524288, '3pdmFn29WHJBoqXwpO8Oq3vHtRA=']) 2020-03-31 09:25:14,870 [538] [DEBUG] [app] Ending request: urn:request:f873e880-50bc-413d-94d1-585aa754b477 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) gunicorn-registry stdout | 2020-03-31 09:25:14,870 [538] [DEBUG] [app] Ending request: urn:request:f873e880-50bc-413d-94d1-585aa754b477 (/v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3) 2020-03-31 09:25:14,871 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,871 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?digest=sha256%3Abf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.206 122 0.206) gunicorn-registry stdout | 2020-03-31 09:25:14,871 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,871 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,872 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?digest=sha256%3Abf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?digest=sha256%3Abf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/2.0" 201 0 "-" "Go-http-client/2.0" (0.206 122 0.206) gunicorn-registry stdout | 2020-03-31 09:25:14,872 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "PUT /v2/admin/redhat-operators/blobs/uploads/ba120b65-7e6d-4f0d-b5c6-3eac6bbdc9d3?digest=sha256%3Abf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 201 0 "-" "Go-http-client/2.0" 2020-03-31 09:25:14,874 [556] [DEBUG] [app] Starting request: urn:request:2cef784b-90c4-4b1a-a1a2-a069b4eb4763 (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) 2020-03-31 09:25:14,874 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,875 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,875 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,875 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:14,874 [556] [DEBUG] [app] Starting request: urn:request:2cef784b-90c4-4b1a-a1a2-a069b4eb4763 (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) 2020-03-31 09:25:14,874 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,875 [556] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'admin', 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'repo_name': 'redhat-operators'} 2020-03-31 09:25:14,875 [556] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,875 [556] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,876 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,876 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin gunicorn-registry stdout | 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,877 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,877 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,878 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 gunicorn-registry stdout | 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,877 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,877 [556] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,877 [556] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,878 [556] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 2020-03-31 09:25:14,878 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2; calling loader gunicorn-registry stdout | 2020-03-31 09:25:14,878 [556] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2; calling loader 2020-03-31 09:25:14,880 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,880 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,885 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,885 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) 2020-03-31 09:25:14,888 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,888 [556] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) 2020-03-31 09:25:14,892 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) gunicorn-registry stdout | 2020-03-31 09:25:14,892 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:25:14,898 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) gunicorn-registry stdout | 2020-03-31 09:25:14,898 [556] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:25:14,901 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97'}, 'compressed_size': 1832, 'db_id': 4, 'uuid': u'7a10c766-d963-4234-89c4-998a563408cc', 'uploading': False, 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'uncompressed_size': None} gunicorn-registry stdout | 2020-03-31 09:25:14,901 [556] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2: {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97'}, 'compressed_size': 1832, 'db_id': 4, 'uuid': u'7a10c766-d963-4234-89c4-998a563408cc', 'uploading': False, 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'uncompressed_size': None} 2020-03-31 09:25:14,901 [556] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97'}, 'compressed_size': 1832, 'db_id': 4, 'uuid': u'7a10c766-d963-4234-89c4-998a563408cc', 'uploading': False, 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'uncompressed_size': None}: 60s 2020-03-31 09:25:14,901 [556] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97'}, 'compressed_size': 1832, 'db_id': 4, 'uuid': u'7a10c766-d963-4234-89c4-998a563408cc', 'uploading': False, 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'uncompressed_size': None}: 60s gunicorn-registry stdout | 2020-03-31 09:25:14,901 [556] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97'}, 'compressed_size': 1832, 'db_id': 4, 'uuid': u'7a10c766-d963-4234-89c4-998a563408cc', 'uploading': False, 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'uncompressed_size': None}: 60s 2020-03-31 09:25:14,901 [556] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__admin_redhat-operators_sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97_2 with expiration {'inputs': {'placements': [u'azure-us'], 'storage_path': u'sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97'}, 'compressed_size': 1832, 'db_id': 4, 'uuid': u'7a10c766-d963-4234-89c4-998a563408cc', 'uploading': False, 'digest': u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 'uncompressed_size': None}: 60s 2020-03-31 09:25:14,902 [556] [DEBUG] [app] Ending request: urn:request:2cef784b-90c4-4b1a-a1a2-a069b4eb4763 (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) gunicorn-registry stdout | 2020-03-31 09:25:14,902 [556] [DEBUG] [app] Ending request: urn:request:2cef784b-90c4-4b1a-a1a2-a069b4eb4763 (/v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97) 2020-03-31 09:25:14,902 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,903 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:14,902 [556] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:14,903 [556] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:14,903 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.030 6 0.030) gunicorn-registry stdout | 2020-03-31 09:25:14,903 [556] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 200 0 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:14 +0000] "HEAD /v2/admin/redhat-operators/blobs/sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/2.0" 200 0 "-" "Go-http-client/2.0" (0.030 6 0.030) 2020-03-31 09:25:14,906 [538] [DEBUG] [app] Starting request: urn:request:39c4de58-e281-426f-9191-4a9079bf4e85 (/v2/admin/redhat-operators/manifests/v1) gunicorn-registry stdout | 2020-03-31 09:25:14,906 [538] [DEBUG] [app] Starting request: urn:request:39c4de58-e281-426f-9191-4a9079bf4e85 (/v2/admin/redhat-operators/manifests/v1) 2020-03-31 09:25:14,906 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,906 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'manifest_ref': u'v1', 'repo_name': 'redhat-operators', 'namespace_name': 'admin'} 2020-03-31 09:25:14,906 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,907 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg gunicorn-registry stdout | 2020-03-31 09:25:14,906 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:14,906 [538] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'manifest_ref': u'v1', 'repo_name': 'redhat-operators', 'namespace_name': 'admin'} 2020-03-31 09:25:14,906 [538] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,907 [538] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjI5YWZiZDI4NjQwNjI1YmIyMzJiZWQ3OTBmOWY5OGUwMzNjMmJkZGEyMGNkM2ViNTM3ZjMzMTQ1ZjI0YzM5YzQifQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiLCJhY3Rpb25zIjpbInB1c2giLCJwdWxsIl19XSwiY29udGV4dCI6eyJlbnRpdHlfa2luZCI6InVzZXIiLCJraW5kIjoidXNlciIsInZlcnNpb24iOjIsImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCIsInVzZXIiOiJhZG1pbiIsImVudGl0eV9yZWZlcmVuY2UiOiI4MzUzY2FhZS04NWVjLTQ3NjQtOGMxMi1iZGEzZmViMDcyYWMiLCJjb20uYXBvc3RpbGxlLnJvb3RzIjp7ImFkbWluL3JlZGhhdC1vcGVyYXRvcnMiOiIkZGlzYWJsZWQifX0sImF1ZCI6InF1YXktZW50ZXJwcmlzZS5hcHBzLmJtd3VzLmZhYnJpYy1vY3B2NC1wb2MuY29tIiwiZXhwIjoxNTg1NjUwMzEyLCJpc3MiOiJxdWF5IiwiaWF0IjoxNTg1NjQ2NzEyLCJuYmYiOjE1ODU2NDY3MTIsInN1YiI6ImFkbWluIn0.d1tClOGvGDD8_Cg6GU9CSkYlXcox3TUXS4WmWpvnNq981DMnSPp9XEhfoEWpE5OixSYIUCvc7EVhHvCPvchVn1B4yqI-OHvtA1iMeSgWm2DSmWuHNO-F2qjOaqI8nxAjBLosqCpK_R2xJDl81IxZD8ZrA0w9KZLCUWAQiXEqmdPZjSQ80ToYK2Dvhnn-wtAeVd6K-o4fbo6x_ESE0uFfjcgXm3KPWF7Cx2-10C3UKkZ3cB4Iu0RtxS2EuiEhZrAqMJtmp3dZKtSHoWQrBp2Ox8IU6X3BXvW3o9Bqj555hc1wqHMxEaARB0EDlEcnvutTLMdjZZ4q2pjNINUy00jwSg 2020-03-31 09:25:14,907 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:14,907 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:14,908 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:14,908 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.permissions] Identity loaded: 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,911 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators gunicorn-registry stdout | 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: admin 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.registry_jwt_auth] Identity changed to admin 2020-03-31 09:25:14,910 [538] [DEBUG] [auth.registry_jwt_auth] Auth context set to {u'entity_kind': u'user', u'kind': u'user', u'entity_reference': u'8353caae-85ec-4764-8c12-bda3feb072ac', u'com.apostille.roots': {u'admin/redhat-operators': u'$disabled'}, u'version': 2, u'com.apostille.root': u'$disabled', u'user': u'admin'} 2020-03-31 09:25:14,911 [538] [DEBUG] [endpoints.v2] Checking permission for repo: admin/redhat-operators 2020-03-31 09:25:14,915 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,915 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE (("t2"."username" = %s) AND ("t1"."name" = %s)) LIMIT %s OFFSET %s', [u'admin', u'redhat-operators', 1, 0]) 2020-03-31 09:25:14,919 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes" FROM "manifest" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."digest" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:a9919eca0bb2f5ef8bd2f80d91940b16cd85ac00fb501bfcf09a02edd2c25f8e', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,919 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes" FROM "manifest" AS "t1" WHERE (("t1"."repository_id" = %s) AND ("t1"."digest" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:a9919eca0bb2f5ef8bd2f80d91940b16cd85ac00fb501bfcf09a02edd2c25f8e', 1, 0]) 2020-03-31 09:25:14,926 [538] [DEBUG] [peewee] ('((SELECT * FROM (SELECT "t1"."id", "t1"."content_checksum", "t1"."image_size", "t1"."uuid", "t1"."cas_path", "t1"."uncompressed_size", "t1"."uploading" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s) AS "q0") UNION ALL (SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "image" AS "t4" ON ("t4"."storage_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q1")) UNION ALL ((SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "image" AS "t4" ON ("t4"."storage_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q2") UNION ALL (SELECT * FROM (SELECT "t5"."id", "t5"."content_checksum", "t5"."image_size", "t5"."uuid", "t5"."cas_path", "t5"."uncompressed_size", "t5"."uploading" FROM "imagestorage" AS "t5" INNER JOIN "image" AS "t6" ON ("t6"."storage_id" = "t5"."id") WHERE (("t6"."repository_id" = %s) AND ("t5"."content_checksum" = %s)) LIMIT %s) AS "q3"))', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 1, 1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 1, 1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 1, 1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 1]) gunicorn-registry stdout | 2020-03-31 09:25:14,926 [538] [DEBUG] [peewee] ('((SELECT * FROM (SELECT "t1"."id", "t1"."content_checksum", "t1"."image_size", "t1"."uuid", "t1"."cas_path", "t1"."uncompressed_size", "t1"."uploading" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s) AS "q0") UNION ALL (SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "image" AS "t4" ON ("t4"."storage_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q1")) UNION ALL ((SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "image" AS "t4" ON ("t4"."storage_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q2") UNION ALL (SELECT * FROM (SELECT "t5"."id", "t5"."content_checksum", "t5"."image_size", "t5"."uuid", "t5"."cas_path", "t5"."uncompressed_size", "t5"."uploading" FROM "imagestorage" AS "t5" INNER JOIN "image" AS "t6" ON ("t6"."storage_id" = "t5"."id") WHERE (("t6"."repository_id" = %s) AND ("t5"."content_checksum" = %s)) LIMIT %s) AS "q3"))', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 1, 1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 1, 1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 1, 1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 1]) 2020-03-31 09:25:14,931 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,931 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) 2020-03-31 09:25:14,935 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,935 [538] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "image" AS "t2" ON ("t2"."storage_id" = "t1"."id") WHERE ((("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', False, 1, 0]) 2020-03-31 09:25:14,938 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) gunicorn-registry stdout | 2020-03-31 09:25:14,938 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:25:14,942 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=GET x-ms-client-request-id:8756cf94-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-range:bytes=0-33554431 x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 gunicorn-registry stdout | 2020-03-31 09:25:14,942 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=GET x-ms-client-request-id:8756cf94-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-range:bytes=0-33554431 x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:25:14,942 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8756cf94-7331-11ea-bc11-0a580a830015 Outgoing request: Method=GET, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8756cf94-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-range': 'bytes=0-33554431', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Y3uXd3d1fVFZXxJWf46rm/NJ6mDxeA0p6/Ybu/gDCU4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,942 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8756cf94-7331-11ea-bc11-0a580a830015 Outgoing request: Method=GET, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8756cf94-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-range': 'bytes=0-33554431', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Y3uXd3d1fVFZXxJWf46rm/NJ6mDxeA0p6/Ybu/gDCU4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,972 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "GET /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 206 1832 gunicorn-registry stdout | 2020-03-31 09:25:14,972 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "GET /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 206 1832 2020-03-31 09:25:14,973 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8756cf94-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1162-501e-0049-3f3e-079d84000000, HTTP Status Code=206, Message=Partial Content, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8756cf94-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-range': 'bytes=0-33554431', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Y3uXd3d1fVFZXxJWf46rm/NJ6mDxeA0p6/Ybu/gDCU4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,973 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8756cf94-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1162-501e-0049-3f3e-079d84000000, HTTP Status Code=206, Message=Partial Content, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8756cf94-7331-11ea-bc11-0a580a830015', 'x-ms-lease-id': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-range': 'bytes=0-33554431', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Y3uXd3d1fVFZXxJWf46rm/NJ6mDxeA0p6/Ybu/gDCU4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:14,978 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE (("t1"."content_checksum" = %s) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [u'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,978 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE (("t1"."content_checksum" = %s) AND ("t1"."uploading" = %s)) LIMIT %s OFFSET %s', [u'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', False, 1, 0]) 2020-03-31 09:25:14,982 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'98183bb5-7f9a-4d7d-ab4f-6bc45984f87c', 32, True, True, u'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4']) gunicorn-registry stdout | 2020-03-31 09:25:14,982 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', [u'98183bb5-7f9a-4d7d-ab4f-6bc45984f87c', 32, True, True, u'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4']) 2020-03-31 09:25:14,988 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'azure-us', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:14,988 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'azure-us', 1, 0]) 2020-03-31 09:25:14,991 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 32 x-ms-blob-type:BlockBlob x-ms-client-request-id:875e2bc2-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 gunicorn-registry stdout | 2020-03-31 09:25:14,991 [538] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT 32 x-ms-blob-type:BlockBlob x-ms-client-request-id:875e2bc2-7331-11ea-bc11-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:14 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 2020-03-31 09:25:14,991 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=875e2bc2-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'timeout': None}, Headers={'Content-Length': '32', 'x-ms-blob-type': 'BlockBlob', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '875e2bc2-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:m00x8ZTBkwqnARMg/Ox7FzuZU5ciPFtmjFWdIazVoTs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:14,991 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=875e2bc2-7331-11ea-bc11-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'timeout': None}, Headers={'Content-Length': '32', 'x-ms-blob-type': 'BlockBlob', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '875e2bc2-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:m00x8ZTBkwqnARMg/Ox7FzuZU5ciPFtmjFWdIazVoTs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:15,000 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.1" 201 0 gunicorn-registry stdout | 2020-03-31 09:25:15,000 [538] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.1" 201 0 2020-03-31 09:25:15,001 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=875e2bc2-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1165-501e-0049-423e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '32', 'x-ms-blob-type': 'BlockBlob', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '875e2bc2-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:m00x8ZTBkwqnARMg/Ox7FzuZU5ciPFtmjFWdIazVoTs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. gunicorn-registry stdout | 2020-03-31 09:25:15,001 [538] [INFO] [azure.storage.common.storageclient] Client-Request-ID=875e2bc2-7331-11ea-bc11-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:14 GMT, Server-Request-ID=2f3c1165-501e-0049-423e-079d84000000, HTTP Status Code=201, Message=Created, Headers={'Content-Length': '32', 'x-ms-blob-type': 'BlockBlob', 'x-ms-version': '2017-07-29', 'x-ms-lease-id': None, 'x-ms-client-request-id': '875e2bc2-7331-11ea-bc11-0a580a830015', 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:m00x8ZTBkwqnARMg/Ox7FzuZU5ciPFtmjFWdIazVoTs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:14 GMT'}. 2020-03-31 09:25:15,002 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [5, 10]) gunicorn-registry stdout | 2020-03-31 09:25:15,002 [538] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [5, 10]) 2020-03-31 09:25:15,008 [538] [DEBUG] [peewee] ('UPDATE "imagestorage" SET "uuid" = %s, "image_size" = %s, "uploading" = %s, "cas_path" = %s, "content_checksum" = %s WHERE ("imagestorage"."id" = %s)', [u'98183bb5-7f9a-4d7d-ab4f-6bc45984f87c', 32, False, True, u'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 5]) gunicorn-registry stdout | 2020-03-31 09:25:15,008 [538] [DEBUG] [peewee] ('UPDATE "imagestorage" SET "uuid" = %s, "image_size" = %s, "uploading" = %s, "cas_path" = %s, "content_checksum" = %s WHERE ("imagestorage"."id" = %s)', [u'98183bb5-7f9a-4d7d-ab4f-6bc45984f87c', 32, False, True, u'sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4', 5]) 2020-03-31 09:25:15,016 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE (("t1"."repository_id" = %s) AND ("t1"."docker_image_id" IN (%s, %s, %s, %s, %s)))', [1, u'8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075', u'37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b', u'2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb', u'3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1', u'69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129']) gunicorn-registry stdout | 2020-03-31 09:25:15,016 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE (("t1"."repository_id" = %s) AND ("t1"."docker_image_id" IN (%s, %s, %s, %s, %s)))', [1, u'8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075', u'37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b', u'2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb', u'3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1', u'69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129']) 2020-03-31 09:25:15,025 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075', 1, u'/', 3, datetime.datetime(2020, 3, 30, 18, 50, 2, 594822), None, u'["/bin/sh -c #(nop) COPY file:ae20c11896641c567481dcfe0fc0c177314e203253c80bb80b8503d6be134c9b in /registry-server "]', 9506905, u'{"container_config": {"Cmd": ["/bin/sh -c #(nop) COPY file:ae20c11896641c567481dcfe0fc0c177314e203253c80bb80b8503d6be134c9b in /registry-server "]}, "Size": 9506905, "id": "8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075", "created": "2020-03-30T18:50:02.594822979Z"}', False, -1, None]) gunicorn-registry stdout | 2020-03-31 09:25:15,025 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075', 1, u'/', 3, datetime.datetime(2020, 3, 30, 18, 50, 2, 594822), None, u'["/bin/sh -c #(nop) COPY file:ae20c11896641c567481dcfe0fc0c177314e203253c80bb80b8503d6be134c9b in /registry-server "]', 9506905, u'{"container_config": {"Cmd": ["/bin/sh -c #(nop) COPY file:ae20c11896641c567481dcfe0fc0c177314e203253c80bb80b8503d6be134c9b in /registry-server "]}, "Size": 9506905, "id": "8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075", "created": "2020-03-30T18:50:02.594822979Z"}', False, -1, None]) 2020-03-31 09:25:15,031 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b', 1, u'/5/', 2, datetime.datetime(2020, 3, 30, 18, 50, 2, 805539), None, u'["/bin/sh -c #(nop) COPY file:9501a4e82bb8fa49a1f5b0ba285f0b3f779adbb71346b968b1c4940041ff9c17 in /bin/grpc_health_probe "]', 12912385, u'{"container_config": {"Cmd": ["/bin/sh -c #(nop) COPY file:9501a4e82bb8fa49a1f5b0ba285f0b3f779adbb71346b968b1c4940041ff9c17 in /bin/grpc_health_probe "]}, "Size": 3405480, "id": "37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b", "parent": "8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075", "created": "2020-03-30T18:50:02.805539526Z"}', False, -1, 5]) gunicorn-registry stdout | 2020-03-31 09:25:15,031 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b', 1, u'/5/', 2, datetime.datetime(2020, 3, 30, 18, 50, 2, 805539), None, u'["/bin/sh -c #(nop) COPY file:9501a4e82bb8fa49a1f5b0ba285f0b3f779adbb71346b968b1c4940041ff9c17 in /bin/grpc_health_probe "]', 12912385, u'{"container_config": {"Cmd": ["/bin/sh -c #(nop) COPY file:9501a4e82bb8fa49a1f5b0ba285f0b3f779adbb71346b968b1c4940041ff9c17 in /bin/grpc_health_probe "]}, "Size": 3405480, "id": "37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b", "parent": "8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075", "created": "2020-03-30T18:50:02.805539526Z"}', False, -1, 5]) 2020-03-31 09:25:15,038 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb', 1, u'/5/6/', 5, datetime.datetime(2020, 3, 30, 18, 50, 2, 926782), None, u'["/bin/sh -c #(nop) EXPOSE 50051"]', 12912417, u'{"parent": "37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b", "created": "2020-03-30T18:50:02.926782927Z", "container_config": {"Cmd": ["/bin/sh -c #(nop) EXPOSE 50051"]}, "throwaway": true, "id": "2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb", "Size": 32}', False, -1, 6]) gunicorn-registry stdout | 2020-03-31 09:25:15,038 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb', 1, u'/5/6/', 5, datetime.datetime(2020, 3, 30, 18, 50, 2, 926782), None, u'["/bin/sh -c #(nop) EXPOSE 50051"]', 12912417, u'{"parent": "37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b", "created": "2020-03-30T18:50:02.926782927Z", "container_config": {"Cmd": ["/bin/sh -c #(nop) EXPOSE 50051"]}, "throwaway": true, "id": "2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb", "Size": 32}', False, -1, 6]) 2020-03-31 09:25:15,045 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1', 1, u'/5/6/7/', 5, datetime.datetime(2020, 3, 30, 18, 50, 3, 45937), None, u'["/bin/sh -c #(nop) ENTRYPOINT [\\"/registry-server\\"]"]', 12912449, u'{"parent": "2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb", "created": "2020-03-30T18:50:03.045937018Z", "container_config": {"Cmd": ["/bin/sh -c #(nop) ENTRYPOINT [\\"/registry-server\\"]"]}, "throwaway": true, "id": "3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1", "Size": 32}', False, -1, 7]) gunicorn-registry stdout | 2020-03-31 09:25:15,045 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1', 1, u'/5/6/7/', 5, datetime.datetime(2020, 3, 30, 18, 50, 3, 45937), None, u'["/bin/sh -c #(nop) ENTRYPOINT [\\"/registry-server\\"]"]', 12912449, u'{"parent": "2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb", "created": "2020-03-30T18:50:03.045937018Z", "container_config": {"Cmd": ["/bin/sh -c #(nop) ENTRYPOINT [\\"/registry-server\\"]"]}, "throwaway": true, "id": "3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1", "Size": 32}', False, -1, 7]) 2020-03-31 09:25:15,052 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129', 1, u'/5/6/7/8/', 5, datetime.datetime(2020, 3, 31, 9, 25, 12, 142354), None, u'["/bin/sh","-c","#(nop) ","CMD [\\"--database\\" \\"/bundles.db\\"]"]', 12912481, u'{"size": 15133542, "container": "14d5a2e0cb294d5fce12e5ae96a55dc558a648947804b5781588ac64b8c78fff", "parent": "3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1", "created": "2020-03-31T09:25:12.142354726Z", "config": {"ExposedPorts": {"50051/tcp": {}}, "Image": "sha256:8e33bba57e3b00e658fd00857df3d3c2a2dcd9d77e2d2ceb8fb4e38475c415c6", "Cmd": ["--database", "/bundles.db"], "Entrypoint": ["/registry-server"], "Env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"]}, "throwaway": true, "container_config": {"Cmd": ["/bin/sh", "-c", "#(nop) ", "CMD [\\"--database\\" \\"/bundles.db\\"]"], "Image": "sha256:8e33bba57e3b00e658fd00857df3d3c2a2dcd9d77e2d2ceb8fb4e38475c415c6", "Hostname": "14d5a2e0cb29", "ExposedPorts": {"50051/tcp": {}}, "Env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"], "Entrypoint": ["/registry-server"]}, "Size": 32, "architecture": "amd64", "docker_version": "18.02.0-ce", "os": "linux", "id": "69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129"}', False, -1, 8]) gunicorn-registry stdout | 2020-03-31 09:25:15,052 [538] [DEBUG] [peewee] ('INSERT INTO "image" ("docker_image_id", "repository_id", "ancestors", "storage_id", "created", "comment", "command", "aggregate_size", "v1_json_metadata", "security_indexed", "security_indexed_engine", "parent_id") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "image"."id"', [u'69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129', 1, u'/5/6/7/8/', 5, datetime.datetime(2020, 3, 31, 9, 25, 12, 142354), None, u'["/bin/sh","-c","#(nop) ","CMD [\\"--database\\" \\"/bundles.db\\"]"]', 12912481, u'{"size": 15133542, "container": "14d5a2e0cb294d5fce12e5ae96a55dc558a648947804b5781588ac64b8c78fff", "parent": "3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1", "created": "2020-03-31T09:25:12.142354726Z", "config": {"ExposedPorts": {"50051/tcp": {}}, "Image": "sha256:8e33bba57e3b00e658fd00857df3d3c2a2dcd9d77e2d2ceb8fb4e38475c415c6", "Cmd": ["--database", "/bundles.db"], "Entrypoint": ["/registry-server"], "Env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"]}, "throwaway": true, "container_config": {"Cmd": ["/bin/sh", "-c", "#(nop) ", "CMD [\\"--database\\" \\"/bundles.db\\"]"], "Image": "sha256:8e33bba57e3b00e658fd00857df3d3c2a2dcd9d77e2d2ceb8fb4e38475c415c6", "Hostname": "14d5a2e0cb29", "ExposedPorts": {"50051/tcp": {}}, "Env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"], "Entrypoint": ["/registry-server"]}, "Size": 32, "architecture": "amd64", "docker_version": "18.02.0-ce", "os": "linux", "id": "69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129"}', False, -1, 8]) 2020-03-31 09:25:15,061 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE (("t1"."docker_image_id" = %s) AND ("t1"."repository_id" = %s)) LIMIT %s OFFSET %s', [u'69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129', 1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,061 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE (("t1"."docker_image_id" = %s) AND ("t1"."repository_id" = %s)) LIMIT %s OFFSET %s', [u'69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129', 1, 1, 0]) 2020-03-31 09:25:15,064 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "mediatype" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:15,064 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "mediatype" AS "t1"', []) 2020-03-31 09:25:15,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:25:15,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:45.064514+00:00 (in 29.996778 seconds) gcworker stdout | 2020-03-31 09:25:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:45.064514+00:00 (in 29.996778 seconds) 2020-03-31 09:25:15,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:45 UTC)" (scheduled at 2020-03-31 09:25:15.064514+00:00) gcworker stdout | 2020-03-31 09:25:15,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:45 UTC)" (scheduled at 2020-03-31 09:25:15.064514+00:00) 2020-03-31 09:25:15,069 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes" FROM "manifest" AS "t1" WHERE (("t1"."digest" = %s) AND ("t1"."repository_id" = %s)) LIMIT %s OFFSET %s', [u'sha256:a9919eca0bb2f5ef8bd2f80d91940b16cd85ac00fb501bfcf09a02edd2c25f8e', 1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,069 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes" FROM "manifest" AS "t1" WHERE (("t1"."digest" = %s) AND ("t1"."repository_id" = %s)) LIMIT %s OFFSET %s', [u'sha256:a9919eca0bb2f5ef8bd2f80d91940b16cd85ac00fb501bfcf09a02edd2c25f8e', 1, 1, 0]) 2020-03-31 09:25:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437115, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:25:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437115, 1209600, 500, 1, 0]) 2020-03-31 09:25:15,072 [538] [DEBUG] [peewee] ('INSERT INTO "manifest" ("repository_id", "digest", "media_type_id", "manifest_bytes") VALUES (%s, %s, %s, %s) RETURNING "manifest"."id"', [1, u'sha256:a9919eca0bb2f5ef8bd2f80d91940b16cd85ac00fb501bfcf09a02edd2c25f8e', 15, u'{\n "schemaVersion": 2,\n "mediaType": "application/vnd.docker.distribution.manifest.v2+json",\n "config": {\n "mediaType": "application/vnd.docker.container.image.v1+json",\n "size": 1832,\n "digest": "sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97"\n },\n "layers": [\n {\n "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",\n "size": 9506905,\n "digest": "sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4"\n },\n {\n "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",\n "size": 3405480,\n "digest": "sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29"\n },\n {\n "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",\n "size": 2221157,\n "digest": "sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438"\n }\n ]\n}']) gunicorn-registry stdout | 2020-03-31 09:25:15,072 [538] [DEBUG] [peewee] ('INSERT INTO "manifest" ("repository_id", "digest", "media_type_id", "manifest_bytes") VALUES (%s, %s, %s, %s) RETURNING "manifest"."id"', [1, u'sha256:a9919eca0bb2f5ef8bd2f80d91940b16cd85ac00fb501bfcf09a02edd2c25f8e', 15, u'{\n "schemaVersion": 2,\n "mediaType": "application/vnd.docker.distribution.manifest.v2+json",\n "config": {\n "mediaType": "application/vnd.docker.container.image.v1+json",\n "size": 1832,\n "digest": "sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97"\n },\n "layers": [\n {\n "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",\n "size": 9506905,\n "digest": "sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4"\n },\n {\n "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",\n "size": 3405480,\n "digest": "sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29"\n },\n {\n "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",\n "size": 2221157,\n "digest": "sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438"\n }\n ]\n}']) 2020-03-31 09:25:15,074 [538] [DEBUG] [peewee] ('INSERT INTO "manifestblob" ("repository_id", "manifest_id", "blob_id") VALUES (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s) RETURNING "manifestblob"."id"', [1, 1, 1, 1, 1, 2, 1, 1, 3, 1, 1, 4, 1, 1, 5]) gunicorn-registry stdout | 2020-03-31 09:25:15,074 [538] [DEBUG] [peewee] ('INSERT INTO "manifestblob" ("repository_id", "manifest_id", "blob_id") VALUES (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s), (%s, %s, %s) RETURNING "manifestblob"."id"', [1, 1, 1, 1, 1, 2, 1, 1, 3, 1, 1, 4, 1, 1, 5]) 2020-03-31 09:25:15,078 [538] [DEBUG] [peewee] ('INSERT INTO "manifestlegacyimage" ("repository_id", "manifest_id", "image_id") VALUES (%s, %s, %s) RETURNING "manifestlegacyimage"."id"', [1, 1, 9]) gunicorn-registry stdout | 2020-03-31 09:25:15,078 [538] [DEBUG] [peewee] ('INSERT INTO "manifestlegacyimage" ("repository_id", "manifest_id", "image_id") VALUES (%s, %s, %s) RETURNING "manifestlegacyimage"."id"', [1, 1, 9]) 2020-03-31 09:25:15,086 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t2"."id", "t2"."name" FROM "manifest" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,086 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t2"."id", "t2"."name" FROM "manifest" AS "t1" INNER JOIN "mediatype" AS "t2" ON ("t1"."media_type_id" = "t2"."id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:25:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:25:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:25:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:45 UTC)" executed successfully 2020-03-31 09:25:15,090 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."manifest_id", "t1"."image_id", "t2"."id", "t2"."docker_image_id", "t2"."repository_id", "t2"."ancestors", "t2"."storage_id", "t2"."created", "t2"."comment", "t2"."command", "t2"."aggregate_size", "t2"."v1_json_metadata", "t2"."v1_checksum", "t2"."security_indexed", "t2"."security_indexed_engine", "t2"."parent_id" FROM "manifestlegacyimage" AS "t1" INNER JOIN "image" AS "t2" ON ("t1"."image_id" = "t2"."id") WHERE ("t1"."manifest_id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,090 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."manifest_id", "t1"."image_id", "t2"."id", "t2"."docker_image_id", "t2"."repository_id", "t2"."ancestors", "t2"."storage_id", "t2"."created", "t2"."comment", "t2"."command", "t2"."aggregate_size", "t2"."v1_json_metadata", "t2"."v1_checksum", "t2"."security_indexed", "t2"."security_indexed_engine", "t2"."parent_id" FROM "manifestlegacyimage" AS "t1" INNER JOIN "image" AS "t2" ON ("t1"."image_id" = "t2"."id") WHERE ("t1"."manifest_id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:15,097 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."repository_id", "t2"."digest", "t2"."media_type_id", "t2"."manifest_bytes" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE (((("t1"."repository_id" = %s) AND ("t1"."name" = %s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) LIMIT %s OFFSET %s', [1, u'v1', None, 1585646715096, False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,097 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."repository_id", "t1"."manifest_id", "t1"."lifetime_start_ms", "t1"."lifetime_end_ms", "t1"."hidden", "t1"."reversion", "t1"."tag_kind_id", "t1"."linked_tag_id", "t2"."id", "t2"."repository_id", "t2"."digest", "t2"."media_type_id", "t2"."manifest_bytes" FROM "tag" AS "t1" INNER JOIN "manifest" AS "t2" ON ("t1"."manifest_id" = "t2"."id") WHERE (((("t1"."repository_id" = %s) AND ("t1"."name" = %s)) AND (("t1"."lifetime_end_ms" IS %s) OR ("t1"."lifetime_end_ms" > %s))) AND ("t1"."hidden" = %s)) LIMIT %s OFFSET %s', [1, u'v1', None, 1585646715096, False, 1, 0]) 2020-03-31 09:25:15,101 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "tagkind" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:15,101 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "tagkind" AS "t1"', []) 2020-03-31 09:25:15,103 [538] [DEBUG] [peewee] ('INSERT INTO "tag" ("name", "repository_id", "manifest_id", "lifetime_start_ms", "hidden", "reversion", "tag_kind_id") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "tag"."id"', [u'v1', 1, 1, 1585646715096, False, False, 1]) gunicorn-registry stdout | 2020-03-31 09:25:15,103 [538] [DEBUG] [peewee] ('INSERT INTO "tag" ("name", "repository_id", "manifest_id", "lifetime_start_ms", "hidden", "reversion", "tag_kind_id") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "tag"."id"', [u'v1', 1, 1, 1585646715096, False, False, 1]) 2020-03-31 09:25:15,110 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."manifest_id", "t1"."image_id", "t2"."id", "t2"."docker_image_id", "t2"."repository_id", "t2"."ancestors", "t2"."storage_id", "t2"."created", "t2"."comment", "t2"."command", "t2"."aggregate_size", "t2"."v1_json_metadata", "t2"."v1_checksum", "t2"."security_indexed", "t2"."security_indexed_engine", "t2"."parent_id" FROM "manifestlegacyimage" AS "t1" INNER JOIN "image" AS "t2" ON ("t1"."image_id" = "t2"."id") WHERE ("t1"."manifest_id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,110 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."manifest_id", "t1"."image_id", "t2"."id", "t2"."docker_image_id", "t2"."repository_id", "t2"."ancestors", "t2"."storage_id", "t2"."created", "t2"."comment", "t2"."command", "t2"."aggregate_size", "t2"."v1_json_metadata", "t2"."v1_checksum", "t2"."security_indexed", "t2"."security_indexed_engine", "t2"."parent_id" FROM "manifestlegacyimage" AS "t1" INNER JOIN "image" AS "t2" ON ("t1"."image_id" = "t2"."id") WHERE ("t1"."manifest_id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:15,114 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [5, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,114 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [5, 1, 0]) 2020-03-31 09:25:15,117 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes" FROM "manifest" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,117 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes" FROM "manifest" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:15,125 [538] [DEBUG] [peewee] ('((SELECT * FROM (SELECT "t1"."id", "t1"."content_checksum", "t1"."image_size", "t1"."uuid", "t1"."cas_path", "t1"."uncompressed_size", "t1"."uploading" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s) AS "q0") UNION ALL (SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "manifestblob" AS "t4" ON ("t4"."blob_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q1")) UNION ALL ((SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "manifestblob" AS "t4" ON ("t4"."blob_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q2") UNION ALL (SELECT * FROM (SELECT "t5"."id", "t5"."content_checksum", "t5"."image_size", "t5"."uuid", "t5"."cas_path", "t5"."uncompressed_size", "t5"."uploading" FROM "imagestorage" AS "t5" INNER JOIN "manifestblob" AS "t6" ON ("t6"."blob_id" = "t5"."id") WHERE (("t6"."repository_id" = %s) AND ("t5"."content_checksum" = %s)) LIMIT %s) AS "q3"))', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 1, 1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 1, 1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 1, 1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 1]) gunicorn-registry stdout | 2020-03-31 09:25:15,125 [538] [DEBUG] [peewee] ('((SELECT * FROM (SELECT "t1"."id", "t1"."content_checksum", "t1"."image_size", "t1"."uuid", "t1"."cas_path", "t1"."uncompressed_size", "t1"."uploading" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s) AS "q0") UNION ALL (SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "manifestblob" AS "t4" ON ("t4"."blob_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q1")) UNION ALL ((SELECT * FROM (SELECT "t3"."id", "t3"."content_checksum", "t3"."image_size", "t3"."uuid", "t3"."cas_path", "t3"."uncompressed_size", "t3"."uploading" FROM "imagestorage" AS "t3" INNER JOIN "manifestblob" AS "t4" ON ("t4"."blob_id" = "t3"."id") WHERE (("t4"."repository_id" = %s) AND ("t3"."content_checksum" = %s)) LIMIT %s) AS "q2") UNION ALL (SELECT * FROM (SELECT "t5"."id", "t5"."content_checksum", "t5"."image_size", "t5"."uuid", "t5"."cas_path", "t5"."uncompressed_size", "t5"."uploading" FROM "imagestorage" AS "t5" INNER JOIN "manifestblob" AS "t6" ON ("t6"."blob_id" = "t5"."id") WHERE (("t6"."repository_id" = %s) AND ("t5"."content_checksum" = %s)) LIMIT %s) AS "q3"))', [1, u'sha256:bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97', 1, 1, u'sha256:224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4', 1, 1, u'sha256:e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29', 1, 1, u'sha256:070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438', 1]) 2020-03-31 09:25:15,131 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,131 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) 2020-03-31 09:25:15,135 [538] [DEBUG] [peewee] ('INSERT INTO "queueitem" ("queue_name", "body", "available_after", "available", "retries_remaining", "state_id") VALUES (%s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s) RETURNING "queueitem"."id"', [u'imagestoragereplication/7a10c766-d963-4234-89c4-998a563408cc/', u'{"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134298), True, 5, u'3b2761af-2d96-4a58-8a4e-582182e7c8c5', u'imagestoragereplication/d910da7a-e575-40de-8965-a8fbd51c3b85/', u'{"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134324), True, 5, u'91392f91-1861-468f-9dae-ed27557543bc', u'imagestoragereplication/0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8/', u'{"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134341), True, 5, u'793df7d5-772a-44b3-9e94-f86d64243281', u'imagestoragereplication/229ce905-1640-4594-aae8-fb66fcc5e51f/', u'{"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134357), True, 5, u'b44bb566-5001-4789-abca-e578ed46f39b']) gunicorn-registry stdout | 2020-03-31 09:25:15,135 [538] [DEBUG] [peewee] ('INSERT INTO "queueitem" ("queue_name", "body", "available_after", "available", "retries_remaining", "state_id") VALUES (%s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s) RETURNING "queueitem"."id"', [u'imagestoragereplication/7a10c766-d963-4234-89c4-998a563408cc/', u'{"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134298), True, 5, u'3b2761af-2d96-4a58-8a4e-582182e7c8c5', u'imagestoragereplication/d910da7a-e575-40de-8965-a8fbd51c3b85/', u'{"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134324), True, 5, u'91392f91-1861-468f-9dae-ed27557543bc', u'imagestoragereplication/0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8/', u'{"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134341), True, 5, u'793df7d5-772a-44b3-9e94-f86d64243281', u'imagestoragereplication/229ce905-1640-4594-aae8-fb66fcc5e51f/', u'{"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"}', datetime.datetime(2020, 3, 31, 9, 25, 15, 134357), True, 5, u'b44bb566-5001-4789-abca-e578ed46f39b']) 2020-03-31 09:25:15,141 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', [u'8353caae-85ec-4764-8c12-bda3feb072ac', False, 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,141 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', [u'8353caae-85ec-4764-8c12-bda3feb072ac', False, 1, 0]) 2020-03-31 09:25:15,146 [538] [DEBUG] [util.audit] Checking publishing push_repo to the user events system gunicorn-registry stdout | 2020-03-31 09:25:15,146 [538] [DEBUG] [util.audit] Checking publishing push_repo to the user events system 2020-03-31 09:25:15,146 [538] [DEBUG] [util.audit] Publishing push_repo to the user events system gunicorn-registry stdout | 2020-03-31 09:25:15,146 [538] [DEBUG] [util.audit] Publishing push_repo to the user events system 2020-03-31 09:25:15,146 [538] [DEBUG] [util.audit] Logging the push_repo to analytics engine gunicorn-registry stdout | 2020-03-31 09:25:15,146 [538] [DEBUG] [util.audit] Logging the push_repo to analytics engine 2020-03-31 09:25:15,147 [538] [DEBUG] [util.audit] Resolving IP address 10.131.0.7 gunicorn-registry stdout | 2020-03-31 09:25:15,147 [538] [DEBUG] [util.audit] Resolving IP address 10.131.0.7 2020-03-31 09:25:15,179 [538] [DEBUG] [util.audit] Resolved IP address 10.131.0.7 gunicorn-registry stdout | 2020-03-31 09:25:15,179 [538] [DEBUG] [util.audit] Resolved IP address 10.131.0.7 2020-03-31 09:25:15,179 [538] [DEBUG] [util.audit] Logging the push_repo to logs system gunicorn-registry stdout | 2020-03-31 09:25:15,179 [538] [DEBUG] [util.audit] Logging the push_repo to logs system 2020-03-31 09:25:15,181 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) gunicorn-registry stdout | 2020-03-31 09:25:15,181 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', [u'admin', 1, 0]) 2020-03-31 09:25:15,185 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "logentrykind" AS "t1"', []) gunicorn-registry stdout | 2020-03-31 09:25:15,185 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "logentrykind" AS "t1"', []) 2020-03-31 09:25:15,190 [538] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [42, 1, 1, 1, datetime.datetime(2020, 3, 31, 9, 25, 15, 179990), u'10.131.0.7', u'{"repo": "redhat-operators", "username": "admin", "tag": "v1", "namespace": "admin", "resolved_ip": {"provider": "internet", "service": null, "sync_token": "1576558391", "country_iso_code": null}}']) gunicorn-registry stdout | 2020-03-31 09:25:15,190 [538] [DEBUG] [peewee] ('INSERT INTO "logentry3" ("kind_id", "account_id", "performer_id", "repository_id", "datetime", "ip", "metadata_json") VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING "logentry3"."id"', [42, 1, 1, 1, datetime.datetime(2020, 3, 31, 9, 25, 15, 179990), u'10.131.0.7', u'{"repo": "redhat-operators", "username": "admin", "tag": "v1", "namespace": "admin", "resolved_ip": {"provider": "internet", "service": null, "sync_token": "1576558391", "country_iso_code": null}}']) 2020-03-31 09:25:15,197 [538] [DEBUG] [util.audit] Track and log of push_repo complete gunicorn-registry stdout | 2020-03-31 09:25:15,197 [538] [DEBUG] [util.audit] Track and log of push_repo complete 2020-03-31 09:25:15,199 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."event_id", "t1"."method_id", "t1"."title", "t1"."config_json", "t1"."event_config_json", "t1"."number_of_failures", "t2"."id", "t2"."namespace_user_id", "t2"."name", "t2"."visibility_id", "t2"."description", "t2"."badge_token", "t2"."kind_id", "t2"."trust_enabled", "t2"."state", "t3"."id", "t3"."uuid", "t3"."username", "t3"."password_hash", "t3"."email", "t3"."verified", "t3"."stripe_id", "t3"."organization", "t3"."robot", "t3"."invoice_email", "t3"."invalid_login_attempts", "t3"."last_invalid_login", "t3"."removed_tag_expiration_s", "t3"."enabled", "t3"."invoice_email_address", "t3"."given_name", "t3"."family_name", "t3"."company", "t3"."location", "t3"."maximum_queued_builds_count", "t3"."creation_date", "t3"."last_accessed" FROM "repositorynotification" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") INNER JOIN "externalnotificationevent" AS "t4" ON ("t1"."event_id" = "t4"."id") WHERE ((("t3"."username" = %s) AND ("t2"."name" = %s)) AND ("t4"."name" = %s))', [u'admin', u'redhat-operators', u'repo_push']) gunicorn-registry stdout | 2020-03-31 09:25:15,199 [538] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."repository_id", "t1"."event_id", "t1"."method_id", "t1"."title", "t1"."config_json", "t1"."event_config_json", "t1"."number_of_failures", "t2"."id", "t2"."namespace_user_id", "t2"."name", "t2"."visibility_id", "t2"."description", "t2"."badge_token", "t2"."kind_id", "t2"."trust_enabled", "t2"."state", "t3"."id", "t3"."uuid", "t3"."username", "t3"."password_hash", "t3"."email", "t3"."verified", "t3"."stripe_id", "t3"."organization", "t3"."robot", "t3"."invoice_email", "t3"."invalid_login_attempts", "t3"."last_invalid_login", "t3"."removed_tag_expiration_s", "t3"."enabled", "t3"."invoice_email_address", "t3"."given_name", "t3"."family_name", "t3"."company", "t3"."location", "t3"."maximum_queued_builds_count", "t3"."creation_date", "t3"."last_accessed" FROM "repositorynotification" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") INNER JOIN "externalnotificationevent" AS "t4" ON ("t1"."event_id" = "t4"."id") WHERE ((("t3"."username" = %s) AND ("t2"."name" = %s)) AND ("t4"."name" = %s))', [u'admin', u'redhat-operators', u'repo_push']) 2020-03-31 09:25:15,205 [538] [DEBUG] [app] Ending request: urn:request:39c4de58-e281-426f-9191-4a9079bf4e85 (/v2/admin/redhat-operators/manifests/v1) gunicorn-registry stdout | 2020-03-31 09:25:15,205 [538] [DEBUG] [app] Ending request: urn:request:39c4de58-e281-426f-9191-4a9079bf4e85 (/v2/admin/redhat-operators/manifests/v1) 2020-03-31 09:25:15,206 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:15,206 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. gunicorn-registry stdout | 2020-03-31 09:25:15,206 [538] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:15,206 [538] [DEBUG] [peewee.pool] Returning 140043679892496 to pool. 2020-03-31 09:25:15,207 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:15 +0000] "PUT /v2/admin/redhat-operators/manifests/v1 HTTP/1.1" 202 2 "-" "Go-http-client/2.0" 10.131.0.7 () - - [31/Mar/2020:09:25:15 +0000] "PUT /v2/admin/redhat-operators/manifests/v1 HTTP/2.0" 202 2 "-" "Go-http-client/2.0" (0.302 1030 0.301) gunicorn-registry stdout | 2020-03-31 09:25:15,207 [538] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:15 +0000] "PUT /v2/admin/redhat-operators/manifests/v1 HTTP/1.1" 202 2 "-" "Go-http-client/2.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:15 +0000] "PUT /v2/admin/redhat-operators/manifests/v1 HTTP/2.0" 202 2 "-" "Go-http-client/2.0" (0.302 1030 0.301) 2020-03-31 09:25:15,209 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:15,209 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:15,210 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:15,210 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:15,211 [538] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:15,211 [538] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:25:15,215 [538] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'push_repo', 'namespace': 'admin', 'repository': 'redhat-operators'} gunicorn-registry stdout | 2020-03-31 09:25:15,215 [538] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'push_repo', 'namespace': 'admin', 'repository': 'redhat-operators'} 2020-03-31 09:25:16,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:25:16,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:16,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:26.924497+00:00 (in 9.996431 seconds) 2020-03-31 09:25:16,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:26 UTC)" (scheduled at 2020-03-31 09:25:16.924497+00:00) 2020-03-31 09:25:16,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:25:16,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:26.924497+00:00 (in 9.996431 seconds) 2020-03-31 09:25:16,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:26 UTC)" (scheduled at 2020-03-31 09:25:16.924497+00:00) 2020-03-31 09:25:16,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:16,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 16, 928894), True, datetime.datetime(2020, 3, 31, 9, 25, 16, 928894), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:25:16,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 16, 928894), True, datetime.datetime(2020, 3, 31, 9, 25, 16, 928894), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:25:16,947 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 45, 16, 928894), 1, u'6766a8ba-132f-4105-ad19-3f579da2b007', 3, u'793df7d5-772a-44b3-9e94-f86d64243281']) storagereplication stdout | 2020-03-31 09:25:16,947 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 45, 16, 928894), 1, u'6766a8ba-132f-4105-ad19-3f579da2b007', 3, u'793df7d5-772a-44b3-9e94-f86d64243281']) 2020-03-31 09:25:16,956 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} storagereplication stdout | 2020-03-31 09:25:16,956 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:25:16,957 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:16,957 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 storagereplication stdout | 2020-03-31 09:25:16,957 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:16,957 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 2020-03-31 09:25:16,960 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:25:16,960 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:25:16,973 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:25:16,973 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:25:16,977 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) storagereplication stdout | 2020-03-31 09:25:16,977 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) 2020-03-31 09:25:16,982 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) storagereplication stdout | 2020-03-31 09:25:16,982 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) 2020-03-31 09:25:16,984 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:25:16,984 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:25:16,985 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:888e8384-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:16 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:25:16,985 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:888e8384-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:16 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:16,986 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=888e8384-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '888e8384-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BQ8nRxJ9j2ysNKydphXeYt26AmF/M4GFOP9FRuFXh7I=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:16 GMT'}. storagereplication stdout | 2020-03-31 09:25:16,986 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=888e8384-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '888e8384-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BQ8nRxJ9j2ysNKydphXeYt26AmF/M4GFOP9FRuFXh7I=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:16 GMT'}. 2020-03-31 09:25:16,989 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 storagereplication stdout | 2020-03-31 09:25:16,989 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 2020-03-31 09:25:17,093 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:25:17,093 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:17,095 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=888e8384-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=feecb497-c01e-004b-3e3e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '888e8384-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BQ8nRxJ9j2ysNKydphXeYt26AmF/M4GFOP9FRuFXh7I=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:16 GMT'}. storagereplication stdout | 2020-03-31 09:25:17,095 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=888e8384-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=feecb497-c01e-004b-3e3e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '888e8384-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BQ8nRxJ9j2ysNKydphXeYt26AmF/M4GFOP9FRuFXh7I=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:16 GMT'}. 2020-03-31 09:25:17,096 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:25:17,096 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:17,097 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:25:17,096 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:25:17,096 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:17,097 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:25:17,098 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:889fab82-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=ojWDyx/dN/FSqeVLF/xm0krCCBfrvGEYvXHt48u7Z5A%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A17Z x-ms-date:Tue, 31 Mar 2020 09:25:17 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:25:17,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=889fab82-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=ojWDyx/dN/FSqeVLF/xm0krCCBfrvGEYvXHt48u7Z5A%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:shNXBWezl3m2RGJsG/nS5eLbLSDBb/cxd0srHm4mwX4=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '889fab82-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. storagereplication stdout | 2020-03-31 09:25:17,098 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:889fab82-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=ojWDyx/dN/FSqeVLF/xm0krCCBfrvGEYvXHt48u7Z5A%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A17Z x-ms-date:Tue, 31 Mar 2020 09:25:17 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:25:17,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=889fab82-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=ojWDyx/dN/FSqeVLF/xm0krCCBfrvGEYvXHt48u7Z5A%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:shNXBWezl3m2RGJsG/nS5eLbLSDBb/cxd0srHm4mwX4=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '889fab82-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. 2020-03-31 09:25:17,136 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:25:17,136 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 2020-03-31 09:25:17,137 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=889fab82-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=feecb49e-c01e-004b-433e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=ojWDyx/dN/FSqeVLF/xm0krCCBfrvGEYvXHt48u7Z5A%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:shNXBWezl3m2RGJsG/nS5eLbLSDBb/cxd0srHm4mwX4=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '889fab82-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. storagereplication stdout | 2020-03-31 09:25:17,137 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=889fab82-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=feecb49e-c01e-004b-433e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=ojWDyx/dN/FSqeVLF/xm0krCCBfrvGEYvXHt48u7Z5A%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:shNXBWezl3m2RGJsG/nS5eLbLSDBb/cxd0srHm4mwX4=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '889fab82-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. 2020-03-31 09:25:17,138 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:25:17,138 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:25:17,139 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:88a60d56-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:17 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:17,140 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '88a60d56-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:By+QVRe0L0FBU6onUQH2dGGtB0Kv252Q6ym9/sWrEmQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. storagereplication stdout | 2020-03-31 09:25:17,139 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:88a60d56-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:17 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:17,140 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '88a60d56-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:By+QVRe0L0FBU6onUQH2dGGtB0Kv252Q6ym9/sWrEmQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. 2020-03-31 09:25:17,143 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): ukquaybackendbmwuskq7s7.blob.core.windows.net:443 storagereplication stdout | 2020-03-31 09:25:17,143 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): ukquaybackendbmwuskq7s7.blob.core.windows.net:443 2020-03-31 09:25:17,564 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:25:17,564 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:17,565 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=d7234297-e01e-0058-4f3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '88a60d56-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:By+QVRe0L0FBU6onUQH2dGGtB0Kv252Q6ym9/sWrEmQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. storagereplication stdout | 2020-03-31 09:25:17,565 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=d7234297-e01e-0058-4f3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '88a60d56-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:By+QVRe0L0FBU6onUQH2dGGtB0Kv252Q6ym9/sWrEmQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:17 GMT'}. 2020-03-31 09:25:17,565 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=d7234297-e01e-0058-4f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:17,565 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=d7234297-e01e-0058-4f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:17,565 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:25:17,565 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=d7234297-e01e-0058-4f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:17,565 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=88a60d56-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:17 GMT, Server-Request-ID=d7234297-e01e-0058-4f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:17,565 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:25:18,273 [527] [DEBUG] [app] Starting request: urn:request:2ae15356-8d5d-4fec-843b-11c7d18e81fa (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:18,273 [527] [DEBUG] [app] Starting request: urn:request:2ae15356-8d5d-4fec-843b-11c7d18e81fa (/health/instance) 2020-03-31 09:25:18,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:18,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:18,282 [556] [DEBUG] [app] Starting request: urn:request:3c88f003-598e-4ebf-b531-f05d9a87d833 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:18,282 [556] [DEBUG] [app] Starting request: urn:request:3c88f003-598e-4ebf-b531-f05d9a87d833 (/v1/_internal_ping) 2020-03-31 09:25:18,283 [556] [DEBUG] [app] Ending request: urn:request:3c88f003-598e-4ebf-b531-f05d9a87d833 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:18,283 [556] [DEBUG] [app] Ending request: urn:request:3c88f003-598e-4ebf-b531-f05d9a87d833 (/v1/_internal_ping) 2020-03-31 09:25:18,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:25:18,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:18,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:18,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:25:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:18,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:18,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:18,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:18,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:18,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:18,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:18,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:18,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:18,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:18,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:18,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:18,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:18,315 [527] [DEBUG] [app] Starting request: urn:request:27df2434-7fdb-4897-8fee-5039c925d1eb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:18,315 [527] [DEBUG] [app] Starting request: urn:request:27df2434-7fdb-4897-8fee-5039c925d1eb (/_internal_ping) 2020-03-31 09:25:18,316 [527] [DEBUG] [app] Ending request: urn:request:27df2434-7fdb-4897-8fee-5039c925d1eb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:18,316 [527] [DEBUG] [app] Ending request: urn:request:27df2434-7fdb-4897-8fee-5039c925d1eb (/_internal_ping) 2020-03-31 09:25:18,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:18,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:18,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:18,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:25:18,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:18,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:18,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:18,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:18,331 [543] [DEBUG] [app] Starting request: urn:request:e2305012-5061-47a6-84ac-7f8da8174b22 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:18,331 [543] [DEBUG] [app] Starting request: urn:request:e2305012-5061-47a6-84ac-7f8da8174b22 (/c1/_internal_ping) 2020-03-31 09:25:18,332 [543] [DEBUG] [app] Ending request: urn:request:e2305012-5061-47a6-84ac-7f8da8174b22 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:18,332 [543] [DEBUG] [app] Ending request: urn:request:e2305012-5061-47a6-84ac-7f8da8174b22 (/c1/_internal_ping) 2020-03-31 09:25:18,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:18,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:18,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:18,334 [527] [DEBUG] [app] Ending request: urn:request:2ae15356-8d5d-4fec-843b-11c7d18e81fa (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:25:18,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:18,334 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:18,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:25:18,334 [527] [DEBUG] [app] Ending request: urn:request:2ae15356-8d5d-4fec-843b-11c7d18e81fa (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:18,334 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:18,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:25:19,567 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8a1877aa-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:19 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:19,567 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1877aa-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:oy4aWFL1aiZD1Wg85q+39CVHrY3G+ANmjs6kvqARyzU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:19 GMT'}. storagereplication stdout | 2020-03-31 09:25:19,567 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8a1877aa-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:19 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:19,567 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1877aa-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:oy4aWFL1aiZD1Wg85q+39CVHrY3G+ANmjs6kvqARyzU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:19 GMT'}. 2020-03-31 09:25:19,647 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:25:19,647 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:19,648 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:19 GMT, Server-Request-ID=d7234699-e01e-0058-6f3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1877aa-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:oy4aWFL1aiZD1Wg85q+39CVHrY3G+ANmjs6kvqARyzU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:19 GMT'}. 2020-03-31 09:25:19,648 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:19 GMT, Server-Request-ID=d7234699-e01e-0058-6f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:19,648 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:19 GMT, Server-Request-ID=d7234699-e01e-0058-6f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:25:19,648 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:19 GMT, Server-Request-ID=d7234699-e01e-0058-6f3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1877aa-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:oy4aWFL1aiZD1Wg85q+39CVHrY3G+ANmjs6kvqARyzU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:19 GMT'}. 2020-03-31 09:25:19,648 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:19 GMT, Server-Request-ID=d7234699-e01e-0058-6f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:19,648 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8a1877aa-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:19 GMT, Server-Request-ID=d7234699-e01e-0058-6f3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:19,648 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:25:19,648 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:25:21,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:25:21,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:31.440707+00:00 (in 9.996887 seconds) 2020-03-31 09:25:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:31 UTC)" (scheduled at 2020-03-31 09:25:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:25:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:31.440707+00:00 (in 9.996887 seconds) 2020-03-31 09:25:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:31 UTC)" (scheduled at 2020-03-31 09:25:21.440707+00:00) 2020-03-31 09:25:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:25:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:25:21,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:25:21,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:25:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:25:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:31 UTC)" executed successfully 2020-03-31 09:25:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:25:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:52 UTC)" (scheduled at 2020-03-31 09:25:22.113221+00:00) securityworker stdout | 2020-03-31 09:25:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:52 UTC)" (scheduled at 2020-03-31 09:25:22.113221+00:00) 2020-03-31 09:25:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:25:22,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:52.113221+00:00 (in 29.997885 seconds) securityworker stdout | 2020-03-31 09:25:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:25:22,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:52.113221+00:00 (in 29.997885 seconds) 2020-03-31 09:25:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:25:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:25:22,137 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE (((("t1"."security_indexed_engine" < %s) AND ("t2"."uploading" = %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [3, False, 3, 7]) securityworker stdout | 2020-03-31 09:25:22,137 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE (((("t1"."security_indexed_engine" < %s) AND ("t2"."uploading" = %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [3, False, 3, 7]) 2020-03-31 09:25:22,143 [187] [INFO] [util.migrate.allocator] Found 4 candidates, processing block securityworker stdout | 2020-03-31 09:25:22,143 [187] [INFO] [util.migrate.allocator] Found 4 candidates, processing block 2020-03-31 09:25:22,143 [187] [INFO] [util.secscan.analyzer] Analyzing layer d2bc9bf1-53b4-4871-925c-c0f25454067a securityworker stdout | 2020-03-31 09:25:22,143 [187] [INFO] [util.secscan.analyzer] Analyzing layer d2bc9bf1-53b4-4871-925c-c0f25454067a 2020-03-31 09:25:22,145 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8ba1bb36-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:22 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:25:22,145 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8ba1bb36-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8ba1bb36-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:7+TUQz92xOFx79utpBbeTj6s9Mq3B+FkXiV4yvvrUlY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:22 GMT'}. securityworker stdout | 2020-03-31 09:25:22,145 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8ba1bb36-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:22 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:25:22,145 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8ba1bb36-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8ba1bb36-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:7+TUQz92xOFx79utpBbeTj6s9Mq3B+FkXiV4yvvrUlY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:22 GMT'}. 2020-03-31 09:25:22,148 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 securityworker stdout | 2020-03-31 09:25:22,148 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaybackendbmwuskq7s7.blob.core.windows.net:443 2020-03-31 09:25:22,247 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:22,247 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:22,248 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8ba1bb36-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:21 GMT, Server-Request-ID=85f9e9be-301e-0094-093e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8ba1bb36-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:7+TUQz92xOFx79utpBbeTj6s9Mq3B+FkXiV4yvvrUlY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:22 GMT'}. securityworker stdout | 2020-03-31 09:25:22,248 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8ba1bb36-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:21 GMT, Server-Request-ID=85f9e9be-301e-0094-093e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8ba1bb36-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:7+TUQz92xOFx79utpBbeTj6s9Mq3B+FkXiV4yvvrUlY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:22 GMT'}. 2020-03-31 09:25:22,249 [187] [INFO] [util.secscan.api] Analyzing layer d2bc9bf1-53b4-4871-925c-c0f25454067a.d910da7a-e575-40de-8965-a8fbd51c3b85 securityworker stdout | 2020-03-31 09:25:22,249 [187] [INFO] [util.secscan.api] Analyzing layer d2bc9bf1-53b4-4871-925c-c0f25454067a.d910da7a-e575-40de-8965-a8fbd51c3b85 2020-03-31 09:25:22,249 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:22,250 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:22,249 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:22,250 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:22,253 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): example-quayecosystem-clair.quay-poc.svc:6060 securityworker stdout | 2020-03-31 09:25:22,253 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): example-quayecosystem-clair.quay-poc.svc:6060 2020-03-31 09:25:22,572 [526] [DEBUG] [app] Starting request: urn:request:013b843d-0f68-4ee0-aa97-ddd38a5eca81 (/keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4) gunicorn-web stdout | 2020-03-31 09:25:22,572 [526] [DEBUG] [app] Starting request: urn:request:013b843d-0f68-4ee0-aa97-ddd38a5eca81 (/keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4) 2020-03-31 09:25:22,574 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', [u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', datetime.datetime(2020, 3, 24, 9, 25, 22, 573378), None, 1, 0]) gunicorn-web stdout | 2020-03-31 09:25:22,574 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', [u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', datetime.datetime(2020, 3, 24, 9, 25, 22, 573378), None, 1, 0]) 2020-03-31 09:25:22,590 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."approver_id", "t1"."approval_type", "t1"."approved_date", "t1"."notes" FROM "servicekeyapproval" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) gunicorn-web stdout | 2020-03-31 09:25:22,590 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."approver_id", "t1"."approval_type", "t1"."approved_date", "t1"."notes" FROM "servicekeyapproval" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [1, 1, 0]) 2020-03-31 09:25:22,594 [526] [DEBUG] [app] Ending request: urn:request:013b843d-0f68-4ee0-aa97-ddd38a5eca81 (/keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4) gunicorn-web stdout | 2020-03-31 09:25:22,594 [526] [DEBUG] [app] Ending request: urn:request:013b843d-0f68-4ee0-aa97-ddd38a5eca81 (/keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4) 2020-03-31 09:25:22,594 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:22,594 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:22,595 [526] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:22 +0000] "GET /keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 HTTP/1.0" 200 374 "-" "KeyRegistryClient/0.1.0" 10.131.0.7 () - - [31/Mar/2020:09:25:22 +0000] "GET /keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 HTTP/2.0" 200 374 "-" "KeyRegistryClient/0.1.0" (0.025 128 0.025) gunicorn-web stdout | 2020-03-31 09:25:22,595 [526] [INFO] [gunicorn.access] 10.131.0.7 - - [31/Mar/2020:09:25:22 +0000] "GET /keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 HTTP/1.0" 200 374 "-" "KeyRegistryClient/0.1.0" nginx stdout | 10.131.0.7 () - - [31/Mar/2020:09:25:22 +0000] "GET /keys/services/quay/keys/29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 HTTP/2.0" 200 374 "-" "KeyRegistryClient/0.1.0" (0.025 128 0.025) 2020-03-31 09:25:23,030 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:23,030 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:23,031 [187] [INFO] [util.secscan.analyzer] Analyzed layer d2bc9bf1-53b4-4871-925c-c0f25454067a successfully with version 3 securityworker stdout | 2020-03-31 09:25:23,031 [187] [INFO] [util.secscan.analyzer] Analyzed layer d2bc9bf1-53b4-4871-925c-c0f25454067a successfully with version 3 2020-03-31 09:25:23,032 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 3, 3, True]) securityworker stdout | 2020-03-31 09:25:23,032 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 3, 3, True]) 2020-03-31 09:25:23,047 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) securityworker stdout | 2020-03-31 09:25:23,047 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:23,053 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [3, False, None, 1585646723, 6]) securityworker stdout | 2020-03-31 09:25:23,053 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [3, False, None, 1585646723, 6]) 2020-03-31 09:25:23,058 [187] [INFO] [util.secscan.analyzer] Analyzing layer bb3f54d6-659e-4c70-afeb-6cc56b6d2d54 securityworker stdout | 2020-03-31 09:25:23,058 [187] [INFO] [util.secscan.analyzer] Analyzing layer bb3f54d6-659e-4c70-afeb-6cc56b6d2d54 2020-03-31 09:25:23,060 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c2d5e20-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 securityworker stdout | 2020-03-31 09:25:23,060 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c2d5e20-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:25:23,062 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c2d5e20-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c2d5e20-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:6uE6Gj10ZodBiGvp5uPPrgmiftNZTa07z98SR7T2E94=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,062 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c2d5e20-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c2d5e20-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:6uE6Gj10ZodBiGvp5uPPrgmiftNZTa07z98SR7T2E94=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,065 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:23,067 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 54 securityworker stdout | 2020-03-31 09:25:23,065 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:23,067 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 500 54 2020-03-31 09:25:23,070 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:23,072 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:23,070 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:23,072 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:23,073 [187] [DEBUG] [util.metrics.prometheus] Register returned 200 for 38 metrics; setting next to 2020-03-31 10:25:23.073479 2020-03-31 09:25:23,074 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c2d5e20-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9ea78-301e-0094-3c3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c2d5e20-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:6uE6Gj10ZodBiGvp5uPPrgmiftNZTa07z98SR7T2E94=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,072 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:23,072 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:23,073 [187] [DEBUG] [util.metrics.prometheus] Register returned 200 for 38 metrics; setting next to 2020-03-31 10:25:23.073479 2020-03-31 09:25:23,074 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c2d5e20-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9ea78-301e-0094-3c3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c2d5e20-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:6uE6Gj10ZodBiGvp5uPPrgmiftNZTa07z98SR7T2E94=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,076 [187] [INFO] [util.secscan.api] Analyzing layer bb3f54d6-659e-4c70-afeb-6cc56b6d2d54.7a10c766-d963-4234-89c4-998a563408cc 2020-03-31 09:25:23,076 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:23,077 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:23,076 [187] [INFO] [util.secscan.api] Analyzing layer bb3f54d6-659e-4c70-afeb-6cc56b6d2d54.7a10c766-d963-4234-89c4-998a563408cc 2020-03-31 09:25:23,076 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:23,077 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:23,126 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 422 None securityworker stdout | 2020-03-31 09:25:23,126 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 422 None 2020-03-31 09:25:23,127 [187] [WARNING] [util.secscan.api] A warning event occurred when analyzing layer bb3f54d6-659e-4c70-afeb-6cc56b6d2d54.7a10c766-d963-4234-89c4-998a563408cc (status code 422): tarutil: could not extract the archive securityworker stdout | 2020-03-31 09:25:23,127 [187] [WARNING] [util.secscan.api] A warning event occurred when analyzing layer bb3f54d6-659e-4c70-afeb-6cc56b6d2d54.7a10c766-d963-4234-89c4-998a563408cc (status code 422): tarutil: could not extract the archive 2020-03-31 09:25:23,128 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [False, 3, 4, 3, False]) securityworker stdout | 2020-03-31 09:25:23,128 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [False, 3, 4, 3, False]) 2020-03-31 09:25:23,143 [187] [INFO] [util.secscan.analyzer] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 securityworker stdout | 2020-03-31 09:25:23,143 [187] [INFO] [util.secscan.analyzer] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 2020-03-31 09:25:23,146 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c3a3e4c-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 securityworker stdout | 2020-03-31 09:25:23,146 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c3a3e4c-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:25:23,146 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c3a3e4c-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c3a3e4c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:q+kVLFmJzqXNsiEokG3ZUbZ+kONFXcGJVCZpUvTOD4M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,146 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c3a3e4c-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c3a3e4c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:q+kVLFmJzqXNsiEokG3ZUbZ+kONFXcGJVCZpUvTOD4M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,148 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:23,148 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:23,151 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:23,151 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:23,152 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics securityworker stdout | 2020-03-31 09:25:23,152 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:23,153 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:23,153 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:23,154 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c3a3e4c-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9ea7a-301e-0094-3e3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c3a3e4c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:q+kVLFmJzqXNsiEokG3ZUbZ+kONFXcGJVCZpUvTOD4M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,154 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c3a3e4c-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9ea7a-301e-0094-3e3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c3a3e4c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:q+kVLFmJzqXNsiEokG3ZUbZ+kONFXcGJVCZpUvTOD4M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,156 [187] [INFO] [util.secscan.api] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075.d910da7a-e575-40de-8965-a8fbd51c3b85 2020-03-31 09:25:23,156 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:25:23,156 [187] [INFO] [util.secscan.api] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075.d910da7a-e575-40de-8965-a8fbd51c3b85 2020-03-31 09:25:23,156 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:23,156 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:23,156 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:23,615 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:23,615 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:23,616 [187] [INFO] [util.secscan.analyzer] Analyzed layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 successfully with version 3 securityworker stdout | 2020-03-31 09:25:23,616 [187] [INFO] [util.secscan.analyzer] Analyzed layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 successfully with version 3 2020-03-31 09:25:23,617 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 5, 3, True]) securityworker stdout | 2020-03-31 09:25:23,617 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 5, 3, True]) 2020-03-31 09:25:23,633 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) securityworker stdout | 2020-03-31 09:25:23,633 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:23,638 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [5, False, None, 1585646723, 6]) securityworker stdout | 2020-03-31 09:25:23,638 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [5, False, None, 1585646723, 6]) 2020-03-31 09:25:23,648 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [5, 1, 0]) securityworker stdout | 2020-03-31 09:25:23,648 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [5, 1, 0]) 2020-03-31 09:25:23,650 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:23,650 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:23,651 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:23,651 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:23,652 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c87d210-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:23,651 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics storagereplication stdout | 2020-03-31 09:25:23,652 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c87d210-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:23,652 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c87d210-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:SjtdhRR/2V+8yYoaxMuaetXPQxnofuNDOmZ0DeCB3IE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,653 [187] [INFO] [util.secscan.analyzer] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 securityworker stdout | 2020-03-31 09:25:23,651 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:23,653 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c88033e-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:25:23,654 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c88033e-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c88033e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:CjsXhAN7jROkR7baLxTwtJa3O5oSZ+sVFI38rAVlmwM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. storagereplication stdout | 2020-03-31 09:25:23,652 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c87d210-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:SjtdhRR/2V+8yYoaxMuaetXPQxnofuNDOmZ0DeCB3IE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,653 [187] [INFO] [util.secscan.analyzer] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 securityworker stdout | 2020-03-31 09:25:23,653 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c88033e-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:25:23,654 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c88033e-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c88033e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:CjsXhAN7jROkR7baLxTwtJa3O5oSZ+sVFI38rAVlmwM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,660 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:23,660 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:23,660 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c88033e-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9eaa3-301e-0094-643e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c88033e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:CjsXhAN7jROkR7baLxTwtJa3O5oSZ+sVFI38rAVlmwM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,660 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c88033e-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9eaa3-301e-0094-643e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c88033e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:CjsXhAN7jROkR7baLxTwtJa3O5oSZ+sVFI38rAVlmwM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,662 [187] [INFO] [util.secscan.api] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075.d910da7a-e575-40de-8965-a8fbd51c3b85 securityworker stdout | 2020-03-31 09:25:23,662 [187] [INFO] [util.secscan.api] Analyzing layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075.d910da7a-e575-40de-8965-a8fbd51c3b85 2020-03-31 09:25:23,663 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:23,663 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:23,663 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:23,663 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:23,684 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:23,684 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:23,685 [187] [INFO] [util.secscan.analyzer] Analyzed layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 successfully with version 3 securityworker stdout | 2020-03-31 09:25:23,685 [187] [INFO] [util.secscan.analyzer] Analyzed layer 8c54d890cf1e549c1a694674ea24d66ae8449d4ae7325e42e9829ae6c9fa9075 successfully with version 3 2020-03-31 09:25:23,686 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 5, 3, True]) securityworker stdout | 2020-03-31 09:25:23,686 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 5, 3, True]) 2020-03-31 09:25:23,700 [187] [INFO] [util.secscan.analyzer] Analyzing layer 37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b securityworker stdout | 2020-03-31 09:25:23,700 [187] [INFO] [util.secscan.analyzer] Analyzing layer 37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b 2020-03-31 09:25:23,701 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c8f33ac-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:23,701 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c8f33ac-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c8f33ac-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1eHAQMdSOVD40NunngTY3gvlHUgxZkFl2u1i+ri0eUY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,701 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8c8f33ac-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:23 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:23,701 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c8f33ac-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c8f33ac-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1eHAQMdSOVD40NunngTY3gvlHUgxZkFl2u1i+ri0eUY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,706 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:23,706 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:23,707 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c8f33ac-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9eaa7-301e-0094-673e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c8f33ac-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1eHAQMdSOVD40NunngTY3gvlHUgxZkFl2u1i+ri0eUY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. securityworker stdout | 2020-03-31 09:25:23,707 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c8f33ac-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:22 GMT, Server-Request-ID=85f9eaa7-301e-0094-673e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c8f33ac-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1eHAQMdSOVD40NunngTY3gvlHUgxZkFl2u1i+ri0eUY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,708 [187] [INFO] [util.secscan.api] Analyzing layer 37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b.0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 2020-03-31 09:25:23,708 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:23,709 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:23,708 [187] [INFO] [util.secscan.api] Analyzing layer 37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b.0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 securityworker stdout | 2020-03-31 09:25:23,708 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:23,709 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:23,732 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:25:23,732 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:23,733 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=d7234e8c-e01e-0058-5a3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c87d210-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:SjtdhRR/2V+8yYoaxMuaetXPQxnofuNDOmZ0DeCB3IE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,733 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=d7234e8c-e01e-0058-5a3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:23,733 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=d7234e8c-e01e-0058-5a3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:23,733 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:25:23,733 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=d7234e8c-e01e-0058-5a3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8c87d210-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:SjtdhRR/2V+8yYoaxMuaetXPQxnofuNDOmZ0DeCB3IE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:23 GMT'}. 2020-03-31 09:25:23,733 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=d7234e8c-e01e-0058-5a3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:23,733 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8c87d210-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=d7234e8c-e01e-0058-5a3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:23,733 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:25:23,989 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:23,989 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:23,990 [187] [INFO] [util.secscan.analyzer] Analyzed layer 37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b successfully with version 3 securityworker stdout | 2020-03-31 09:25:23,990 [187] [INFO] [util.secscan.analyzer] Analyzed layer 37afdd2a15d0e50ae3fd1f445a460296b2f3d9616678685225a97fc26cbaee3b successfully with version 3 2020-03-31 09:25:23,991 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 6, 3, True]) securityworker stdout | 2020-03-31 09:25:23,991 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 6, 3, True]) 2020-03-31 09:25:24,008 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) securityworker stdout | 2020-03-31 09:25:24,008 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:24,014 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [6, False, None, 1585646724, 6]) securityworker stdout | 2020-03-31 09:25:24,014 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [6, False, None, 1585646724, 6]) 2020-03-31 09:25:24,019 [187] [INFO] [util.migrate.allocator] Marking id range as completed: 3-7 securityworker stdout | 2020-03-31 09:25:24,019 [187] [INFO] [util.migrate.allocator] Marking id range as completed: 3-7 2020-03-31 09:25:24,022 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:24,022 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:24,027 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE (((("t1"."security_indexed_engine" < %s) AND ("t2"."uploading" = %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [3, False, 1, 5]) securityworker stdout | 2020-03-31 09:25:24,027 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE (((("t1"."security_indexed_engine" < %s) AND ("t2"."uploading" = %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [3, False, 1, 5]) 2020-03-31 09:25:24,027 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,027 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:24,028 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics securityworker stdout | 2020-03-31 09:25:24,028 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:24,032 [187] [INFO] [util.migrate.allocator] Found 2 candidates, processing block securityworker stdout | 2020-03-31 09:25:24,032 [187] [INFO] [util.migrate.allocator] Found 2 candidates, processing block 2020-03-31 09:25:24,032 [187] [INFO] [util.secscan.analyzer] Analyzing layer b32fd55c-5018-402a-9b45-5f6a883e89ad 2020-03-31 09:25:24,033 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8cc1e19e-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 securityworker stdout | 2020-03-31 09:25:24,032 [187] [INFO] [util.secscan.analyzer] Analyzing layer b32fd55c-5018-402a-9b45-5f6a883e89ad 2020-03-31 09:25:24,033 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8cc1e19e-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:24,033 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cc1e19e-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cc1e19e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aVZ/8U5ZnOUcvyGmkqYvOMSuYABJ1bunKQOOykONHlg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,033 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cc1e19e-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cc1e19e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aVZ/8U5ZnOUcvyGmkqYvOMSuYABJ1bunKQOOykONHlg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,038 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,038 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:24,039 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cc1e19e-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eab2-301e-0094-723e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cc1e19e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aVZ/8U5ZnOUcvyGmkqYvOMSuYABJ1bunKQOOykONHlg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,039 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cc1e19e-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eab2-301e-0094-723e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cc1e19e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:aVZ/8U5ZnOUcvyGmkqYvOMSuYABJ1bunKQOOykONHlg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,040 [187] [INFO] [util.secscan.api] Analyzing layer b32fd55c-5018-402a-9b45-5f6a883e89ad.229ce905-1640-4594-aae8-fb66fcc5e51f 2020-03-31 09:25:24,040 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:25:24,040 [187] [INFO] [util.secscan.api] Analyzing layer b32fd55c-5018-402a-9b45-5f6a883e89ad.229ce905-1640-4594-aae8-fb66fcc5e51f 2020-03-31 09:25:24,040 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,040 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:24,040 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:24,192 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:24,192 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:24,192 [187] [INFO] [util.secscan.analyzer] Analyzed layer b32fd55c-5018-402a-9b45-5f6a883e89ad successfully with version 3 securityworker stdout | 2020-03-31 09:25:24,192 [187] [INFO] [util.secscan.analyzer] Analyzed layer b32fd55c-5018-402a-9b45-5f6a883e89ad successfully with version 3 2020-03-31 09:25:24,193 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 1, 3, True]) securityworker stdout | 2020-03-31 09:25:24,193 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 1, 3, True]) 2020-03-31 09:25:24,209 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) securityworker stdout | 2020-03-31 09:25:24,209 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:24,214 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [1, False, None, 1585646724, 6]) securityworker stdout | 2020-03-31 09:25:24,214 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [1, False, None, 1585646724, 6]) 2020-03-31 09:25:24,221 [187] [INFO] [util.secscan.analyzer] Analyzing layer dd3b0723-1ea2-44b7-ad13-8d2b207b61b3 securityworker stdout | 2020-03-31 09:25:24,221 [187] [INFO] [util.secscan.analyzer] Analyzing layer dd3b0723-1ea2-44b7-ad13-8d2b207b61b3 2020-03-31 09:25:24,224 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8cdeced0-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 securityworker stdout | 2020-03-31 09:25:24,224 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8cdeced0-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:24,224 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cdeced0-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cdeced0-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/Wfd66nKjxhr4Mjyqe5vzG2Ay+ssnMBf9D0Q3toXprQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,225 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:24,224 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cdeced0-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cdeced0-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/Wfd66nKjxhr4Mjyqe5vzG2Ay+ssnMBf9D0Q3toXprQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,225 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:24,228 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,228 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:24,230 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics securityworker stdout | 2020-03-31 09:25:24,230 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:24,232 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,232 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:24,233 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cdeced0-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eac4-301e-0094-033e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cdeced0-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/Wfd66nKjxhr4Mjyqe5vzG2Ay+ssnMBf9D0Q3toXprQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,233 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8cdeced0-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eac4-301e-0094-033e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8cdeced0-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/Wfd66nKjxhr4Mjyqe5vzG2Ay+ssnMBf9D0Q3toXprQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,234 [187] [INFO] [util.secscan.api] Analyzing layer dd3b0723-1ea2-44b7-ad13-8d2b207b61b3.0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 securityworker stdout | 2020-03-31 09:25:24,234 [187] [INFO] [util.secscan.api] Analyzing layer dd3b0723-1ea2-44b7-ad13-8d2b207b61b3.0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 2020-03-31 09:25:24,234 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,234 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:24,234 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,234 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:25:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:34.323352+00:00 (in 9.999296 seconds) 2020-03-31 09:25:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:34 UTC)" (scheduled at 2020-03-31 09:25:24.323352+00:00) 2020-03-31 09:25:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:25:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:34.323352+00:00 (in 9.999296 seconds) 2020-03-31 09:25:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:34 UTC)" (scheduled at 2020-03-31 09:25:24.323352+00:00) 2020-03-31 09:25:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 24, 325089), True, datetime.datetime(2020, 3, 31, 9, 25, 24, 325089), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:25:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 24, 325089), True, datetime.datetime(2020, 3, 31, 9, 25, 24, 325089), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:25:24,354 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:25:24,354 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:24,354 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,354 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:25:24,354 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,354 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:34 UTC)" executed successfully 2020-03-31 09:25:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:25:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:33.330112+00:00 (in 8.918154 seconds) 2020-03-31 09:25:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:24 UTC)" (scheduled at 2020-03-31 09:25:24.410927+00:00) 2020-03-31 09:25:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:25:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:33.330112+00:00 (in 8.918154 seconds) 2020-03-31 09:25:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:24 UTC)" (scheduled at 2020-03-31 09:25:24.410927+00:00) 2020-03-31 09:25:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:24 UTC)" executed successfully 2020-03-31 09:25:24,445 [518] [DEBUG] [app] Starting request: urn:request:e8b24179-5341-47ef-a248-6342fbf97fb8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:24,445 [518] [DEBUG] [app] Starting request: urn:request:e8b24179-5341-47ef-a248-6342fbf97fb8 (/health/instance) 2020-03-31 09:25:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:24,453 [538] [DEBUG] [app] Starting request: urn:request:26283879-bed9-4830-b1df-ef75e2e0c7d5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:24,453 [538] [DEBUG] [app] Starting request: urn:request:26283879-bed9-4830-b1df-ef75e2e0c7d5 (/v1/_internal_ping) 2020-03-31 09:25:24,454 [538] [DEBUG] [app] Ending request: urn:request:26283879-bed9-4830-b1df-ef75e2e0c7d5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:24,454 [538] [DEBUG] [app] Ending request: urn:request:26283879-bed9-4830-b1df-ef75e2e0c7d5 (/v1/_internal_ping) 2020-03-31 09:25:24,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:25:24,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:25:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:24,457 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:24,457 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:24,458 [187] [INFO] [util.secscan.analyzer] Analyzed layer dd3b0723-1ea2-44b7-ad13-8d2b207b61b3 successfully with version 3 securityworker stdout | 2020-03-31 09:25:24,458 [187] [INFO] [util.secscan.analyzer] Analyzed layer dd3b0723-1ea2-44b7-ad13-8d2b207b61b3 successfully with version 3 2020-03-31 09:25:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:24,459 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 2, 3, True]) gunicorn-registry stdout | 2020-03-31 09:25:24,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,459 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 2, 3, True]) 2020-03-31 09:25:24,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:24,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:24,476 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:24,476 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) gunicorn-web stdout | 2020-03-31 09:25:24,476 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) securityworker stdout | 2020-03-31 09:25:24,476 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:24,485 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [2, False, None, 1585646724, 6]) securityworker stdout | 2020-03-31 09:25:24,485 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [2, False, None, 1585646724, 6]) 2020-03-31 09:25:24,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:24,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:24,495 [187] [INFO] [util.migrate.allocator] Marking id range as completed: 1-3 securityworker stdout | 2020-03-31 09:25:24,495 [187] [INFO] [util.migrate.allocator] Marking id range as completed: 1-3 2020-03-31 09:25:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:24,502 [518] [DEBUG] [app] Starting request: urn:request:85853f6b-8f9d-48e7-aa37-e277cd5201b7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:24,502 [518] [DEBUG] [app] Starting request: urn:request:85853f6b-8f9d-48e7-aa37-e277cd5201b7 (/_internal_ping) 2020-03-31 09:25:24,503 [518] [DEBUG] [app] Ending request: urn:request:85853f6b-8f9d-48e7-aa37-e277cd5201b7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:24,503 [518] [DEBUG] [app] Ending request: urn:request:85853f6b-8f9d-48e7-aa37-e277cd5201b7 (/_internal_ping) 2020-03-31 09:25:24,504 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:25:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:24,504 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE (((("t1"."security_indexed_engine" < %s) AND ("t2"."uploading" = %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [3, False, 7, 10]) 2020-03-31 09:25:24,504 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:25:24,504 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:25:24,505 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:25:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:24,506 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics securityworker stdout | 2020-03-31 09:25:24,504 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE (((("t1"."security_indexed_engine" < %s) AND ("t2"."uploading" = %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [3, False, 7, 10]) 2020-03-31 09:25:24,504 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:24,505 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) securityworker stdout | 2020-03-31 09:25:24,506 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:24,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:24,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:25:24,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:24,512 [187] [INFO] [util.migrate.allocator] Found 3 candidates, processing block 2020-03-31 09:25:24,512 [187] [INFO] [util.secscan.analyzer] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb 2020-03-31 09:25:24,513 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d0b2a84-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 gunicorn-web stdout | 2020-03-31 09:25:24,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:24,513 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d0b2a84-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d0b2a84-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:mo0h1ptY6K0aLlufUGdCN53d6E8iM10Wym5JoIr4oSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,512 [187] [INFO] [util.migrate.allocator] Found 3 candidates, processing block 2020-03-31 09:25:24,512 [187] [INFO] [util.secscan.analyzer] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb 2020-03-31 09:25:24,513 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d0b2a84-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 securityworker stdout | 2020-03-31 09:25:24,513 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d0b2a84-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d0b2a84-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:mo0h1ptY6K0aLlufUGdCN53d6E8iM10Wym5JoIr4oSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,516 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:24,516 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:24,518 [539] [DEBUG] [app] Starting request: urn:request:cdd3017d-4ab5-4a18-8813-4e0f02c03003 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:24,518 [539] [DEBUG] [app] Starting request: urn:request:cdd3017d-4ab5-4a18-8813-4e0f02c03003 (/c1/_internal_ping) 2020-03-31 09:25:24,519 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,519 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:24,520 [539] [DEBUG] [app] Ending request: urn:request:cdd3017d-4ab5-4a18-8813-4e0f02c03003 (/c1/_internal_ping) 2020-03-31 09:25:24,520 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:24,520 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:24,520 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d0b2a84-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eaea-301e-0094-273e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d0b2a84-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:mo0h1ptY6K0aLlufUGdCN53d6E8iM10Wym5JoIr4oSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. gunicorn-verbs stdout | 2020-03-31 09:25:24,520 [539] [DEBUG] [app] Ending request: urn:request:cdd3017d-4ab5-4a18-8813-4e0f02c03003 (/c1/_internal_ping) 2020-03-31 09:25:24,520 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:24,522 [187] [INFO] [util.secscan.api] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,522 [187] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:24,520 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:24,522 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:24,520 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d0b2a84-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eaea-301e-0094-273e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d0b2a84-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:mo0h1ptY6K0aLlufUGdCN53d6E8iM10Wym5JoIr4oSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,523 [518] [DEBUG] [app] Ending request: urn:request:e8b24179-5341-47ef-a248-6342fbf97fb8 (/health/instance) 2020-03-31 09:25:24,523 [518] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:25:24,522 [187] [INFO] [util.secscan.api] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,522 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,524 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) securityworker stdout | 2020-03-31 09:25:24,522 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers gunicorn-web stdout | 2020-03-31 09:25:24,523 [518] [DEBUG] [app] Ending request: urn:request:e8b24179-5341-47ef-a248-6342fbf97fb8 (/health/instance) 2020-03-31 09:25:24,523 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:24,524 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) 2020-03-31 09:25:24,585 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:24,585 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:24,586 [187] [INFO] [util.secscan.analyzer] Analyzed layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb successfully with version 3 securityworker stdout | 2020-03-31 09:25:24,586 [187] [INFO] [util.secscan.analyzer] Analyzed layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb successfully with version 3 2020-03-31 09:25:24,587 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 7, 3, True]) securityworker stdout | 2020-03-31 09:25:24,587 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 7, 3, True]) 2020-03-31 09:25:24,603 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) securityworker stdout | 2020-03-31 09:25:24,603 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:24,608 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [7, False, None, 1585646724, 6]) securityworker stdout | 2020-03-31 09:25:24,608 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [7, False, None, 1585646724, 6]) 2020-03-31 09:25:24,621 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [7, 1, 0]) securityworker stdout | 2020-03-31 09:25:24,621 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [7, 1, 0]) 2020-03-31 09:25:24,623 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:24,623 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:24,624 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,624 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:24,624 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics securityworker stdout | 2020-03-31 09:25:24,624 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:24,627 [187] [INFO] [util.secscan.analyzer] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb securityworker stdout | 2020-03-31 09:25:24,627 [187] [INFO] [util.secscan.analyzer] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb 2020-03-31 09:25:24,627 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d1c959e-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,627 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d1c959e-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d1c959e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:phlTTmogAq+/ymwUiCnLDrGwa6FhtocqE4FSAGHl9p8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,627 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d1c959e-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,627 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d1c959e-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d1c959e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:phlTTmogAq+/ymwUiCnLDrGwa6FhtocqE4FSAGHl9p8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,632 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,632 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:24,633 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d1c959e-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eaf9-301e-0094-363e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d1c959e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:phlTTmogAq+/ymwUiCnLDrGwa6FhtocqE4FSAGHl9p8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,634 [187] [INFO] [util.secscan.api] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c securityworker stdout | 2020-03-31 09:25:24,633 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d1c959e-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eaf9-301e-0094-363e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d1c959e-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:phlTTmogAq+/ymwUiCnLDrGwa6FhtocqE4FSAGHl9p8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,635 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,635 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:24,634 [187] [INFO] [util.secscan.api] Analyzing layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c securityworker stdout | 2020-03-31 09:25:24,635 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,635 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:24,663 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:24,663 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:24,664 [187] [INFO] [util.secscan.analyzer] Analyzed layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb successfully with version 3 securityworker stdout | 2020-03-31 09:25:24,664 [187] [INFO] [util.secscan.analyzer] Analyzed layer 2dabbd6aa3cb2de3e21a052b821ce0b581061cb008c13da616360cf8bf7f94eb successfully with version 3 2020-03-31 09:25:24,665 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 7, 3, True]) securityworker stdout | 2020-03-31 09:25:24,665 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 7, 3, True]) 2020-03-31 09:25:24,676 [187] [INFO] [util.secscan.analyzer] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 securityworker stdout | 2020-03-31 09:25:24,676 [187] [INFO] [util.secscan.analyzer] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 2020-03-31 09:25:24,677 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d242d7c-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,677 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d242d7c-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d242d7c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:jLioxnOi+NZtHbqQq3kWuFxP+4tIMKUBZweeMgI8GUE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,677 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d242d7c-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,677 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d242d7c-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d242d7c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:jLioxnOi+NZtHbqQq3kWuFxP+4tIMKUBZweeMgI8GUE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,683 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,683 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:24,684 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d242d7c-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eaff-301e-0094-3c3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d242d7c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:jLioxnOi+NZtHbqQq3kWuFxP+4tIMKUBZweeMgI8GUE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,684 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d242d7c-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:23 GMT, Server-Request-ID=85f9eaff-301e-0094-3c3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d242d7c-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:jLioxnOi+NZtHbqQq3kWuFxP+4tIMKUBZweeMgI8GUE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,685 [187] [INFO] [util.secscan.api] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,685 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,685 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:24,685 [187] [INFO] [util.secscan.api] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,685 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,685 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:24,738 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:24,738 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:24,739 [187] [INFO] [util.secscan.analyzer] Analyzed layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 successfully with version 3 securityworker stdout | 2020-03-31 09:25:24,739 [187] [INFO] [util.secscan.analyzer] Analyzed layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 successfully with version 3 2020-03-31 09:25:24,740 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 8, 3, True]) securityworker stdout | 2020-03-31 09:25:24,740 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 8, 3, True]) 2020-03-31 09:25:24,756 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) securityworker stdout | 2020-03-31 09:25:24,756 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:24,761 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [8, False, None, 1585646724, 6]) securityworker stdout | 2020-03-31 09:25:24,761 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [8, False, None, 1585646724, 6]) 2020-03-31 09:25:24,771 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [8, 1, 0]) securityworker stdout | 2020-03-31 09:25:24,771 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."docker_image_id", "t1"."repository_id", "t1"."ancestors", "t1"."storage_id", "t1"."created", "t1"."comment", "t1"."command", "t1"."aggregate_size", "t1"."v1_json_metadata", "t1"."v1_checksum", "t1"."security_indexed", "t1"."security_indexed_engine", "t1"."parent_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum", "t3"."id", "t3"."docker_image_id", "t3"."repository_id", "t3"."ancestors", "t3"."storage_id", "t3"."created", "t3"."comment", "t3"."command", "t3"."aggregate_size", "t3"."v1_json_metadata", "t3"."v1_checksum", "t3"."security_indexed", "t3"."security_indexed_engine", "t3"."parent_id", "t4"."id", "t4"."uuid", "t4"."image_size", "t4"."uncompressed_size", "t4"."uploading", "t4"."cas_path", "t4"."content_checksum" FROM "image" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") LEFT OUTER JOIN "image" AS "t3" ON ("t1"."parent_id" = "t3"."id") LEFT OUTER JOIN "imagestorage" AS "t4" ON ("t4"."id" = "t3"."storage_id") WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [8, 1, 0]) 2020-03-31 09:25:24,773 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:24,773 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:24,774 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,774 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:24,775 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics securityworker stdout | 2020-03-31 09:25:24,775 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:24,777 [187] [INFO] [util.secscan.analyzer] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 securityworker stdout | 2020-03-31 09:25:24,777 [187] [INFO] [util.secscan.analyzer] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 2020-03-31 09:25:24,778 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d339348-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,778 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d339348-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d339348-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1jswVoioPTjB4tnXiU6PIzRO5fODkoiu4hEzISSdHtg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,778 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d339348-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,778 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d339348-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d339348-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1jswVoioPTjB4tnXiU6PIzRO5fODkoiu4hEzISSdHtg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,783 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,783 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:24,784 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d339348-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:24 GMT, Server-Request-ID=85f9eb02-301e-0094-3f3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d339348-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1jswVoioPTjB4tnXiU6PIzRO5fODkoiu4hEzISSdHtg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,784 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d339348-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:24 GMT, Server-Request-ID=85f9eb02-301e-0094-3f3e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d339348-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:1jswVoioPTjB4tnXiU6PIzRO5fODkoiu4hEzISSdHtg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,786 [187] [INFO] [util.secscan.api] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,786 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,786 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:24,786 [187] [INFO] [util.secscan.api] Analyzing layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,786 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,786 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:24,818 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:24,818 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:24,819 [187] [INFO] [util.secscan.analyzer] Analyzed layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 successfully with version 3 securityworker stdout | 2020-03-31 09:25:24,819 [187] [INFO] [util.secscan.analyzer] Analyzed layer 3ff6bb3c87ca1f24fb5f064e132ffe109ace73f5ec72caba0229b324fe0f1ba1 successfully with version 3 2020-03-31 09:25:24,820 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 8, 3, True]) securityworker stdout | 2020-03-31 09:25:24,820 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 8, 3, True]) 2020-03-31 09:25:24,834 [187] [INFO] [util.secscan.analyzer] Analyzing layer 69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129 securityworker stdout | 2020-03-31 09:25:24,834 [187] [INFO] [util.secscan.analyzer] Analyzing layer 69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129 2020-03-31 09:25:24,835 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d3c3d40-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,835 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d3c3d40-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d3c3d40-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/en3Jlc8oNSRrU93Lgoklkv5aC5ZuHjitVZheRmPEY0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,835 [187] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8d3c3d40-7331-11ea-a779-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:24 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 timeout:20 2020-03-31 09:25:24,835 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d3c3d40-7331-11ea-a779-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d3c3d40-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/en3Jlc8oNSRrU93Lgoklkv5aC5ZuHjitVZheRmPEY0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,841 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:24,841 [187] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:24,842 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d3c3d40-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:24 GMT, Server-Request-ID=85f9eb09-301e-0094-453e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d3c3d40-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/en3Jlc8oNSRrU93Lgoklkv5aC5ZuHjitVZheRmPEY0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. securityworker stdout | 2020-03-31 09:25:24,842 [187] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8d3c3d40-7331-11ea-a779-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:24 GMT, Server-Request-ID=85f9eb09-301e-0094-453e-076806000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8d3c3d40-7331-11ea-a779-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/en3Jlc8oNSRrU93Lgoklkv5aC5ZuHjitVZheRmPEY0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:24 GMT'}. 2020-03-31 09:25:24,844 [187] [INFO] [util.secscan.api] Analyzing layer 69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,844 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,844 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers securityworker stdout | 2020-03-31 09:25:24,844 [187] [INFO] [util.secscan.api] Analyzing layer 69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129.98183bb5-7f9a-4d7d-ab4f-6bc45984f87c 2020-03-31 09:25:24,844 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,844 [187] [DEBUG] [util.secscan.api] POSTing security URL https://example-quayecosystem-clair.quay-poc.svc:6060/v1/layers 2020-03-31 09:25:24,966 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None securityworker stdout | 2020-03-31 09:25:24,966 [187] [DEBUG] [urllib3.connectionpool] https://example-quayecosystem-clair.quay-poc.svc:6060 "POST /v1/layers HTTP/1.1" 201 None 2020-03-31 09:25:24,967 [187] [INFO] [util.secscan.analyzer] Analyzed layer 69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129 successfully with version 3 securityworker stdout | 2020-03-31 09:25:24,967 [187] [INFO] [util.secscan.analyzer] Analyzed layer 69ab8dd63cef7371fbbd624de76c6fe89e1f7fd237563fc414ca74ff70eb5129 successfully with version 3 2020-03-31 09:25:24,968 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 9, 3, True]) securityworker stdout | 2020-03-31 09:25:24,968 [187] [DEBUG] [peewee] ('UPDATE "image" SET "security_indexed" = %s, "security_indexed_engine" = %s WHERE (("image"."id" = %s) AND (("image"."security_indexed_engine" != %s) OR ("image"."security_indexed" != %s)))', [True, 3, 9, 3, True]) 2020-03-31 09:25:24,984 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) securityworker stdout | 2020-03-31 09:25:24,984 [187] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "externalnotificationevent" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', [u'vulnerability_found', 1, 0]) 2020-03-31 09:25:24,990 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [9, False, None, 1585646724, 6]) securityworker stdout | 2020-03-31 09:25:24,990 [187] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."id", "t1"."name", "t1"."image_id", "t1"."repository_id", "t1"."lifetime_start_ts", "t1"."lifetime_end_ts", "t1"."hidden", "t1"."reversion" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "repositorynotification" AS "t3" ON ("t3"."repository_id" = "t2"."id") INNER JOIN "image" AS "t4" ON ("t1"."image_id" = "t4"."id") INNER JOIN "imagestorage" AS "t5" ON ("t4"."storage_id" = "t5"."id") WHERE (((("t1"."image_id" = %s) AND ("t1"."hidden" = %s)) AND (("t1"."lifetime_end_ts" IS %s) OR ("t1"."lifetime_end_ts" > %s))) AND ("t3"."event_id" = %s)) ORDER BY "t1"."lifetime_start_ts" DESC', [9, False, None, 1585646724, 6]) 2020-03-31 09:25:24,996 [187] [INFO] [util.migrate.allocator] Marking id range as completed: 7-10 securityworker stdout | 2020-03-31 09:25:24,996 [187] [INFO] [util.migrate.allocator] Marking id range as completed: 7-10 2020-03-31 09:25:24,997 [187] [INFO] [util.migrate.allocator] No more work 2020-03-31 09:25:24,998 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:25:24,997 [187] [INFO] [util.migrate.allocator] No more work 2020-03-31 09:25:24,998 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:24,999 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:25:24,999 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:25:52 UTC)" executed successfully 2020-03-31 09:25:25,000 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 securityworker stdout | 2020-03-31 09:25:25,000 [187] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:25,001 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 securityworker stdout | 2020-03-31 09:25:25,001 [187] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:25,002 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics securityworker stdout | 2020-03-31 09:25:25,002 [187] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:25:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:25:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:26,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:36.924497+00:00 (in 9.998112 seconds) storagereplication stdout | 2020-03-31 09:25:26,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:36.924497+00:00 (in 9.998112 seconds) 2020-03-31 09:25:28,272 [518] [DEBUG] [app] Starting request: urn:request:9817ff6c-138e-4c7b-b3ab-74f08969d213 (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:28,272 [518] [DEBUG] [app] Starting request: urn:request:9817ff6c-138e-4c7b-b3ab-74f08969d213 (/health/instance) 2020-03-31 09:25:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:28,278 [538] [DEBUG] [app] Starting request: urn:request:2fb8072e-1d4b-455e-8735-31147dacfe16 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:28,278 [538] [DEBUG] [app] Starting request: urn:request:2fb8072e-1d4b-455e-8735-31147dacfe16 (/v1/_internal_ping) 2020-03-31 09:25:28,279 [538] [DEBUG] [app] Ending request: urn:request:2fb8072e-1d4b-455e-8735-31147dacfe16 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:28,279 [538] [DEBUG] [app] Ending request: urn:request:2fb8072e-1d4b-455e-8735-31147dacfe16 (/v1/_internal_ping) 2020-03-31 09:25:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:25:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:28,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:25:28,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:25:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:25:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:28,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:28,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:28,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:28,314 [520] [DEBUG] [app] Starting request: urn:request:a0048909-d432-45fb-9b4a-7a2c462a8353 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:28,314 [520] [DEBUG] [app] Starting request: urn:request:a0048909-d432-45fb-9b4a-7a2c462a8353 (/_internal_ping) 2020-03-31 09:25:28,315 [520] [DEBUG] [app] Ending request: urn:request:a0048909-d432-45fb-9b4a-7a2c462a8353 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:28,315 [520] [DEBUG] [app] Ending request: urn:request:a0048909-d432-45fb-9b4a-7a2c462a8353 (/_internal_ping) 2020-03-31 09:25:28,316 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:28,316 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:25:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:25:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:28,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:28,329 [541] [DEBUG] [app] Starting request: urn:request:fd064217-9304-4b5e-98c3-5f9179dc0b31 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:28,329 [541] [DEBUG] [app] Starting request: urn:request:fd064217-9304-4b5e-98c3-5f9179dc0b31 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:28,329 [541] [DEBUG] [app] Ending request: urn:request:fd064217-9304-4b5e-98c3-5f9179dc0b31 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:28,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:25:28,329 [541] [DEBUG] [app] Ending request: urn:request:fd064217-9304-4b5e-98c3-5f9179dc0b31 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:28,331 [518] [DEBUG] [app] Ending request: urn:request:9817ff6c-138e-4c7b-b3ab-74f08969d213 (/health/instance) 2020-03-31 09:25:28,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:25:28,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:28,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:25:28,331 [518] [DEBUG] [app] Ending request: urn:request:9817ff6c-138e-4c7b-b3ab-74f08969d213 (/health/instance) 2020-03-31 09:25:28,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:28,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:25:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:25:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:00.337699+00:00 (in 29.996954 seconds) 2020-03-31 09:25:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:00 UTC)" (scheduled at 2020-03-31 09:25:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:25:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:00.337699+00:00 (in 29.996954 seconds) 2020-03-31 09:25:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:00 UTC)" (scheduled at 2020-03-31 09:25:30.337699+00:00) 2020-03-31 09:25:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 25, 30, 341592), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:25:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 25, 30, 341592), False, 50, 1, 0]) 2020-03-31 09:25:30,359 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:25:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:25:30,359 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:25:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:00 UTC)" executed successfully 2020-03-31 09:25:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:25:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:31 UTC)" (scheduled at 2020-03-31 09:25:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:25:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:31 UTC)" (scheduled at 2020-03-31 09:25:31.440707+00:00) 2020-03-31 09:25:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:25:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:25:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:41.440707+00:00 (in 9.999164 seconds) repositoryactioncounter stdout | 2020-03-31 09:25:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:41.440707+00:00 (in 9.999164 seconds) 2020-03-31 09:25:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:25:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:25:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:25:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:41 UTC)" executed successfully 2020-03-31 09:25:31,741 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:915a1a64-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:31 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:25:31,741 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:915a1a64-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:31 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:25:31,741 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '915a1a64-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tfWz6Pr8qtt56ziMCuTInN0hPiybw3CeTK9uxXwsnSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:31 GMT'}. storagereplication stdout | 2020-03-31 09:25:31,741 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '915a1a64-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tfWz6Pr8qtt56ziMCuTInN0hPiybw3CeTK9uxXwsnSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:31 GMT'}. 2020-03-31 09:25:31,827 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:25:31,827 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:31,828 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:31 GMT, Server-Request-ID=d7235d9f-e01e-0058-1b3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '915a1a64-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tfWz6Pr8qtt56ziMCuTInN0hPiybw3CeTK9uxXwsnSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:31 GMT'}. 2020-03-31 09:25:31,828 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:31 GMT, Server-Request-ID=d7235d9f-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:31,828 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:31 GMT, Server-Request-ID=d7235d9f-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:31,828 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:25:31,828 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:31 GMT, Server-Request-ID=d7235d9f-e01e-0058-1b3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '915a1a64-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tfWz6Pr8qtt56ziMCuTInN0hPiybw3CeTK9uxXwsnSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:31 GMT'}. 2020-03-31 09:25:31,828 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:31 GMT, Server-Request-ID=d7235d9f-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:31,828 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=915a1a64-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:31 GMT, Server-Request-ID=d7235d9f-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:25:31,828 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:25:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:25:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:03.330112+00:00 (in 29.999228 seconds) 2020-03-31 09:25:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:03 UTC)" (scheduled at 2020-03-31 09:25:33.330112+00:00) 2020-03-31 09:25:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:25:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:03.330112+00:00 (in 29.999228 seconds) 2020-03-31 09:25:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:03 UTC)" (scheduled at 2020-03-31 09:25:33.330112+00:00) 2020-03-31 09:25:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 33, 331979), True, datetime.datetime(2020, 3, 31, 9, 25, 33, 331979), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:25:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 33, 331979), True, datetime.datetime(2020, 3, 31, 9, 25, 33, 331979), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:25:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:25:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:03 UTC)" executed successfully 2020-03-31 09:25:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:25:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:43.539999+00:00 (in 10.172404 seconds) exportactionlogsworker stdout | 2020-03-31 09:25:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:43.539999+00:00 (in 10.172404 seconds) 2020-03-31 09:25:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:33 UTC)" (scheduled at 2020-03-31 09:25:33.366791+00:00) 2020-03-31 09:25:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:25:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:33 UTC)" (scheduled at 2020-03-31 09:25:33.366791+00:00) 2020-03-31 09:25:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 33, 368477), True, datetime.datetime(2020, 3, 31, 9, 25, 33, 368477), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:25:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 33, 368477), True, datetime.datetime(2020, 3, 31, 9, 25, 33, 368477), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:25:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:25:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:25:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:33 UTC)" executed successfully 2020-03-31 09:25:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:25:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:44.323352+00:00 (in 9.998706 seconds) 2020-03-31 09:25:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:44 UTC)" (scheduled at 2020-03-31 09:25:34.323352+00:00) 2020-03-31 09:25:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:25:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:44.323352+00:00 (in 9.998706 seconds) 2020-03-31 09:25:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:44 UTC)" (scheduled at 2020-03-31 09:25:34.323352+00:00) 2020-03-31 09:25:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 34, 325595), True, datetime.datetime(2020, 3, 31, 9, 25, 34, 325595), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:25:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 34, 325595), True, datetime.datetime(2020, 3, 31, 9, 25, 34, 325595), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:25:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:25:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:25:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:44 UTC)" executed successfully 2020-03-31 09:25:34,446 [526] [DEBUG] [app] Starting request: urn:request:216626d4-dade-4c2c-8ed9-da33ebb3c9e2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:34,446 [526] [DEBUG] [app] Starting request: urn:request:216626d4-dade-4c2c-8ed9-da33ebb3c9e2 (/health/instance) 2020-03-31 09:25:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:34,454 [538] [DEBUG] [app] Starting request: urn:request:4fe5fdb3-2507-4a43-bd25-384189d79c1f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:34,454 [538] [DEBUG] [app] Starting request: urn:request:4fe5fdb3-2507-4a43-bd25-384189d79c1f (/v1/_internal_ping) 2020-03-31 09:25:34,456 [538] [DEBUG] [app] Ending request: urn:request:4fe5fdb3-2507-4a43-bd25-384189d79c1f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:34,456 [538] [DEBUG] [app] Ending request: urn:request:4fe5fdb3-2507-4a43-bd25-384189d79c1f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:25:34,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:34,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:34,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:25:34,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:34,457 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:34,458 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:34,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:34,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:34,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:34,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:34,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:34,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:34,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:34,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:34,489 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:34,489 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:34,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:34,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:34,498 [527] [DEBUG] [app] Starting request: urn:request:fd1605ec-c11b-4e52-a032-32917573eeaa (/_internal_ping) 2020-03-31 09:25:34,499 [527] [DEBUG] [app] Ending request: urn:request:fd1605ec-c11b-4e52-a032-32917573eeaa (/_internal_ping) 2020-03-31 09:25:34,500 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:34,498 [527] [DEBUG] [app] Starting request: urn:request:fd1605ec-c11b-4e52-a032-32917573eeaa (/_internal_ping) 2020-03-31 09:25:34,499 [527] [DEBUG] [app] Ending request: urn:request:fd1605ec-c11b-4e52-a032-32917573eeaa (/_internal_ping) 2020-03-31 09:25:34,500 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:25:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:34,508 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:34,508 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:34,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:25:34,512 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:34,513 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:34,517 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:25:34,519 [539] [DEBUG] [app] Starting request: urn:request:ba5d926a-0263-418e-9aba-0529b90fe57e (/c1/_internal_ping) 2020-03-31 09:25:34,520 [539] [DEBUG] [app] Ending request: urn:request:ba5d926a-0263-418e-9aba-0529b90fe57e (/c1/_internal_ping) 2020-03-31 09:25:34,520 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:34,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:25:34,512 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:34,513 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:34,517 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 127.0.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-verbs stdout | 2020-03-31 09:25:34,519 [539] [DEBUG] [app] Starting request: urn:request:ba5d926a-0263-418e-9aba-0529b90fe57e (/c1/_internal_ping) 2020-03-31 09:25:34,520 [539] [DEBUG] [app] Ending request: urn:request:ba5d926a-0263-418e-9aba-0529b90fe57e (/c1/_internal_ping) 2020-03-31 09:25:34,520 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:34,523 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:34,523 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:34,530 [526] [DEBUG] [app] Ending request: urn:request:216626d4-dade-4c2c-8ed9-da33ebb3c9e2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:34,530 [526] [DEBUG] [app] Ending request: urn:request:216626d4-dade-4c2c-8ed9-da33ebb3c9e2 (/health/instance) 2020-03-31 09:25:34,531 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:34,531 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:34,531 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.087 49 0.087) gunicorn-web stdout | 2020-03-31 09:25:34,531 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.087 49 0.087) 2020-03-31 09:25:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:25:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:38.010934+00:00 (in 1.085209 seconds) storagereplication stdout | 2020-03-31 09:25:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:36,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:38.010934+00:00 (in 1.085209 seconds) 2020-03-31 09:25:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:25:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:38 UTC)" (scheduled at 2020-03-31 09:25:38.010934+00:00) 2020-03-31 09:25:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:25:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:25:38 UTC)" (scheduled at 2020-03-31 09:25:38.010934+00:00) 2020-03-31 09:25:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:46.924497+00:00 (in 8.912765 seconds) 2020-03-31 09:25:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:25:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:46.924497+00:00 (in 8.912765 seconds) 2020-03-31 09:25:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:38 UTC)" executed successfully 2020-03-31 09:25:38,272 [527] [DEBUG] [app] Starting request: urn:request:611f8d4e-2170-40f6-bc78-eda8e48885f0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:38,272 [527] [DEBUG] [app] Starting request: urn:request:611f8d4e-2170-40f6-bc78-eda8e48885f0 (/health/instance) 2020-03-31 09:25:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:38,280 [549] [DEBUG] [app] Starting request: urn:request:c2cee21c-4e10-4c38-a602-1b561ceea023 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:38,280 [549] [DEBUG] [app] Starting request: urn:request:c2cee21c-4e10-4c38-a602-1b561ceea023 (/v1/_internal_ping) 2020-03-31 09:25:38,281 [549] [DEBUG] [app] Ending request: urn:request:c2cee21c-4e10-4c38-a602-1b561ceea023 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:38,281 [549] [DEBUG] [app] Ending request: urn:request:c2cee21c-4e10-4c38-a602-1b561ceea023 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:38,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:25:38,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:38,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:38,285 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:38,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:38,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:38,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:38,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:38,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:38,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:38,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:38,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:38,314 [527] [DEBUG] [app] Starting request: urn:request:253d2e05-0099-4b83-8df7-4834aea830c4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:38,314 [527] [DEBUG] [app] Starting request: urn:request:253d2e05-0099-4b83-8df7-4834aea830c4 (/_internal_ping) 2020-03-31 09:25:38,314 [527] [DEBUG] [app] Ending request: urn:request:253d2e05-0099-4b83-8df7-4834aea830c4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:38,314 [527] [DEBUG] [app] Ending request: urn:request:253d2e05-0099-4b83-8df7-4834aea830c4 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:25:38,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:38,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:25:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:25:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:25:38,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:38,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:38,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:38,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:38,327 [541] [DEBUG] [app] Starting request: urn:request:407943c3-d734-4304-a038-986c80ac11a0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:38,327 [541] [DEBUG] [app] Starting request: urn:request:407943c3-d734-4304-a038-986c80ac11a0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:38,328 [541] [DEBUG] [app] Ending request: urn:request:407943c3-d734-4304-a038-986c80ac11a0 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:38,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:38,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:25:38,328 [541] [DEBUG] [app] Ending request: urn:request:407943c3-d734-4304-a038-986c80ac11a0 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:38,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:38,330 [527] [DEBUG] [app] Ending request: urn:request:611f8d4e-2170-40f6-bc78-eda8e48885f0 (/health/instance) 2020-03-31 09:25:38,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:25:38,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:25:38,330 [527] [DEBUG] [app] Ending request: urn:request:611f8d4e-2170-40f6-bc78-eda8e48885f0 (/health/instance) 2020-03-31 09:25:38,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:25:41,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:25:41,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:41,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:51.440707+00:00 (in 9.996813 seconds) 2020-03-31 09:25:41,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:51 UTC)" (scheduled at 2020-03-31 09:25:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:25:41,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:51.440707+00:00 (in 9.996813 seconds) 2020-03-31 09:25:41,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:51 UTC)" (scheduled at 2020-03-31 09:25:41.440707+00:00) 2020-03-31 09:25:41,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:25:41,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:25:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:25:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:25:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:25:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:51 UTC)" executed successfully 2020-03-31 09:25:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:25:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:33.366791+00:00 (in 49.826017 seconds) 2020-03-31 09:25:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:43 UTC)" (scheduled at 2020-03-31 09:25:43.539999+00:00) 2020-03-31 09:25:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:25:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:33.366791+00:00 (in 49.826017 seconds) 2020-03-31 09:25:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:43 UTC)" (scheduled at 2020-03-31 09:25:43.539999+00:00) 2020-03-31 09:25:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:43 UTC)" executed successfully 2020-03-31 09:25:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:25:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:44 UTC)" (scheduled at 2020-03-31 09:25:44.323352+00:00) 2020-03-31 09:25:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 44, 325056), True, datetime.datetime(2020, 3, 31, 9, 25, 44, 325056), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:25:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:49.498966+00:00 (in 5.174831 seconds) notificationworker stdout | 2020-03-31 09:25:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:44 UTC)" (scheduled at 2020-03-31 09:25:44.323352+00:00) 2020-03-31 09:25:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:25:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 44, 325056), True, datetime.datetime(2020, 3, 31, 9, 25, 44, 325056), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:25:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:49.498966+00:00 (in 5.174831 seconds) 2020-03-31 09:25:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:25:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:25:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:54 UTC)" executed successfully 2020-03-31 09:25:44,446 [518] [DEBUG] [app] Starting request: urn:request:421475ce-6faf-4bce-a372-0c90ab11d9fd (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:44,446 [518] [DEBUG] [app] Starting request: urn:request:421475ce-6faf-4bce-a372-0c90ab11d9fd (/health/instance) 2020-03-31 09:25:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:44,454 [545] [DEBUG] [app] Starting request: urn:request:f742897d-116f-4bd8-9b63-74d2f4045e68 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:44,454 [545] [DEBUG] [app] Starting request: urn:request:f742897d-116f-4bd8-9b63-74d2f4045e68 (/v1/_internal_ping) 2020-03-31 09:25:44,455 [545] [DEBUG] [app] Ending request: urn:request:f742897d-116f-4bd8-9b63-74d2f4045e68 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:44,455 [545] [DEBUG] [app] Ending request: urn:request:f742897d-116f-4bd8-9b63-74d2f4045e68 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:25:44,456 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:25:44,456 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:44,461 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:44,461 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:44,463 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:44,463 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:44,464 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:44,464 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:44,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:44,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:44,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:44,492 [525] [DEBUG] [app] Starting request: urn:request:04fb1e96-27da-4b59-9adf-aef16764cc96 (/_internal_ping) 2020-03-31 09:25:44,493 [525] [DEBUG] [app] Ending request: urn:request:04fb1e96-27da-4b59-9adf-aef16764cc96 (/_internal_ping) 2020-03-31 09:25:44,493 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:25:44,492 [525] [DEBUG] [app] Starting request: urn:request:04fb1e96-27da-4b59-9adf-aef16764cc96 (/_internal_ping) 2020-03-31 09:25:44,493 [525] [DEBUG] [app] Ending request: urn:request:04fb1e96-27da-4b59-9adf-aef16764cc96 (/_internal_ping) 2020-03-31 09:25:44,493 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:25:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:25:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:25:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:44,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:44,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:44,510 [541] [DEBUG] [app] Starting request: urn:request:45d733fd-642b-4292-a83d-1569efabe205 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:44,510 [541] [DEBUG] [app] Starting request: urn:request:45d733fd-642b-4292-a83d-1569efabe205 (/c1/_internal_ping) 2020-03-31 09:25:44,511 [541] [DEBUG] [app] Ending request: urn:request:45d733fd-642b-4292-a83d-1569efabe205 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:44,511 [541] [DEBUG] [app] Ending request: urn:request:45d733fd-642b-4292-a83d-1569efabe205 (/c1/_internal_ping) 2020-03-31 09:25:44,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:25:44,512 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:44,514 [518] [DEBUG] [app] Ending request: urn:request:421475ce-6faf-4bce-a372-0c90ab11d9fd (/health/instance) 2020-03-31 09:25:44,514 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:44,514 [518] [DEBUG] [app] Ending request: urn:request:421475ce-6faf-4bce-a372-0c90ab11d9fd (/health/instance) 2020-03-31 09:25:44,514 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:44,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:25:44,512 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:44,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-verbs stdout | 2020-03-31 09:25:44,512 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:25:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:15.064514+00:00 (in 29.996792 seconds) 2020-03-31 09:25:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:15 UTC)" (scheduled at 2020-03-31 09:25:45.064514+00:00) gcworker stdout | 2020-03-31 09:25:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:15.064514+00:00 (in 29.996792 seconds) 2020-03-31 09:25:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:15 UTC)" (scheduled at 2020-03-31 09:25:45.064514+00:00) 2020-03-31 09:25:45,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437145, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:25:45,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437145, 1209600, 500, 1, 0]) 2020-03-31 09:25:45,089 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:25:45,089 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:25:45,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:45,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:25:45,089 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:45,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:15 UTC)" executed successfully 2020-03-31 09:25:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:25:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:46,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:56.924497+00:00 (in 9.998390 seconds) storagereplication stdout | 2020-03-31 09:25:46,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:56.924497+00:00 (in 9.998390 seconds) 2020-03-31 09:25:47,844 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:25:47,844 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy 2020-03-31 09:25:47,844 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:25:47,844 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:25:47,844 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:25:47,844 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:25:47,856 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) storagereplication stdout | 2020-03-31 09:25:47,856 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) 2020-03-31 09:25:47,861 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 30, 47, 856069), True, u'1485479f-4b38-48af-b179-ecc0a45d59a5', 3]) storagereplication stdout | 2020-03-31 09:25:47,861 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 30, 47, 856069), True, u'1485479f-4b38-48af-b179-ecc0a45d59a5', 3]) 2020-03-31 09:25:47,867 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 47, 866713), True, datetime.datetime(2020, 3, 31, 9, 25, 47, 866713), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:25:47,867 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 47, 866713), True, datetime.datetime(2020, 3, 31, 9, 25, 47, 866713), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:25:47,872 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 45, 47, 866713), 1, u'4f9f4f52-9a56-4240-bd8e-eac0ab38a837', 4, u'b44bb566-5001-4789-abca-e578ed46f39b']) storagereplication stdout | 2020-03-31 09:25:47,872 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 45, 47, 866713), 1, u'4f9f4f52-9a56-4240-bd8e-eac0ab38a837', 4, u'b44bb566-5001-4789-abca-e578ed46f39b']) 2020-03-31 09:25:47,878 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} storagereplication stdout | 2020-03-31 09:25:47,878 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:25:47,878 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:47,879 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 storagereplication stdout | 2020-03-31 09:25:47,878 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:47,879 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 2020-03-31 09:25:47,881 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:25:47,881 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:25:47,893 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:25:47,893 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:25:47,897 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) storagereplication stdout | 2020-03-31 09:25:47,897 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) 2020-03-31 09:25:47,900 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:25:47,900 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:25:47,901 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9afbe37c-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:47,901 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afbe37c-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9afbe37c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uT/k1k2silxlqN+M1iGi3b3AAuC9y7yFi511awFUx0M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. storagereplication stdout | 2020-03-31 09:25:47,901 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9afbe37c-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:47,901 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afbe37c-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9afbe37c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uT/k1k2silxlqN+M1iGi3b3AAuC9y7yFi511awFUx0M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. 2020-03-31 09:25:47,909 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:25:47,909 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:25:47,910 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afbe37c-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=feecbde8-c01e-004b-513e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9afbe37c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uT/k1k2silxlqN+M1iGi3b3AAuC9y7yFi511awFUx0M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. storagereplication stdout | 2020-03-31 09:25:47,910 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afbe37c-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=feecbde8-c01e-004b-513e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9afbe37c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uT/k1k2silxlqN+M1iGi3b3AAuC9y7yFi511awFUx0M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. 2020-03-31 09:25:47,911 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us storagereplication stdout | 2020-03-31 09:25:47,911 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us 2020-03-31 09:25:47,911 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:47,911 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:25:47,911 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:47,911 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:25:47,912 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:9afd898e-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=zjQznBUwF/DOttwVqrjDpLzXD6q3GUyRQ8c8aqOpGbQ%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A47Z x-ms-date:Tue, 31 Mar 2020 09:25:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:25:47,912 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afd898e-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=zjQznBUwF/DOttwVqrjDpLzXD6q3GUyRQ8c8aqOpGbQ%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:YHwpIGiYBJ5LX1NFXKbkQpc565deFv/7BG0BPIwBdkA=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '9afd898e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. storagereplication stdout | 2020-03-31 09:25:47,912 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:9afd898e-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=zjQznBUwF/DOttwVqrjDpLzXD6q3GUyRQ8c8aqOpGbQ%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A47Z x-ms-date:Tue, 31 Mar 2020 09:25:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:25:47,912 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afd898e-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=zjQznBUwF/DOttwVqrjDpLzXD6q3GUyRQ8c8aqOpGbQ%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:YHwpIGiYBJ5LX1NFXKbkQpc565deFv/7BG0BPIwBdkA=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '9afd898e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. 2020-03-31 09:25:47,927 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:25:47,927 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 2020-03-31 09:25:47,927 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afd898e-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=feecbdeb-c01e-004b-523e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=zjQznBUwF/DOttwVqrjDpLzXD6q3GUyRQ8c8aqOpGbQ%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:YHwpIGiYBJ5LX1NFXKbkQpc565deFv/7BG0BPIwBdkA=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '9afd898e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. storagereplication stdout | 2020-03-31 09:25:47,927 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9afd898e-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=feecbdeb-c01e-004b-523e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=zjQznBUwF/DOttwVqrjDpLzXD6q3GUyRQ8c8aqOpGbQ%3D&sv=2017-07-29&se=2020-03-31T09%3A26%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:YHwpIGiYBJ5LX1NFXKbkQpc565deFv/7BG0BPIwBdkA=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '9afd898e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. 2020-03-31 09:25:47,928 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:25:47,928 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:25:47,928 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9b000bc8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:47 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:47,928 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9b000bc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1yh2HX9iEmW0RISMGdpRVXp2p6ZE+ZkT7WVnEeqN/N0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. storagereplication stdout | 2020-03-31 09:25:47,928 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9b000bc8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:47 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:47,928 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9b000bc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1yh2HX9iEmW0RISMGdpRVXp2p6ZE+ZkT7WVnEeqN/N0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. 2020-03-31 09:25:48,013 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:25:48,013 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:48,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=d7237ebb-e01e-0058-753e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9b000bc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1yh2HX9iEmW0RISMGdpRVXp2p6ZE+ZkT7WVnEeqN/N0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. 2020-03-31 09:25:48,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=d7237ebb-e01e-0058-753e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:48,014 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=d7237ebb-e01e-0058-753e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:25:48,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=d7237ebb-e01e-0058-753e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9b000bc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1yh2HX9iEmW0RISMGdpRVXp2p6ZE+ZkT7WVnEeqN/N0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:47 GMT'}. 2020-03-31 09:25:48,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=d7237ebb-e01e-0058-753e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:48,014 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=9b000bc8-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:47 GMT, Server-Request-ID=d7237ebb-e01e-0058-753e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:48,014 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:25:48,014 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:25:48,273 [518] [DEBUG] [app] Starting request: urn:request:542375b3-12a8-4ca6-9697-bc5fe29b7580 (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:48,273 [518] [DEBUG] [app] Starting request: urn:request:542375b3-12a8-4ca6-9697-bc5fe29b7580 (/health/instance) 2020-03-31 09:25:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:48,278 [538] [DEBUG] [app] Starting request: urn:request:287edeb2-a53c-4ef8-ad30-51f6fba45151 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:48,278 [538] [DEBUG] [app] Starting request: urn:request:287edeb2-a53c-4ef8-ad30-51f6fba45151 (/v1/_internal_ping) 2020-03-31 09:25:48,279 [538] [DEBUG] [app] Ending request: urn:request:287edeb2-a53c-4ef8-ad30-51f6fba45151 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:48,279 [538] [DEBUG] [app] Ending request: urn:request:287edeb2-a53c-4ef8-ad30-51f6fba45151 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:48,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:48,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:25:48,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:48,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:48,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:48,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:25:48,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:48,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:48,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:48,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:48,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:48,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:48,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:48,343 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:48,343 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:48,347 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:48,347 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:48,348 [518] [DEBUG] [app] Starting request: urn:request:6401eb2c-a0fb-4d1a-a796-3546e89175c1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:48,348 [518] [DEBUG] [app] Starting request: urn:request:6401eb2c-a0fb-4d1a-a796-3546e89175c1 (/_internal_ping) 2020-03-31 09:25:48,349 [518] [DEBUG] [app] Ending request: urn:request:6401eb2c-a0fb-4d1a-a796-3546e89175c1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:48,349 [518] [DEBUG] [app] Ending request: urn:request:6401eb2c-a0fb-4d1a-a796-3546e89175c1 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:25:48,350 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:48,350 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:48,350 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:48,350 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:48,354 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:48,354 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:48,354 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:25:48,354 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:25:48,357 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:48,357 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:48,358 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:25:48,358 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:25:48,361 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:48,361 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:48,363 [539] [DEBUG] [app] Starting request: urn:request:eeb91694-ea0d-4d3a-b630-8ebcbaf31c0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:48,363 [539] [DEBUG] [app] Starting request: urn:request:eeb91694-ea0d-4d3a-b630-8ebcbaf31c0b (/c1/_internal_ping) 2020-03-31 09:25:48,364 [539] [DEBUG] [app] Ending request: urn:request:eeb91694-ea0d-4d3a-b630-8ebcbaf31c0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:48,364 [539] [DEBUG] [app] Ending request: urn:request:eeb91694-ea0d-4d3a-b630-8ebcbaf31c0b (/c1/_internal_ping) 2020-03-31 09:25:48,365 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:48,365 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:48,365 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:25:48,365 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:48,367 [518] [DEBUG] [app] Ending request: urn:request:542375b3-12a8-4ca6-9697-bc5fe29b7580 (/health/instance) 2020-03-31 09:25:48,367 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:48,367 [518] [DEBUG] [app] Ending request: urn:request:542375b3-12a8-4ca6-9697-bc5fe29b7580 (/health/instance) 2020-03-31 09:25:48,367 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:48,367 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.096 49 0.096) gunicorn-web stdout | 2020-03-31 09:25:48,367 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.096 49 0.096) 2020-03-31 09:25:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:25:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:54.323352+00:00 (in 4.820530 seconds) 2020-03-31 09:25:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:49 UTC)" (scheduled at 2020-03-31 09:25:49.498966+00:00) 2020-03-31 09:25:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:49,503 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:25:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:54.323352+00:00 (in 4.820530 seconds) 2020-03-31 09:25:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:49 UTC)" (scheduled at 2020-03-31 09:25:49.498966+00:00) 2020-03-31 09:25:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:49,503 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:49 UTC)" executed successfully 2020-03-31 09:25:50,016 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9c3ea940-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:50 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:25:50,016 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9c3ea940-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:50 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:50,016 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9c3ea940-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cfngudDOKQJVDSEgekgrEunhZAD67PBfWWpbqRfvRJI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:50 GMT'}. storagereplication stdout | 2020-03-31 09:25:50,016 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9c3ea940-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cfngudDOKQJVDSEgekgrEunhZAD67PBfWWpbqRfvRJI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:50 GMT'}. 2020-03-31 09:25:50,096 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:25:50,096 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:50,097 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:49 GMT, Server-Request-ID=d7238345-e01e-0058-123e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9c3ea940-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cfngudDOKQJVDSEgekgrEunhZAD67PBfWWpbqRfvRJI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:50 GMT'}. 2020-03-31 09:25:50,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:49 GMT, Server-Request-ID=d7238345-e01e-0058-123e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:25:50,097 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:49 GMT, Server-Request-ID=d7238345-e01e-0058-123e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9c3ea940-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cfngudDOKQJVDSEgekgrEunhZAD67PBfWWpbqRfvRJI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:50 GMT'}. 2020-03-31 09:25:50,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:49 GMT, Server-Request-ID=d7238345-e01e-0058-123e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:50,098 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:49 GMT, Server-Request-ID=d7238345-e01e-0058-123e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:50,098 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:25:50,098 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=9c3ea940-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:49 GMT, Server-Request-ID=d7238345-e01e-0058-123e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:50,098 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:25:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:25:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:59.560326+00:00 (in 8.448721 seconds) 2020-03-31 09:25:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:51 UTC)" (scheduled at 2020-03-31 09:25:51.110610+00:00) 2020-03-31 09:25:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:25:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:25:59.560326+00:00 (in 8.448721 seconds) 2020-03-31 09:25:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:51 UTC)" (scheduled at 2020-03-31 09:25:51.110610+00:00) 2020-03-31 09:25:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 51, 112524), True, datetime.datetime(2020, 3, 31, 9, 25, 51, 112524), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:25:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 51, 112524), True, datetime.datetime(2020, 3, 31, 9, 25, 51, 112524), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:25:51,131 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:25:51,131 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:25:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:51 UTC)" executed successfully 2020-03-31 09:25:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:25:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:01.440707+00:00 (in 9.999152 seconds) 2020-03-31 09:25:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:01 UTC)" (scheduled at 2020-03-31 09:25:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:25:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:01.440707+00:00 (in 9.999152 seconds) 2020-03-31 09:25:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:01 UTC)" (scheduled at 2020-03-31 09:25:51.440707+00:00) 2020-03-31 09:25:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:25:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:25:51,456 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:25:51,456 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:25:51,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:51,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:25:51,457 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:51,457 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:01 UTC)" executed successfully 2020-03-31 09:25:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:25:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:22.113221+00:00 (in 29.997612 seconds) 2020-03-31 09:25:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:22 UTC)" (scheduled at 2020-03-31 09:25:52.113221+00:00) securityworker stdout | 2020-03-31 09:25:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:22.113221+00:00 (in 29.997612 seconds) 2020-03-31 09:25:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:22 UTC)" (scheduled at 2020-03-31 09:25:52.113221+00:00) 2020-03-31 09:25:52,116 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:25:52,116 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:25:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:25:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:25:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:25:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:22 UTC)" executed successfully 2020-03-31 09:25:54,102 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9eae073e-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:54 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:25:54,102 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:9eae073e-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:25:54 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:25:54,102 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9eae073e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:14xv3Tfjw8jinsrlMvJTcTtIfscvW0xChRr4bHW6SCU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:54 GMT'}. storagereplication stdout | 2020-03-31 09:25:54,102 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9eae073e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:14xv3Tfjw8jinsrlMvJTcTtIfscvW0xChRr4bHW6SCU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:54 GMT'}. 2020-03-31 09:25:54,182 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:25:54,182 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:25:54,183 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:54 GMT, Server-Request-ID=d7238bb5-e01e-0058-6b3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9eae073e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:14xv3Tfjw8jinsrlMvJTcTtIfscvW0xChRr4bHW6SCU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:54 GMT'}. 2020-03-31 09:25:54,183 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:54 GMT, Server-Request-ID=d7238bb5-e01e-0058-6b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:25:54,183 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:25:54 GMT, Server-Request-ID=d7238bb5-e01e-0058-6b3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '9eae073e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:14xv3Tfjw8jinsrlMvJTcTtIfscvW0xChRr4bHW6SCU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:25:54 GMT'}. 2020-03-31 09:25:54,183 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:25:54 GMT, Server-Request-ID=d7238bb5-e01e-0058-6b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:54,183 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:54 GMT, Server-Request-ID=d7238bb5-e01e-0058-6b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:54,184 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:25:54,183 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=9eae073e-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:25:54 GMT, Server-Request-ID=d7238bb5-e01e-0058-6b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:25:54,184 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:25:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:25:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:04.323352+00:00 (in 9.998523 seconds) 2020-03-31 09:25:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:04 UTC)" (scheduled at 2020-03-31 09:25:54.323352+00:00) 2020-03-31 09:25:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:25:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:04.323352+00:00 (in 9.998523 seconds) 2020-03-31 09:25:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:04 UTC)" (scheduled at 2020-03-31 09:25:54.323352+00:00) 2020-03-31 09:25:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:25:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 54, 326217), True, datetime.datetime(2020, 3, 31, 9, 25, 54, 326217), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:25:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 25, 54, 326217), True, datetime.datetime(2020, 3, 31, 9, 25, 54, 326217), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:25:54,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:25:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:25:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:25:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:04 UTC)" executed successfully 2020-03-31 09:25:54,446 [527] [DEBUG] [app] Starting request: urn:request:d75056f3-64f6-4bfa-89dc-10b725b9b7ad (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:54,446 [527] [DEBUG] [app] Starting request: urn:request:d75056f3-64f6-4bfa-89dc-10b725b9b7ad (/health/instance) 2020-03-31 09:25:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:54,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:54,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:54,454 [556] [DEBUG] [app] Starting request: urn:request:44ac9186-7bdc-4ddc-9b0d-1bcdc2368f42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:54,454 [556] [DEBUG] [app] Starting request: urn:request:44ac9186-7bdc-4ddc-9b0d-1bcdc2368f42 (/v1/_internal_ping) 2020-03-31 09:25:54,455 [556] [DEBUG] [app] Ending request: urn:request:44ac9186-7bdc-4ddc-9b0d-1bcdc2368f42 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:54,455 [556] [DEBUG] [app] Ending request: urn:request:44ac9186-7bdc-4ddc-9b0d-1bcdc2368f42 (/v1/_internal_ping) 2020-03-31 09:25:54,464 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.011 162 0.011) gunicorn-registry stdout | 2020-03-31 09:25:54,464 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.011 162 0.011) 2020-03-31 09:25:54,465 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:25:54,465 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:54,467 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:54,467 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:25:54,467 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:25:54,467 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:25:54,468 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:54,468 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:54,469 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:54,469 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:54,488 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:54,488 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:54,501 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:54,501 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:54,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:54,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:54,506 [527] [DEBUG] [app] Starting request: urn:request:b83e8749-f533-426e-a655-40988c52ce5e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:54,506 [527] [DEBUG] [app] Starting request: urn:request:b83e8749-f533-426e-a655-40988c52ce5e (/_internal_ping) 2020-03-31 09:25:54,507 [527] [DEBUG] [app] Ending request: urn:request:b83e8749-f533-426e-a655-40988c52ce5e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:54,507 [527] [DEBUG] [app] Ending request: urn:request:b83e8749-f533-426e-a655-40988c52ce5e (/_internal_ping) 2020-03-31 09:25:54,508 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:25:54,508 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:54,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:25:54,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:54,511 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:54,511 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:54,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:54,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:54,515 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:54,515 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:25:54,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:54,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:54,519 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:54,519 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:54,520 [541] [DEBUG] [app] Starting request: urn:request:8e3b4c3b-4dff-48d5-93e7-2acfc0fd7b9c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:54,520 [541] [DEBUG] [app] Starting request: urn:request:8e3b4c3b-4dff-48d5-93e7-2acfc0fd7b9c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:54,521 [541] [DEBUG] [app] Ending request: urn:request:8e3b4c3b-4dff-48d5-93e7-2acfc0fd7b9c (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:54,521 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:54,521 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:25:54,521 [541] [DEBUG] [app] Ending request: urn:request:8e3b4c3b-4dff-48d5-93e7-2acfc0fd7b9c (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:54,521 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:54,522 [527] [DEBUG] [app] Ending request: urn:request:d75056f3-64f6-4bfa-89dc-10b725b9b7ad (/health/instance) 2020-03-31 09:25:54,523 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:25:54,521 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:54,523 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.078 49 0.078) gunicorn-web stdout | 2020-03-31 09:25:54,522 [527] [DEBUG] [app] Ending request: urn:request:d75056f3-64f6-4bfa-89dc-10b725b9b7ad (/health/instance) 2020-03-31 09:25:54,523 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:54,523 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.078 49 0.078) 2020-03-31 09:25:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:25:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:06.924497+00:00 (in 9.999260 seconds) storagereplication stdout | 2020-03-31 09:25:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:25:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:25:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:06.924497+00:00 (in 9.999260 seconds) 2020-03-31 09:25:58,273 [527] [DEBUG] [app] Starting request: urn:request:8da69775-4866-42ab-8eaf-ca36e09c5f10 (/health/instance) gunicorn-web stdout | 2020-03-31 09:25:58,273 [527] [DEBUG] [app] Starting request: urn:request:8da69775-4866-42ab-8eaf-ca36e09c5f10 (/health/instance) 2020-03-31 09:25:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:58,278 [556] [DEBUG] [app] Starting request: urn:request:f02ea5e2-a663-4980-a0e7-44f09f5e37f8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:58,278 [556] [DEBUG] [app] Starting request: urn:request:f02ea5e2-a663-4980-a0e7-44f09f5e37f8 (/v1/_internal_ping) 2020-03-31 09:25:58,279 [556] [DEBUG] [app] Ending request: urn:request:f02ea5e2-a663-4980-a0e7-44f09f5e37f8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:25:58,279 [556] [DEBUG] [app] Ending request: urn:request:f02ea5e2-a663-4980-a0e7-44f09f5e37f8 (/v1/_internal_ping) 2020-03-31 09:25:58,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:25:58,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:25:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:25:58,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:25:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:25:58,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:25:58,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:25:58,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:25:58,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:25:58,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:25:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:25:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:25:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:25:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:25:58,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:58,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:58,312 [527] [DEBUG] [app] Starting request: urn:request:47b3d2c6-dacd-41ce-800e-54407a281ec4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:58,312 [527] [DEBUG] [app] Starting request: urn:request:47b3d2c6-dacd-41ce-800e-54407a281ec4 (/_internal_ping) 2020-03-31 09:25:58,313 [527] [DEBUG] [app] Ending request: urn:request:47b3d2c6-dacd-41ce-800e-54407a281ec4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:25:58,313 [527] [DEBUG] [app] Ending request: urn:request:47b3d2c6-dacd-41ce-800e-54407a281ec4 (/_internal_ping) 2020-03-31 09:25:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:25:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:25:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:25:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:25:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:25:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:25:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:25:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:25:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:25:58,327 [541] [DEBUG] [app] Starting request: urn:request:6b442afd-f61c-4e31-80e7-10890cf307f0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:58,327 [541] [DEBUG] [app] Starting request: urn:request:6b442afd-f61c-4e31-80e7-10890cf307f0 (/c1/_internal_ping) 2020-03-31 09:25:58,328 [541] [DEBUG] [app] Ending request: urn:request:6b442afd-f61c-4e31-80e7-10890cf307f0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:25:58,328 [541] [DEBUG] [app] Ending request: urn:request:6b442afd-f61c-4e31-80e7-10890cf307f0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:25:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:25:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:25:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:25:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:25:58,330 [527] [DEBUG] [app] Ending request: urn:request:8da69775-4866-42ab-8eaf-ca36e09c5f10 (/health/instance) 2020-03-31 09:25:58,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:25:58,330 [527] [DEBUG] [app] Ending request: urn:request:8da69775-4866-42ab-8eaf-ca36e09c5f10 (/health/instance) 2020-03-31 09:25:58,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:25:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:25:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:25:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:25:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:25:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:25:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:25:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:51.110610+00:00 (in 51.548981 seconds) 2020-03-31 09:25:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:59 UTC)" (scheduled at 2020-03-31 09:25:59.560326+00:00) 2020-03-31 09:25:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:25:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:51.110610+00:00 (in 51.548981 seconds) 2020-03-31 09:25:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:59 UTC)" (scheduled at 2020-03-31 09:25:59.560326+00:00) 2020-03-31 09:25:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:25:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:26:59 UTC)" executed successfully 2020-03-31 09:26:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:26:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:30.337699+00:00 (in 29.999221 seconds) 2020-03-31 09:26:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:30 UTC)" (scheduled at 2020-03-31 09:26:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:26:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:30.337699+00:00 (in 29.999221 seconds) 2020-03-31 09:26:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:30 UTC)" (scheduled at 2020-03-31 09:26:00.337699+00:00) 2020-03-31 09:26:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 26, 0, 339368), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:26:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 26, 0, 339368), False, 50, 1, 0]) 2020-03-31 09:26:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:26:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:26:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:26:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:30 UTC)" executed successfully 2020-03-31 09:26:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:26:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:11 UTC)" (scheduled at 2020-03-31 09:26:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:26:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:11 UTC)" (scheduled at 2020-03-31 09:26:01.440707+00:00) 2020-03-31 09:26:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:11.440707+00:00 (in 9.998855 seconds) repositoryactioncounter stdout | 2020-03-31 09:26:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:11.440707+00:00 (in 9.998855 seconds) 2020-03-31 09:26:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:26:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:26:01,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:26:01,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:26:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:01,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:26:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:01,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:11 UTC)" executed successfully 2020-03-31 09:26:02,192 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:a3808cbe-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:02 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:26:02,192 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'a3808cbe-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bQhiVSCFtKUzjHVFLmK7nw8Bd6ldsSYu2fvE1VG8D0U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:02 GMT'}. storagereplication stdout | 2020-03-31 09:26:02,192 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:a3808cbe-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:02 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:26:02,192 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'a3808cbe-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bQhiVSCFtKUzjHVFLmK7nw8Bd6ldsSYu2fvE1VG8D0U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:02 GMT'}. 2020-03-31 09:26:02,294 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:02,294 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:02,295 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:02 GMT, Server-Request-ID=d7239ba0-e01e-0058-4d3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'a3808cbe-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bQhiVSCFtKUzjHVFLmK7nw8Bd6ldsSYu2fvE1VG8D0U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:02 GMT'}. 2020-03-31 09:26:02,295 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:02 GMT, Server-Request-ID=d7239ba0-e01e-0058-4d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:02,295 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:02 GMT, Server-Request-ID=d7239ba0-e01e-0058-4d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:26:02,295 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:02 GMT, Server-Request-ID=d7239ba0-e01e-0058-4d3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'a3808cbe-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bQhiVSCFtKUzjHVFLmK7nw8Bd6ldsSYu2fvE1VG8D0U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:02 GMT'}. 2020-03-31 09:26:02,295 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:02 GMT, Server-Request-ID=d7239ba0-e01e-0058-4d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:02,295 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=a3808cbe-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:02 GMT, Server-Request-ID=d7239ba0-e01e-0058-4d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:02,296 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:26:02,296 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:26:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:26:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:24.410927+00:00 (in 21.079341 seconds) 2020-03-31 09:26:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:33 UTC)" (scheduled at 2020-03-31 09:26:03.330112+00:00) 2020-03-31 09:26:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:26:03,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:24.410927+00:00 (in 21.079341 seconds) 2020-03-31 09:26:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:33 UTC)" (scheduled at 2020-03-31 09:26:03.330112+00:00) 2020-03-31 09:26:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 3, 333272), True, datetime.datetime(2020, 3, 31, 9, 26, 3, 333272), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:26:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 3, 333272), True, datetime.datetime(2020, 3, 31, 9, 26, 3, 333272), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:26:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:26:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:33 UTC)" executed successfully 2020-03-31 09:26:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:26:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:14.323352+00:00 (in 9.998421 seconds) 2020-03-31 09:26:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:14 UTC)" (scheduled at 2020-03-31 09:26:04.323352+00:00) 2020-03-31 09:26:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:26:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:14.323352+00:00 (in 9.998421 seconds) 2020-03-31 09:26:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:14 UTC)" (scheduled at 2020-03-31 09:26:04.323352+00:00) 2020-03-31 09:26:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 4, 325693), True, datetime.datetime(2020, 3, 31, 9, 26, 4, 325693), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:26:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 4, 325693), True, datetime.datetime(2020, 3, 31, 9, 26, 4, 325693), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:26:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:26:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:26:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:14 UTC)" executed successfully 2020-03-31 09:26:04,446 [518] [DEBUG] [app] Starting request: urn:request:c59ef2cf-14d1-4e82-be38-8f1b45c9a995 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:04,446 [518] [DEBUG] [app] Starting request: urn:request:c59ef2cf-14d1-4e82-be38-8f1b45c9a995 (/health/instance) 2020-03-31 09:26:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:04,454 [552] [DEBUG] [app] Starting request: urn:request:6609c2cd-3468-4b0c-afef-1f4091a8aaf4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:04,454 [552] [DEBUG] [app] Starting request: urn:request:6609c2cd-3468-4b0c-afef-1f4091a8aaf4 (/v1/_internal_ping) 2020-03-31 09:26:04,455 [552] [DEBUG] [app] Ending request: urn:request:6609c2cd-3468-4b0c-afef-1f4091a8aaf4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:04,455 [552] [DEBUG] [app] Ending request: urn:request:6609c2cd-3468-4b0c-afef-1f4091a8aaf4 (/v1/_internal_ping) 2020-03-31 09:26:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:26:04,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:04,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:04,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:04,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:04,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:04,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:04,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:04,486 [518] [DEBUG] [app] Starting request: urn:request:cc0c81e9-0412-4a0f-8f9a-fb1a06a83a93 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:04,486 [518] [DEBUG] [app] Starting request: urn:request:cc0c81e9-0412-4a0f-8f9a-fb1a06a83a93 (/_internal_ping) 2020-03-31 09:26:04,487 [518] [DEBUG] [app] Ending request: urn:request:cc0c81e9-0412-4a0f-8f9a-fb1a06a83a93 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:04,487 [518] [DEBUG] [app] Ending request: urn:request:cc0c81e9-0412-4a0f-8f9a-fb1a06a83a93 (/_internal_ping) 2020-03-31 09:26:04,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:04,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:04,501 [537] [DEBUG] [app] Starting request: urn:request:1e5ad1d0-5a02-473b-abe5-6209edeb3e67 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:04,501 [537] [DEBUG] [app] Starting request: urn:request:1e5ad1d0-5a02-473b-abe5-6209edeb3e67 (/c1/_internal_ping) 2020-03-31 09:26:04,502 [537] [DEBUG] [app] Ending request: urn:request:1e5ad1d0-5a02-473b-abe5-6209edeb3e67 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:04,502 [537] [DEBUG] [app] Ending request: urn:request:1e5ad1d0-5a02-473b-abe5-6209edeb3e67 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:04,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:26:04,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:04,504 [518] [DEBUG] [app] Ending request: urn:request:c59ef2cf-14d1-4e82-be38-8f1b45c9a995 (/health/instance) 2020-03-31 09:26:04,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:26:04,504 [518] [DEBUG] [app] Ending request: urn:request:c59ef2cf-14d1-4e82-be38-8f1b45c9a995 (/health/instance) 2020-03-31 09:26:04,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:26:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:26:06,935 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:06,935 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:06,935 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:16.924497+00:00 (in 9.989042 seconds) storagereplication stdout | 2020-03-31 09:26:06,935 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:06,935 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:06,935 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:16.924497+00:00 (in 9.989042 seconds) 2020-03-31 09:26:08,272 [527] [DEBUG] [app] Starting request: urn:request:9c78444b-b409-42ad-b1cf-b7aa302041b2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:08,272 [527] [DEBUG] [app] Starting request: urn:request:9c78444b-b409-42ad-b1cf-b7aa302041b2 (/health/instance) 2020-03-31 09:26:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:08,281 [544] [DEBUG] [app] Starting request: urn:request:4d51b840-c679-442f-953a-b581cebb134d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:08,281 [544] [DEBUG] [app] Starting request: urn:request:4d51b840-c679-442f-953a-b581cebb134d (/v1/_internal_ping) 2020-03-31 09:26:08,282 [544] [DEBUG] [app] Ending request: urn:request:4d51b840-c679-442f-953a-b581cebb134d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:08,282 [544] [DEBUG] [app] Ending request: urn:request:4d51b840-c679-442f-953a-b581cebb134d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:08,282 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:08,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:08,282 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:08,287 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:26:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:26:08,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:08,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:08,287 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:08,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:08,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:08,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:08,313 [527] [DEBUG] [app] Starting request: urn:request:25a5c05c-bcfa-49bb-b21e-0e94d1112a45 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:08,313 [527] [DEBUG] [app] Starting request: urn:request:25a5c05c-bcfa-49bb-b21e-0e94d1112a45 (/_internal_ping) 2020-03-31 09:26:08,314 [527] [DEBUG] [app] Ending request: urn:request:25a5c05c-bcfa-49bb-b21e-0e94d1112a45 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:08,314 [527] [DEBUG] [app] Ending request: urn:request:25a5c05c-bcfa-49bb-b21e-0e94d1112a45 (/_internal_ping) 2020-03-31 09:26:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:08,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:08,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:26:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:08,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:08,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:08,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:08,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:08,327 [541] [DEBUG] [app] Starting request: urn:request:b9c1e265-c567-4e69-a1b8-6a41dd6b4b9a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:08,327 [541] [DEBUG] [app] Starting request: urn:request:b9c1e265-c567-4e69-a1b8-6a41dd6b4b9a (/c1/_internal_ping) 2020-03-31 09:26:08,328 [541] [DEBUG] [app] Ending request: urn:request:b9c1e265-c567-4e69-a1b8-6a41dd6b4b9a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:08,328 [541] [DEBUG] [app] Ending request: urn:request:b9c1e265-c567-4e69-a1b8-6a41dd6b4b9a (/c1/_internal_ping) 2020-03-31 09:26:08,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:08,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:08,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:08,330 [527] [DEBUG] [app] Ending request: urn:request:9c78444b-b409-42ad-b1cf-b7aa302041b2 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:26:08,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:08,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:08,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:26:08,330 [527] [DEBUG] [app] Ending request: urn:request:9c78444b-b409-42ad-b1cf-b7aa302041b2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:08,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:08,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:26:11,458 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:11,458 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:21.440707+00:00 (in 9.982160 seconds) 2020-03-31 09:26:11,480 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:21 UTC)" (scheduled at 2020-03-31 09:26:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:26:11,458 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:11,458 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:21.440707+00:00 (in 9.982160 seconds) 2020-03-31 09:26:11,480 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:21 UTC)" (scheduled at 2020-03-31 09:26:11.440707+00:00) 2020-03-31 09:26:11,483 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:26:11,483 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:26:11,571 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:26:11,571 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:11,572 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:26:11,571 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:26:11,571 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:26:11,572 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:21 UTC)" executed successfully 2020-03-31 09:26:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:26:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:24.323352+00:00 (in 9.999089 seconds) 2020-03-31 09:26:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:24 UTC)" (scheduled at 2020-03-31 09:26:14.323352+00:00) 2020-03-31 09:26:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:26:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:24.323352+00:00 (in 9.999089 seconds) 2020-03-31 09:26:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:24 UTC)" (scheduled at 2020-03-31 09:26:14.323352+00:00) 2020-03-31 09:26:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 14, 325237), True, datetime.datetime(2020, 3, 31, 9, 26, 14, 325237), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:26:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 14, 325237), True, datetime.datetime(2020, 3, 31, 9, 26, 14, 325237), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:26:14,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:26:14,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:14,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:14,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:26:14,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:14,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:24 UTC)" executed successfully 2020-03-31 09:26:14,446 [518] [DEBUG] [app] Starting request: urn:request:b5f74bf3-0278-49d8-9fed-ce797a8f800a (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:14,446 [518] [DEBUG] [app] Starting request: urn:request:b5f74bf3-0278-49d8-9fed-ce797a8f800a (/health/instance) 2020-03-31 09:26:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:14,453 [556] [DEBUG] [app] Starting request: urn:request:21532cdb-c94a-456c-9e96-2815581cfbe9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:14,453 [556] [DEBUG] [app] Starting request: urn:request:21532cdb-c94a-456c-9e96-2815581cfbe9 (/v1/_internal_ping) 2020-03-31 09:26:14,454 [556] [DEBUG] [app] Ending request: urn:request:21532cdb-c94a-456c-9e96-2815581cfbe9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:14,454 [556] [DEBUG] [app] Ending request: urn:request:21532cdb-c94a-456c-9e96-2815581cfbe9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:14,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:26:14,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:14,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:26:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:26:14,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:14,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:14,458 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:14,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:14,459 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:14,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:14,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:14,489 [527] [DEBUG] [app] Starting request: urn:request:9279c25c-0ee0-496c-8770-dddeaa57d248 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:14,489 [527] [DEBUG] [app] Starting request: urn:request:9279c25c-0ee0-496c-8770-dddeaa57d248 (/_internal_ping) 2020-03-31 09:26:14,490 [527] [DEBUG] [app] Ending request: urn:request:9279c25c-0ee0-496c-8770-dddeaa57d248 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:14,490 [527] [DEBUG] [app] Ending request: urn:request:9279c25c-0ee0-496c-8770-dddeaa57d248 (/_internal_ping) 2020-03-31 09:26:14,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:14,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:26:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:26:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:26:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:14,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:14,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:14,506 [541] [DEBUG] [app] Starting request: urn:request:dac56472-5581-4797-86a3-46e51cad513f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:14,506 [541] [DEBUG] [app] Starting request: urn:request:dac56472-5581-4797-86a3-46e51cad513f (/c1/_internal_ping) 2020-03-31 09:26:14,507 [541] [DEBUG] [app] Ending request: urn:request:dac56472-5581-4797-86a3-46e51cad513f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:14,507 [541] [DEBUG] [app] Ending request: urn:request:dac56472-5581-4797-86a3-46e51cad513f (/c1/_internal_ping) 2020-03-31 09:26:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:14,507 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:26:14,507 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:14,509 [518] [DEBUG] [app] Ending request: urn:request:b5f74bf3-0278-49d8-9fed-ce797a8f800a (/health/instance) 2020-03-31 09:26:14,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:26:14,509 [518] [DEBUG] [app] Ending request: urn:request:b5f74bf3-0278-49d8-9fed-ce797a8f800a (/health/instance) 2020-03-31 09:26:14,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:14,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:26:14,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:26:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:26:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:45.064514+00:00 (in 29.999141 seconds) 2020-03-31 09:26:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:45 UTC)" (scheduled at 2020-03-31 09:26:15.064514+00:00) gcworker stdout | 2020-03-31 09:26:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:45.064514+00:00 (in 29.999141 seconds) 2020-03-31 09:26:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:45 UTC)" (scheduled at 2020-03-31 09:26:15.064514+00:00) 2020-03-31 09:26:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437175, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:26:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437175, 1209600, 500, 1, 0]) 2020-03-31 09:26:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:26:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:26:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:26:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:45 UTC)" executed successfully 2020-03-31 09:26:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:26.924497+00:00 (in 9.999233 seconds) storagereplication stdout | 2020-03-31 09:26:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:26.924497+00:00 (in 9.999233 seconds) 2020-03-31 09:26:18,273 [527] [DEBUG] [app] Starting request: urn:request:8ba675a5-ce99-4ec3-a08f-f3d56528243c (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:18,273 [527] [DEBUG] [app] Starting request: urn:request:8ba675a5-ce99-4ec3-a08f-f3d56528243c (/health/instance) 2020-03-31 09:26:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:18,281 [556] [DEBUG] [app] Starting request: urn:request:6b175b5a-6068-467a-9cf7-d4357eda663f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:18,281 [556] [DEBUG] [app] Starting request: urn:request:6b175b5a-6068-467a-9cf7-d4357eda663f (/v1/_internal_ping) 2020-03-31 09:26:18,283 [556] [DEBUG] [app] Ending request: urn:request:6b175b5a-6068-467a-9cf7-d4357eda663f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:18,283 [556] [DEBUG] [app] Ending request: urn:request:6b175b5a-6068-467a-9cf7-d4357eda663f (/v1/_internal_ping) 2020-03-31 09:26:18,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:26:18,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:26:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:18,287 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:18,287 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:18,288 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:18,288 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:18,289 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:18,289 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:18,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:18,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:18,306 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:26:18,306 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy 2020-03-31 09:26:18,307 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:26:18,307 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:26:18,307 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:26:18,307 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:26:18,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:18,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:18,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:18,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:18,314 [527] [DEBUG] [app] Starting request: urn:request:9b5c8bb3-f032-481c-92a9-9fb1b41a825e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:18,314 [527] [DEBUG] [app] Starting request: urn:request:9b5c8bb3-f032-481c-92a9-9fb1b41a825e (/_internal_ping) 2020-03-31 09:26:18,315 [527] [DEBUG] [app] Ending request: urn:request:9b5c8bb3-f032-481c-92a9-9fb1b41a825e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:18,315 [527] [DEBUG] [app] Ending request: urn:request:9b5c8bb3-f032-481c-92a9-9fb1b41a825e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:26:18,316 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) 2020-03-31 09:26:18,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:18,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 storagereplication stdout | 2020-03-31 09:26:18,316 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) gunicorn-web stdout | 2020-03-31 09:26:18,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:18,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:18,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:18,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:18,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:18,320 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 31, 18, 314991), True, u'3ebbd6bd-e213-4a1a-b73f-af66d33ccc61', 4]) gunicorn-web stdout | 2020-03-31 09:26:18,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) storagereplication stdout | 2020-03-31 09:26:18,320 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 31, 18, 314991), True, u'3ebbd6bd-e213-4a1a-b73f-af66d33ccc61', 4]) 2020-03-31 09:26:18,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:18,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:26:18,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:18,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:18,327 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 18, 325913), True, datetime.datetime(2020, 3, 31, 9, 26, 18, 325913), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:26:18,327 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 18, 325913), True, datetime.datetime(2020, 3, 31, 9, 26, 18, 325913), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:26:18,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:18,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:18,330 [541] [DEBUG] [app] Starting request: urn:request:24b40713-1a1a-488f-8e65-91ffff7ef0da (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:18,330 [541] [DEBUG] [app] Starting request: urn:request:24b40713-1a1a-488f-8e65-91ffff7ef0da (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:18,331 [541] [DEBUG] [app] Ending request: urn:request:24b40713-1a1a-488f-8e65-91ffff7ef0da (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:18,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:18,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:26:18,331 [541] [DEBUG] [app] Ending request: urn:request:24b40713-1a1a-488f-8e65-91ffff7ef0da (/c1/_internal_ping) 2020-03-31 09:26:18,332 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 46, 18, 325913), 1, u'f3ae9d98-aeca-4010-bc8e-c3470ea40449', 2, u'91392f91-1861-468f-9dae-ed27557543bc']) gunicorn-web stdout | 2020-03-31 09:26:18,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:18,334 [527] [DEBUG] [app] Ending request: urn:request:8ba675a5-ce99-4ec3-a08f-f3d56528243c (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:26:18,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:18,334 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:18,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) storagereplication stdout | 2020-03-31 09:26:18,332 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 46, 18, 325913), 1, u'f3ae9d98-aeca-4010-bc8e-c3470ea40449', 2, u'91392f91-1861-468f-9dae-ed27557543bc']) gunicorn-web stdout | 2020-03-31 09:26:18,334 [527] [DEBUG] [app] Ending request: urn:request:8ba675a5-ce99-4ec3-a08f-f3d56528243c (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:18,334 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:18,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:26:18,339 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} storagereplication stdout | 2020-03-31 09:26:18,339 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:26:18,340 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:18,340 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 storagereplication stdout | 2020-03-31 09:26:18,340 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:18,340 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 2020-03-31 09:26:18,341 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:26:18,341 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:26:18,359 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:26:18,359 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:26:18,363 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) storagereplication stdout | 2020-03-31 09:26:18,363 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) 2020-03-31 09:26:18,366 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:26:18,366 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:26:18,367 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ad249dc8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:18,367 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad249dc8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad249dc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ila6RqKu3SO8oKOSWVSevqJL/aQP8rK9UF6sNvFS6RM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. storagereplication stdout | 2020-03-31 09:26:18,367 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ad249dc8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:18,367 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad249dc8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad249dc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ila6RqKu3SO8oKOSWVSevqJL/aQP8rK9UF6sNvFS6RM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. 2020-03-31 09:26:18,375 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:26:18,375 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:26:18,376 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad249dc8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=feecc5bc-c01e-004b-6e3e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad249dc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ila6RqKu3SO8oKOSWVSevqJL/aQP8rK9UF6sNvFS6RM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. storagereplication stdout | 2020-03-31 09:26:18,376 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad249dc8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=feecc5bc-c01e-004b-6e3e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad249dc8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ila6RqKu3SO8oKOSWVSevqJL/aQP8rK9UF6sNvFS6RM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. 2020-03-31 09:26:18,377 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us 2020-03-31 09:26:18,377 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:26:18,377 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us 2020-03-31 09:26:18,377 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:18,377 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:26:18,378 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:ad264c5e-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=CtZDPt7c6MaLnZTMsAIJNprHKVdXSqA2Shi5zS7aMCw%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A18Z x-ms-date:Tue, 31 Mar 2020 09:26:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:26:18,378 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad264c5e-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=CtZDPt7c6MaLnZTMsAIJNprHKVdXSqA2Shi5zS7aMCw%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hO/5vxPn9pIwrJeF4knqFvxqg/zcFPI9KrL7WaU03Gc=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'ad264c5e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. storagereplication stdout | 2020-03-31 09:26:18,377 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:26:18,378 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:ad264c5e-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=CtZDPt7c6MaLnZTMsAIJNprHKVdXSqA2Shi5zS7aMCw%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A18Z x-ms-date:Tue, 31 Mar 2020 09:26:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:26:18,378 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad264c5e-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=CtZDPt7c6MaLnZTMsAIJNprHKVdXSqA2Shi5zS7aMCw%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hO/5vxPn9pIwrJeF4knqFvxqg/zcFPI9KrL7WaU03Gc=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'ad264c5e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. 2020-03-31 09:26:18,415 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:26:18,415 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 2020-03-31 09:26:18,416 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad264c5e-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=feecc5bf-c01e-004b-6f3e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=CtZDPt7c6MaLnZTMsAIJNprHKVdXSqA2Shi5zS7aMCw%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hO/5vxPn9pIwrJeF4knqFvxqg/zcFPI9KrL7WaU03Gc=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'ad264c5e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. storagereplication stdout | 2020-03-31 09:26:18,416 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad264c5e-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=feecc5bf-c01e-004b-6f3e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=CtZDPt7c6MaLnZTMsAIJNprHKVdXSqA2Shi5zS7aMCw%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:hO/5vxPn9pIwrJeF4knqFvxqg/zcFPI9KrL7WaU03Gc=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'ad264c5e-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. 2020-03-31 09:26:18,416 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:26:18,416 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:26:18,417 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ad2c3ad8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:18 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:18,417 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad2c3ad8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:On71Ci8pEViU52BFkfBI4k7ecnMGmKkf3zRRHLYamsg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. storagereplication stdout | 2020-03-31 09:26:18,417 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ad2c3ad8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:18 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:18,417 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad2c3ad8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:On71Ci8pEViU52BFkfBI4k7ecnMGmKkf3zRRHLYamsg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. 2020-03-31 09:26:18,499 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:18,499 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:18,500 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=d723bee2-e01e-0058-583e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad2c3ad8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:On71Ci8pEViU52BFkfBI4k7ecnMGmKkf3zRRHLYamsg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. 2020-03-31 09:26:18,500 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=d723bee2-e01e-0058-583e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:18,500 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=d723bee2-e01e-0058-583e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:18,501 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:26:18,500 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=d723bee2-e01e-0058-583e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ad2c3ad8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:On71Ci8pEViU52BFkfBI4k7ecnMGmKkf3zRRHLYamsg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:18 GMT'}. 2020-03-31 09:26:18,500 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=d723bee2-e01e-0058-583e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:18,500 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ad2c3ad8-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:18 GMT, Server-Request-ID=d723bee2-e01e-0058-583e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:18,501 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:26:20,501 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ae6a53b2-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:20 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:26:20,501 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ae6a53b2-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:20 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:20,502 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ae6a53b2-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jZORxn5ncu+j6vmKKXBB5mzEgEVgz4mjsGVjiezx2KU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:20 GMT'}. storagereplication stdout | 2020-03-31 09:26:20,502 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ae6a53b2-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jZORxn5ncu+j6vmKKXBB5mzEgEVgz4mjsGVjiezx2KU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:20 GMT'}. 2020-03-31 09:26:20,582 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:20,582 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:20,583 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:20 GMT, Server-Request-ID=d723c33d-e01e-0058-6e3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ae6a53b2-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jZORxn5ncu+j6vmKKXBB5mzEgEVgz4mjsGVjiezx2KU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:20 GMT'}. 2020-03-31 09:26:20,584 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:20 GMT, Server-Request-ID=d723c33d-e01e-0058-6e3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:20,584 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:20 GMT, Server-Request-ID=d723c33d-e01e-0058-6e3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:20,584 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:26:20,583 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:20 GMT, Server-Request-ID=d723c33d-e01e-0058-6e3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ae6a53b2-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jZORxn5ncu+j6vmKKXBB5mzEgEVgz4mjsGVjiezx2KU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:20 GMT'}. 2020-03-31 09:26:20,584 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:20 GMT, Server-Request-ID=d723c33d-e01e-0058-6e3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:20,584 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ae6a53b2-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:20 GMT, Server-Request-ID=d723c33d-e01e-0058-6e3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:20,584 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:26:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:26:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:31.440707+00:00 (in 9.999123 seconds) 2020-03-31 09:26:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:31 UTC)" (scheduled at 2020-03-31 09:26:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:26:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:31.440707+00:00 (in 9.999123 seconds) 2020-03-31 09:26:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:31 UTC)" (scheduled at 2020-03-31 09:26:21.440707+00:00) 2020-03-31 09:26:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:26:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:26:21,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:26:21,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:26:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:26:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:31 UTC)" executed successfully 2020-03-31 09:26:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:26:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:52.113221+00:00 (in 29.998805 seconds) 2020-03-31 09:26:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:52 UTC)" (scheduled at 2020-03-31 09:26:22.113221+00:00) 2020-03-31 09:26:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:26:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:52.113221+00:00 (in 29.998805 seconds) 2020-03-31 09:26:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:52 UTC)" (scheduled at 2020-03-31 09:26:22.113221+00:00) 2020-03-31 09:26:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:26:22,150 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:26:22,150 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:26:22,153 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:26:22,154 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:22,154 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:26:22,153 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:26:22,154 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:22,154 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:26:52 UTC)" executed successfully 2020-03-31 09:26:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:26:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:34.323352+00:00 (in 9.999012 seconds) 2020-03-31 09:26:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:34 UTC)" (scheduled at 2020-03-31 09:26:24.323352+00:00) 2020-03-31 09:26:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:26:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:34.323352+00:00 (in 9.999012 seconds) 2020-03-31 09:26:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:34 UTC)" (scheduled at 2020-03-31 09:26:24.323352+00:00) 2020-03-31 09:26:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 24, 325457), True, datetime.datetime(2020, 3, 31, 9, 26, 24, 325457), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:26:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 24, 325457), True, datetime.datetime(2020, 3, 31, 9, 26, 24, 325457), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:26:24,370 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:26:24,370 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:24,371 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:24,371 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:26:24,371 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:24,371 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:34 UTC)" executed successfully 2020-03-31 09:26:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:33.330112+00:00 (in 8.918246 seconds) 2020-03-31 09:26:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:24 UTC)" (scheduled at 2020-03-31 09:26:24.410927+00:00) 2020-03-31 09:26:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:26:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:33.330112+00:00 (in 8.918246 seconds) 2020-03-31 09:26:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:24 UTC)" (scheduled at 2020-03-31 09:26:24.410927+00:00) 2020-03-31 09:26:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:24 UTC)" executed successfully 2020-03-31 09:26:24,592 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b0da877a-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:24 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:24,592 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b0da877a-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:18s7N94twwVBS5+RnDIHmmlGXNXQPXUGvSdf/yciN2E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:24 GMT'}. 2020-03-31 09:26:24,491 [518] [DEBUG] [app] Starting request: urn:request:29222ed7-6c8d-4fc3-a1c8-ed4a22527be4 (/health/instance) 2020-03-31 09:26:24,575 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:24,592 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) storagereplication stdout | 2020-03-31 09:26:24,592 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b0da877a-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:24 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:24,592 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b0da877a-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:18s7N94twwVBS5+RnDIHmmlGXNXQPXUGvSdf/yciN2E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:24 GMT'}. gunicorn-web stdout | 2020-03-31 09:26:24,491 [518] [DEBUG] [app] Starting request: urn:request:29222ed7-6c8d-4fc3-a1c8-ed4a22527be4 (/health/instance) 2020-03-31 09:26:24,575 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:24,592 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:24,598 [538] [DEBUG] [app] Starting request: urn:request:784cd31a-10a7-4d7a-9f15-afe3b2ea6ce9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:24,598 [538] [DEBUG] [app] Starting request: urn:request:784cd31a-10a7-4d7a-9f15-afe3b2ea6ce9 (/v1/_internal_ping) 2020-03-31 09:26:24,602 [538] [DEBUG] [app] Ending request: urn:request:784cd31a-10a7-4d7a-9f15-afe3b2ea6ce9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:24,602 [538] [DEBUG] [app] Ending request: urn:request:784cd31a-10a7-4d7a-9f15-afe3b2ea6ce9 (/v1/_internal_ping) 2020-03-31 09:26:24,604 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.011 162 0.011) gunicorn-web stdout | 2020-03-31 09:26:24,604 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:24,606 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.011 162 0.011) gunicorn-web stdout | 2020-03-31 09:26:24,606 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:24,603 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:26:24,603 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:24,616 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:24,616 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:24,620 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:24,620 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:24,623 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:24,623 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:24,626 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:24,626 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:24,652 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:24,659 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:24,661 [519] [DEBUG] [app] Starting request: urn:request:000284c3-29b2-4d7b-ac66-746132f19b04 (/_internal_ping) 2020-03-31 09:26:24,663 [519] [DEBUG] [app] Ending request: urn:request:000284c3-29b2-4d7b-ac66-746132f19b04 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:24,652 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 159 0.005) 2020-03-31 09:26:24,666 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:24,667 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:24,659 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:24,683 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:24,661 [519] [DEBUG] [app] Starting request: urn:request:000284c3-29b2-4d7b-ac66-746132f19b04 (/_internal_ping) 2020-03-31 09:26:24,663 [519] [DEBUG] [app] Ending request: urn:request:000284c3-29b2-4d7b-ac66-746132f19b04 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 159 0.005) 2020-03-31 09:26:24,695 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:26:24,700 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:24,666 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.021 153 0.021) gunicorn-web stdout | 2020-03-31 09:26:24,667 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:24,683 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:24,695 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:26:24,726 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:24,700 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.021 153 0.021) gunicorn-web stdout | 2020-03-31 09:26:24,726 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:24,734 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:24,734 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:24,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:24 GMT, Server-Request-ID=d723ccb2-e01e-0058-023e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b0da877a-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:18s7N94twwVBS5+RnDIHmmlGXNXQPXUGvSdf/yciN2E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:24 GMT'}. 2020-03-31 09:26:24,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:24 GMT, Server-Request-ID=d723ccb2-e01e-0058-023e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:24,776 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:24 GMT, Server-Request-ID=d723ccb2-e01e-0058-023e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:24,776 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:26:24,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:24 GMT, Server-Request-ID=d723ccb2-e01e-0058-023e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b0da877a-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:18s7N94twwVBS5+RnDIHmmlGXNXQPXUGvSdf/yciN2E=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:24 GMT'}. 2020-03-31 09:26:24,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:24 GMT, Server-Request-ID=d723ccb2-e01e-0058-023e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:24,776 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b0da877a-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:24 GMT, Server-Request-ID=d723ccb2-e01e-0058-023e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:24,776 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:26:24,776 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:24,776 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:24,781 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:24,778 [543] [DEBUG] [app] Starting request: urn:request:ebf93f03-b688-4333-9087-998f51ae7828 (/c1/_internal_ping) 2020-03-31 09:26:24,780 [543] [DEBUG] [app] Ending request: urn:request:ebf93f03-b688-4333-9087-998f51ae7828 (/c1/_internal_ping) 2020-03-31 09:26:24,780 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:24,783 [518] [DEBUG] [app] Ending request: urn:request:29222ed7-6c8d-4fc3-a1c8-ed4a22527be4 (/health/instance) 2020-03-31 09:26:24,783 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:24,784 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.295 49 0.295) gunicorn-web stdout | 2020-03-31 09:26:24,781 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:26:24,778 [543] [DEBUG] [app] Starting request: urn:request:ebf93f03-b688-4333-9087-998f51ae7828 (/c1/_internal_ping) 2020-03-31 09:26:24,780 [543] [DEBUG] [app] Ending request: urn:request:ebf93f03-b688-4333-9087-998f51ae7828 (/c1/_internal_ping) 2020-03-31 09:26:24,780 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:24,783 [518] [DEBUG] [app] Ending request: urn:request:29222ed7-6c8d-4fc3-a1c8-ed4a22527be4 (/health/instance) 2020-03-31 09:26:24,783 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:24,784 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.295 49 0.295) 2020-03-31 09:26:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:26 UTC)" skipped: maximum number of running instances reached (1) storagereplication stdout | 2020-03-31 09:26:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:36.924497+00:00 (in 9.999047 seconds) storagereplication stdout | 2020-03-31 09:26:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:36.924497+00:00 (in 9.999047 seconds) 2020-03-31 09:26:28,273 [527] [DEBUG] [app] Starting request: urn:request:6c7ae5ae-c8e0-49a7-b446-c189f9a1fb2c (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:28,273 [527] [DEBUG] [app] Starting request: urn:request:6c7ae5ae-c8e0-49a7-b446-c189f9a1fb2c (/health/instance) 2020-03-31 09:26:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:28,280 [548] [DEBUG] [app] Starting request: urn:request:d6b6178f-5f24-4622-ba99-a442eb1582b3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:28,280 [548] [DEBUG] [app] Starting request: urn:request:d6b6178f-5f24-4622-ba99-a442eb1582b3 (/v1/_internal_ping) 2020-03-31 09:26:28,281 [548] [DEBUG] [app] Ending request: urn:request:d6b6178f-5f24-4622-ba99-a442eb1582b3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:28,281 [548] [DEBUG] [app] Ending request: urn:request:d6b6178f-5f24-4622-ba99-a442eb1582b3 (/v1/_internal_ping) 2020-03-31 09:26:28,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:26:28,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:28,286 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:28,286 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:28,288 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:28,288 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:28,289 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:28,289 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:28,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:28,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:28,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:28,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:28,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:28,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:28,311 [527] [DEBUG] [app] Starting request: urn:request:a9491b3b-64af-492b-9380-187b561b4590 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:28,311 [527] [DEBUG] [app] Starting request: urn:request:a9491b3b-64af-492b-9380-187b561b4590 (/_internal_ping) 2020-03-31 09:26:28,312 [527] [DEBUG] [app] Ending request: urn:request:a9491b3b-64af-492b-9380-187b561b4590 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:28,312 [527] [DEBUG] [app] Ending request: urn:request:a9491b3b-64af-492b-9380-187b561b4590 (/_internal_ping) 2020-03-31 09:26:28,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:28,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:28,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:28,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:26:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:28,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:28,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:28,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:28,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:28,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:26:28,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:26:28,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:28,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:28,325 [532] [DEBUG] [app] Starting request: urn:request:6246952b-570d-4f8f-a30a-30e5c80afe33 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:28,325 [532] [DEBUG] [app] Starting request: urn:request:6246952b-570d-4f8f-a30a-30e5c80afe33 (/c1/_internal_ping) 2020-03-31 09:26:28,326 [532] [DEBUG] [app] Ending request: urn:request:6246952b-570d-4f8f-a30a-30e5c80afe33 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:28,326 [532] [DEBUG] [app] Ending request: urn:request:6246952b-570d-4f8f-a30a-30e5c80afe33 (/c1/_internal_ping) 2020-03-31 09:26:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:28,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:26:28,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:28,329 [527] [DEBUG] [app] Ending request: urn:request:6c7ae5ae-c8e0-49a7-b446-c189f9a1fb2c (/health/instance) 2020-03-31 09:26:28,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:28,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:26:28,329 [527] [DEBUG] [app] Ending request: urn:request:6c7ae5ae-c8e0-49a7-b446-c189f9a1fb2c (/health/instance) 2020-03-31 09:26:28,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:28,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:26:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:26:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:00 UTC)" (scheduled at 2020-03-31 09:26:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:26:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:00 UTC)" (scheduled at 2020-03-31 09:26:30.337699+00:00) 2020-03-31 09:26:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:00.337699+00:00 (in 29.998978 seconds) buildlogsarchiver stdout | 2020-03-31 09:26:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:00.337699+00:00 (in 29.998978 seconds) 2020-03-31 09:26:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 26, 30, 339771), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:26:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 26, 30, 339771), False, 50, 1, 0]) 2020-03-31 09:26:30,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:26:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:26:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:26:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:00 UTC)" executed successfully 2020-03-31 09:26:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:26:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:41.440707+00:00 (in 9.999136 seconds) 2020-03-31 09:26:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:41 UTC)" (scheduled at 2020-03-31 09:26:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:26:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:41.440707+00:00 (in 9.999136 seconds) 2020-03-31 09:26:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:41 UTC)" (scheduled at 2020-03-31 09:26:31.440707+00:00) 2020-03-31 09:26:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:26:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:26:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:26:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:26:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:26:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:41 UTC)" executed successfully 2020-03-31 09:26:32,784 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b5bc8504-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:32 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:26:32,784 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b5bc8504-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:32 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:26:32,784 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b5bc8504-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:wNBvwEKiv9r6X0wn5isJ7hrvEwzrvx1pptE6fIkeTWA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:32 GMT'}. storagereplication stdout | 2020-03-31 09:26:32,784 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b5bc8504-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:wNBvwEKiv9r6X0wn5isJ7hrvEwzrvx1pptE6fIkeTWA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:32 GMT'}. 2020-03-31 09:26:32,870 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:32,870 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:32,871 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:32 GMT, Server-Request-ID=d723e16b-e01e-0058-643e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b5bc8504-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:wNBvwEKiv9r6X0wn5isJ7hrvEwzrvx1pptE6fIkeTWA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:32 GMT'}. 2020-03-31 09:26:32,871 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:32 GMT, Server-Request-ID=d723e16b-e01e-0058-643e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:26:32,871 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:32 GMT, Server-Request-ID=d723e16b-e01e-0058-643e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b5bc8504-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:wNBvwEKiv9r6X0wn5isJ7hrvEwzrvx1pptE6fIkeTWA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:32 GMT'}. 2020-03-31 09:26:32,871 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:32 GMT, Server-Request-ID=d723e16b-e01e-0058-643e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:32,871 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:32 GMT, Server-Request-ID=d723e16b-e01e-0058-643e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:32,871 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:26:32,871 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b5bc8504-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:32 GMT, Server-Request-ID=d723e16b-e01e-0058-643e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:32,871 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:26:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:26:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:03.330112+00:00 (in 29.998809 seconds) 2020-03-31 09:26:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:03 UTC)" (scheduled at 2020-03-31 09:26:33.330112+00:00) 2020-03-31 09:26:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:26:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:03.330112+00:00 (in 29.998809 seconds) 2020-03-31 09:26:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:03 UTC)" (scheduled at 2020-03-31 09:26:33.330112+00:00) 2020-03-31 09:26:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 33, 332301), True, datetime.datetime(2020, 3, 31, 9, 26, 33, 332301), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:26:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 33, 332301), True, datetime.datetime(2020, 3, 31, 9, 26, 33, 332301), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:26:33,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:33,353 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:26:33,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:33,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:26:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:03 UTC)" executed successfully 2020-03-31 09:26:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:26:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:33 UTC)" (scheduled at 2020-03-31 09:26:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:26:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:33 UTC)" (scheduled at 2020-03-31 09:26:33.366791+00:00) 2020-03-31 09:26:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:43.539999+00:00 (in 10.172317 seconds) 2020-03-31 09:26:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:26:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:43.539999+00:00 (in 10.172317 seconds) 2020-03-31 09:26:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 33, 368805), True, datetime.datetime(2020, 3, 31, 9, 26, 33, 368805), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:26:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 33, 368805), True, datetime.datetime(2020, 3, 31, 9, 26, 33, 368805), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:26:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:26:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:26:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:33 UTC)" executed successfully 2020-03-31 09:26:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:26:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:34,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:44.323352+00:00 (in 9.996174 seconds) 2020-03-31 09:26:34,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:44 UTC)" (scheduled at 2020-03-31 09:26:34.323352+00:00) 2020-03-31 09:26:34,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:26:34,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:44.323352+00:00 (in 9.996174 seconds) 2020-03-31 09:26:34,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:44 UTC)" (scheduled at 2020-03-31 09:26:34.323352+00:00) 2020-03-31 09:26:34,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:34,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 34, 328275), True, datetime.datetime(2020, 3, 31, 9, 26, 34, 328275), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:26:34,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 34, 328275), True, datetime.datetime(2020, 3, 31, 9, 26, 34, 328275), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:26:34,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:26:34,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:26:34,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:34,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:44 UTC)" executed successfully 2020-03-31 09:26:34,449 [518] [DEBUG] [app] Starting request: urn:request:2a67e877-ca6f-4723-b864-d705224e9288 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:34,449 [518] [DEBUG] [app] Starting request: urn:request:2a67e877-ca6f-4723-b864-d705224e9288 (/health/instance) 2020-03-31 09:26:34,451 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:34,451 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:34,455 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:34,455 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:34,457 [545] [DEBUG] [app] Starting request: urn:request:5bc698eb-0e94-46ce-b98a-2784d489a770 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:34,457 [545] [DEBUG] [app] Starting request: urn:request:5bc698eb-0e94-46ce-b98a-2784d489a770 (/v1/_internal_ping) 2020-03-31 09:26:34,458 [545] [DEBUG] [app] Ending request: urn:request:5bc698eb-0e94-46ce-b98a-2784d489a770 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:34,458 [545] [DEBUG] [app] Ending request: urn:request:5bc698eb-0e94-46ce-b98a-2784d489a770 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:34,458 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:34,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:26:34,458 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:34,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:34,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:34,460 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:26:34,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:26:34,460 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:34,462 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:34,462 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:34,463 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:34,463 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:34,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:34,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:34,509 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:34,509 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:34,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:34,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:34,514 [518] [DEBUG] [app] Starting request: urn:request:dc484955-43d7-4b88-ad8c-e79fdef81f2e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:34,514 [518] [DEBUG] [app] Starting request: urn:request:dc484955-43d7-4b88-ad8c-e79fdef81f2e (/_internal_ping) 2020-03-31 09:26:34,515 [518] [DEBUG] [app] Ending request: urn:request:dc484955-43d7-4b88-ad8c-e79fdef81f2e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:34,515 [518] [DEBUG] [app] Ending request: urn:request:dc484955-43d7-4b88-ad8c-e79fdef81f2e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:26:34,515 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:34,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:34,515 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:34,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:34,519 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:34,519 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:34,520 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:26:34,520 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:26:34,522 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:34,522 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:34,523 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:26:34,523 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:26:34,526 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:34,526 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:34,528 [543] [DEBUG] [app] Starting request: urn:request:6a76bec8-60e1-480e-9e11-ba722a57f6fa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:34,528 [543] [DEBUG] [app] Starting request: urn:request:6a76bec8-60e1-480e-9e11-ba722a57f6fa (/c1/_internal_ping) 2020-03-31 09:26:34,529 [543] [DEBUG] [app] Ending request: urn:request:6a76bec8-60e1-480e-9e11-ba722a57f6fa (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:34,529 [543] [DEBUG] [app] Ending request: urn:request:6a76bec8-60e1-480e-9e11-ba722a57f6fa (/c1/_internal_ping) 2020-03-31 09:26:34,530 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:34,530 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:34,530 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:26:34,530 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:34,531 [518] [DEBUG] [app] Ending request: urn:request:2a67e877-ca6f-4723-b864-d705224e9288 (/health/instance) 2020-03-31 09:26:34,532 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:26:34,531 [518] [DEBUG] [app] Ending request: urn:request:2a67e877-ca6f-4723-b864-d705224e9288 (/health/instance) 2020-03-31 09:26:34,532 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:34,532 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.084 49 0.084) gunicorn-web stdout | 2020-03-31 09:26:34,532 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.084 49 0.084) 2020-03-31 09:26:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:26:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:38.010934+00:00 (in 1.085317 seconds) storagereplication stdout | 2020-03-31 09:26:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:38.010934+00:00 (in 1.085317 seconds) 2020-03-31 09:26:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:26:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:46.924497+00:00 (in 8.912707 seconds) 2020-03-31 09:26:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:38 UTC)" (scheduled at 2020-03-31 09:26:38.010934+00:00) 2020-03-31 09:26:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:26:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:46.924497+00:00 (in 8.912707 seconds) 2020-03-31 09:26:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:38 UTC)" (scheduled at 2020-03-31 09:26:38.010934+00:00) 2020-03-31 09:26:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:38 UTC)" executed successfully 2020-03-31 09:26:38,272 [527] [DEBUG] [app] Starting request: urn:request:3b020a92-b60e-4e70-9081-11902f71a54b (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:38,272 [527] [DEBUG] [app] Starting request: urn:request:3b020a92-b60e-4e70-9081-11902f71a54b (/health/instance) 2020-03-31 09:26:38,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:38,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:38,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:38,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:38,279 [555] [DEBUG] [app] Starting request: urn:request:e99dc665-597e-4ffe-8fc3-dbd1dd1bba71 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:38,279 [555] [DEBUG] [app] Starting request: urn:request:e99dc665-597e-4ffe-8fc3-dbd1dd1bba71 (/v1/_internal_ping) 2020-03-31 09:26:38,280 [555] [DEBUG] [app] Ending request: urn:request:e99dc665-597e-4ffe-8fc3-dbd1dd1bba71 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:38,280 [555] [DEBUG] [app] Ending request: urn:request:e99dc665-597e-4ffe-8fc3-dbd1dd1bba71 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:38,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:38,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:38,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:38,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:26:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:38,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:38,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:38,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:38,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:38,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:38,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:38,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:38,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:38,315 [527] [DEBUG] [app] Starting request: urn:request:8a028c4b-28e8-4da0-8bae-6c82a7a7cc70 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:38,315 [527] [DEBUG] [app] Starting request: urn:request:8a028c4b-28e8-4da0-8bae-6c82a7a7cc70 (/_internal_ping) 2020-03-31 09:26:38,316 [527] [DEBUG] [app] Ending request: urn:request:8a028c4b-28e8-4da0-8bae-6c82a7a7cc70 (/_internal_ping) 2020-03-31 09:26:38,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:26:38,316 [527] [DEBUG] [app] Ending request: urn:request:8a028c4b-28e8-4da0-8bae-6c82a7a7cc70 (/_internal_ping) 2020-03-31 09:26:38,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:26:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:38,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:38,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:38,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:38,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:38,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:38,331 [541] [DEBUG] [app] Starting request: urn:request:1c290b89-4b45-47ae-a029-f80a23efb8ef (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:38,331 [541] [DEBUG] [app] Starting request: urn:request:1c290b89-4b45-47ae-a029-f80a23efb8ef (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:38,332 [541] [DEBUG] [app] Ending request: urn:request:1c290b89-4b45-47ae-a029-f80a23efb8ef (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:38,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:38,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:26:38,332 [541] [DEBUG] [app] Ending request: urn:request:1c290b89-4b45-47ae-a029-f80a23efb8ef (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:38,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:38,334 [527] [DEBUG] [app] Ending request: urn:request:3b020a92-b60e-4e70-9081-11902f71a54b (/health/instance) 2020-03-31 09:26:38,335 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:26:38,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:38,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:26:38,334 [527] [DEBUG] [app] Ending request: urn:request:3b020a92-b60e-4e70-9081-11902f71a54b (/health/instance) 2020-03-31 09:26:38,335 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:26:38,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:26:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:26:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:51.440707+00:00 (in 9.998928 seconds) 2020-03-31 09:26:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:51 UTC)" (scheduled at 2020-03-31 09:26:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:26:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:51.440707+00:00 (in 9.998928 seconds) 2020-03-31 09:26:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:51 UTC)" (scheduled at 2020-03-31 09:26:41.440707+00:00) 2020-03-31 09:26:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:26:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:26:41,468 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:26:41,468 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:26:41,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:41,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:26:41,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:41,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:51 UTC)" executed successfully 2020-03-31 09:26:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:43 UTC)" (scheduled at 2020-03-31 09:26:43.539999+00:00) 2020-03-31 09:26:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:43 UTC)" executed successfully 2020-03-31 09:26:43,543 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:33.366791+00:00 (in 49.825167 seconds) exportactionlogsworker stdout | 2020-03-31 09:26:43,541 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:26:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:43 UTC)" (scheduled at 2020-03-31 09:26:43.539999+00:00) 2020-03-31 09:26:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:26:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:43 UTC)" executed successfully 2020-03-31 09:26:43,543 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:33.366791+00:00 (in 49.825167 seconds) 2020-03-31 09:26:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:26:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:49.498966+00:00 (in 5.174406 seconds) 2020-03-31 09:26:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:54 UTC)" (scheduled at 2020-03-31 09:26:44.323352+00:00) 2020-03-31 09:26:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:26:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:49.498966+00:00 (in 5.174406 seconds) 2020-03-31 09:26:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:54 UTC)" (scheduled at 2020-03-31 09:26:44.323352+00:00) 2020-03-31 09:26:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 44, 325517), True, datetime.datetime(2020, 3, 31, 9, 26, 44, 325517), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:26:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 44, 325517), True, datetime.datetime(2020, 3, 31, 9, 26, 44, 325517), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:26:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:26:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:26:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:54 UTC)" executed successfully 2020-03-31 09:26:44,446 [527] [DEBUG] [app] Starting request: urn:request:a2a43e50-70e6-40c6-8cdb-9bbc503b85d1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:44,446 [527] [DEBUG] [app] Starting request: urn:request:a2a43e50-70e6-40c6-8cdb-9bbc503b85d1 (/health/instance) 2020-03-31 09:26:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:44,453 [538] [DEBUG] [app] Starting request: urn:request:bfee76a4-6e26-4ab5-9dbc-f68fedc71b4f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:44,453 [538] [DEBUG] [app] Starting request: urn:request:bfee76a4-6e26-4ab5-9dbc-f68fedc71b4f (/v1/_internal_ping) 2020-03-31 09:26:44,454 [538] [DEBUG] [app] Ending request: urn:request:bfee76a4-6e26-4ab5-9dbc-f68fedc71b4f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:44,454 [538] [DEBUG] [app] Ending request: urn:request:bfee76a4-6e26-4ab5-9dbc-f68fedc71b4f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:26:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:44,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:44,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:44,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:44,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:44,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:44,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:44,488 [527] [DEBUG] [app] Starting request: urn:request:bd122360-041f-4989-8fda-ee39dc0a459e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:44,488 [527] [DEBUG] [app] Starting request: urn:request:bd122360-041f-4989-8fda-ee39dc0a459e (/_internal_ping) 2020-03-31 09:26:44,489 [527] [DEBUG] [app] Ending request: urn:request:bd122360-041f-4989-8fda-ee39dc0a459e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:44,489 [527] [DEBUG] [app] Ending request: urn:request:bd122360-041f-4989-8fda-ee39dc0a459e (/_internal_ping) 2020-03-31 09:26:44,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:44,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:44,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:26:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:26:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:44,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:26:44,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:44,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:44,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:44,504 [541] [DEBUG] [app] Starting request: urn:request:9fe55fea-a1c1-452f-b100-2c98cac0ad9f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:44,504 [541] [DEBUG] [app] Starting request: urn:request:9fe55fea-a1c1-452f-b100-2c98cac0ad9f (/c1/_internal_ping) 2020-03-31 09:26:44,505 [541] [DEBUG] [app] Ending request: urn:request:9fe55fea-a1c1-452f-b100-2c98cac0ad9f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:44,505 [541] [DEBUG] [app] Ending request: urn:request:9fe55fea-a1c1-452f-b100-2c98cac0ad9f (/c1/_internal_ping) 2020-03-31 09:26:44,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:44,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:44,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:26:44,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:44,507 [527] [DEBUG] [app] Ending request: urn:request:a2a43e50-70e6-40c6-8cdb-9bbc503b85d1 (/health/instance) 2020-03-31 09:26:44,507 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:26:44,507 [527] [DEBUG] [app] Ending request: urn:request:a2a43e50-70e6-40c6-8cdb-9bbc503b85d1 (/health/instance) 2020-03-31 09:26:44,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:44,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:26:44,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:26:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:26:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:15.064514+00:00 (in 29.999258 seconds) 2020-03-31 09:26:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:15 UTC)" (scheduled at 2020-03-31 09:26:45.064514+00:00) gcworker stdout | 2020-03-31 09:26:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:15.064514+00:00 (in 29.999258 seconds) 2020-03-31 09:26:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:15 UTC)" (scheduled at 2020-03-31 09:26:45.064514+00:00) 2020-03-31 09:26:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437205, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:26:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437205, 1209600, 500, 1, 0]) 2020-03-31 09:26:45,091 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:26:45,091 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:26:45,091 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:45,091 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:26:45,091 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:45,091 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:15 UTC)" executed successfully 2020-03-31 09:26:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:26:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:56.924497+00:00 (in 9.999124 seconds) storagereplication stdout | 2020-03-31 09:26:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:56.924497+00:00 (in 9.999124 seconds) 2020-03-31 09:26:48,273 [526] [DEBUG] [app] Starting request: urn:request:9b749c4b-63b6-4a73-8ace-02887dedefa8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:48,273 [526] [DEBUG] [app] Starting request: urn:request:9b749c4b-63b6-4a73-8ace-02887dedefa8 (/health/instance) 2020-03-31 09:26:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:48,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:48,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:48,280 [553] [DEBUG] [app] Starting request: urn:request:58dbf37f-09eb-4f41-a512-d84d7c1cfb8b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:48,280 [553] [DEBUG] [app] Starting request: urn:request:58dbf37f-09eb-4f41-a512-d84d7c1cfb8b (/v1/_internal_ping) 2020-03-31 09:26:48,281 [553] [DEBUG] [app] Ending request: urn:request:58dbf37f-09eb-4f41-a512-d84d7c1cfb8b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:48,281 [553] [DEBUG] [app] Ending request: urn:request:58dbf37f-09eb-4f41-a512-d84d7c1cfb8b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:26:48,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:48,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:26:48,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:48,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:48,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:48,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:48,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:48,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:48,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:48,286 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:48,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:48,287 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:48,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:48,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:48,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:48,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:48,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:48,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:48,314 [526] [DEBUG] [app] Starting request: urn:request:250df723-2145-49e4-93ec-b4b004be4639 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:48,314 [526] [DEBUG] [app] Starting request: urn:request:250df723-2145-49e4-93ec-b4b004be4639 (/_internal_ping) 2020-03-31 09:26:48,315 [526] [DEBUG] [app] Ending request: urn:request:250df723-2145-49e4-93ec-b4b004be4639 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:48,315 [526] [DEBUG] [app] Ending request: urn:request:250df723-2145-49e4-93ec-b4b004be4639 (/_internal_ping) 2020-03-31 09:26:48,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:48,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:26:48,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:48,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:26:48,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:48,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:48,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:48,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:48,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:48,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:48,328 [543] [DEBUG] [app] Starting request: urn:request:78af2de7-1095-4ad6-9f4f-4b9a0211a0df (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:48,328 [543] [DEBUG] [app] Starting request: urn:request:78af2de7-1095-4ad6-9f4f-4b9a0211a0df (/c1/_internal_ping) 2020-03-31 09:26:48,329 [543] [DEBUG] [app] Ending request: urn:request:78af2de7-1095-4ad6-9f4f-4b9a0211a0df (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:48,329 [543] [DEBUG] [app] Ending request: urn:request:78af2de7-1095-4ad6-9f4f-4b9a0211a0df (/c1/_internal_ping) 2020-03-31 09:26:48,330 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:48,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:48,330 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:26:48,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:48,331 [526] [DEBUG] [app] Ending request: urn:request:9b749c4b-63b6-4a73-8ace-02887dedefa8 (/health/instance) 2020-03-31 09:26:48,332 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:26:48,331 [526] [DEBUG] [app] Ending request: urn:request:9b749c4b-63b6-4a73-8ace-02887dedefa8 (/health/instance) 2020-03-31 09:26:48,332 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:48,332 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:26:48,332 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:26:48,887 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:26:48,887 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy 2020-03-31 09:26:48,887 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:26:48,887 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:26:48,887 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:26:48,887 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:26:48,897 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) storagereplication stdout | 2020-03-31 09:26:48,897 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) 2020-03-31 09:26:48,900 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 31, 48, 896344), True, u'5440e0e5-24a9-4eb6-b61a-88257c21c199', 2]) storagereplication stdout | 2020-03-31 09:26:48,900 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 31, 48, 896344), True, u'5440e0e5-24a9-4eb6-b61a-88257c21c199', 2]) 2020-03-31 09:26:48,907 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 48, 905913), True, datetime.datetime(2020, 3, 31, 9, 26, 48, 905913), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:26:48,907 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 48, 905913), True, datetime.datetime(2020, 3, 31, 9, 26, 48, 905913), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:26:48,912 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 46, 48, 905913), 1, u'420f9ce3-3b19-43f7-93a7-4ec55d2764f9', 1, u'3b2761af-2d96-4a58-8a4e-582182e7c8c5']) storagereplication stdout | 2020-03-31 09:26:48,912 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 46, 48, 905913), 1, u'420f9ce3-3b19-43f7-93a7-4ec55d2764f9', 1, u'3b2761af-2d96-4a58-8a4e-582182e7c8c5']) 2020-03-31 09:26:48,918 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} storagereplication stdout | 2020-03-31 09:26:48,918 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:26:48,918 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:48,918 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 storagereplication stdout | 2020-03-31 09:26:48,918 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:48,918 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 2020-03-31 09:26:48,920 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:26:48,920 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:26:48,938 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:26:48,938 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:26:48,946 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) storagereplication stdout | 2020-03-31 09:26:48,946 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:26:48,957 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:26:48,958 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:bf606fa8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:26:48,958 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf606fa8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf606fa8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZDis7U48BBMQJxxZZpi9GiD5zgxTn9kzrs2RuUMrar8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. storagereplication stdout | 2020-03-31 09:26:48,957 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:26:48,958 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:bf606fa8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:26:48,958 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf606fa8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf606fa8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZDis7U48BBMQJxxZZpi9GiD5zgxTn9kzrs2RuUMrar8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. 2020-03-31 09:26:48,968 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:26:48,968 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:26:48,969 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf606fa8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=feeccd79-c01e-004b-753e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf606fa8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZDis7U48BBMQJxxZZpi9GiD5zgxTn9kzrs2RuUMrar8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. storagereplication stdout | 2020-03-31 09:26:48,969 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf606fa8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=feeccd79-c01e-004b-753e-07233c000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf606fa8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZDis7U48BBMQJxxZZpi9GiD5zgxTn9kzrs2RuUMrar8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. 2020-03-31 09:26:48,971 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:26:48,971 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:48,971 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:26:48,972 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:bf629788-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=wTzOn4Sa8lBEkV4j6gOSuhGUDxI2bG/QohnzcLCE/k4%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A48Z x-ms-date:Tue, 31 Mar 2020 09:26:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:26:48,972 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf629788-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=wTzOn4Sa8lBEkV4j6gOSuhGUDxI2bG/QohnzcLCE/k4%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0/SIOdzuwzGk/+2SrHgfCjz55A+KgGZrfUzvjEwJ51A=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'bf629788-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. storagereplication stdout | 2020-03-31 09:26:48,971 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:26:48,971 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:48,971 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:26:48,972 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:bf629788-7331-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=wTzOn4Sa8lBEkV4j6gOSuhGUDxI2bG/QohnzcLCE/k4%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A48Z x-ms-date:Tue, 31 Mar 2020 09:26:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:26:48,972 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf629788-7331-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=wTzOn4Sa8lBEkV4j6gOSuhGUDxI2bG/QohnzcLCE/k4%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0/SIOdzuwzGk/+2SrHgfCjz55A+KgGZrfUzvjEwJ51A=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'bf629788-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. 2020-03-31 09:26:48,997 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:26:48,997 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 2020-03-31 09:26:49,000 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf629788-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=feeccd82-c01e-004b-7b3e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=wTzOn4Sa8lBEkV4j6gOSuhGUDxI2bG/QohnzcLCE/k4%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0/SIOdzuwzGk/+2SrHgfCjz55A+KgGZrfUzvjEwJ51A=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'bf629788-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. storagereplication stdout | 2020-03-31 09:26:49,000 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf629788-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=feeccd82-c01e-004b-7b3e-07233c000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=wTzOn4Sa8lBEkV4j6gOSuhGUDxI2bG/QohnzcLCE/k4%3D&sv=2017-07-29&se=2020-03-31T09%3A27%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0/SIOdzuwzGk/+2SrHgfCjz55A+KgGZrfUzvjEwJ51A=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'bf629788-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:26:48 GMT'}. 2020-03-31 09:26:49,003 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:26:49,003 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:26:49,004 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:bf67663c-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:49 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:26:49,005 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf67663c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Z7+VzKx3jVg778hwvyJiY4+EvO4cFz3XMUgsTX6o0z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:49 GMT'}. storagereplication stdout | 2020-03-31 09:26:49,004 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:bf67663c-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:49 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:26:49,005 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf67663c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Z7+VzKx3jVg778hwvyJiY4+EvO4cFz3XMUgsTX6o0z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:49 GMT'}. 2020-03-31 09:26:49,097 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:49,097 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:49,123 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=d7240475-e01e-0058-3d3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf67663c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Z7+VzKx3jVg778hwvyJiY4+EvO4cFz3XMUgsTX6o0z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:49 GMT'}. 2020-03-31 09:26:49,124 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=d7240475-e01e-0058-3d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:49,124 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=d7240475-e01e-0058-3d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:49,124 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:26:49,123 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=d7240475-e01e-0058-3d3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bf67663c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Z7+VzKx3jVg778hwvyJiY4+EvO4cFz3XMUgsTX6o0z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:49 GMT'}. 2020-03-31 09:26:49,124 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=d7240475-e01e-0058-3d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:49,124 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=bf67663c-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:48 GMT, Server-Request-ID=d7240475-e01e-0058-3d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:49,124 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:26:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:54.323352+00:00 (in 4.823512 seconds) 2020-03-31 09:26:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:49 UTC)" (scheduled at 2020-03-31 09:26:49.498966+00:00) 2020-03-31 09:26:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:26:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:54.323352+00:00 (in 4.823512 seconds) 2020-03-31 09:26:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:49 UTC)" (scheduled at 2020-03-31 09:26:49.498966+00:00) 2020-03-31 09:26:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:49 UTC)" executed successfully 2020-03-31 09:26:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:26:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:59.560326+00:00 (in 8.448909 seconds) 2020-03-31 09:26:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:51 UTC)" (scheduled at 2020-03-31 09:26:51.110610+00:00) 2020-03-31 09:26:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:26:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:26:59.560326+00:00 (in 8.448909 seconds) 2020-03-31 09:26:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:51 UTC)" (scheduled at 2020-03-31 09:26:51.110610+00:00) 2020-03-31 09:26:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 51, 112068), True, datetime.datetime(2020, 3, 31, 9, 26, 51, 112068), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:26:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 51, 112068), True, datetime.datetime(2020, 3, 31, 9, 26, 51, 112068), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:26:51,126 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c0ab484c-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:51 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:26:51,126 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c0ab484c-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:51 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:26:51,126 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c0ab484c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DIuPrCkppFxSojF+FJ31UQDMpiaAy1OBrJgzgg7d2pU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:51 GMT'}. storagereplication stdout | 2020-03-31 09:26:51,126 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c0ab484c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DIuPrCkppFxSojF+FJ31UQDMpiaAy1OBrJgzgg7d2pU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:51 GMT'}. 2020-03-31 09:26:51,215 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:51,215 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:51,235 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:51 GMT, Server-Request-ID=d72408e4-e01e-0058-1b3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c0ab484c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DIuPrCkppFxSojF+FJ31UQDMpiaAy1OBrJgzgg7d2pU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:51 GMT'}. 2020-03-31 09:26:51,235 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:51 GMT, Server-Request-ID=d72408e4-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:51,236 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:51 GMT, Server-Request-ID=d72408e4-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:51,236 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:26:51,235 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:51 GMT, Server-Request-ID=d72408e4-e01e-0058-1b3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c0ab484c-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DIuPrCkppFxSojF+FJ31UQDMpiaAy1OBrJgzgg7d2pU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:51 GMT'}. 2020-03-31 09:26:51,235 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:51 GMT, Server-Request-ID=d72408e4-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:51,236 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c0ab484c-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:51 GMT, Server-Request-ID=d72408e4-e01e-0058-1b3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:51,236 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:26:51,325 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:51,325 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:51,325 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:26:51,325 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:51,325 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:51,325 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:51 UTC)" executed successfully 2020-03-31 09:26:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:01.440707+00:00 (in 9.998149 seconds) repositoryactioncounter stdout | 2020-03-31 09:26:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:01 UTC)" (scheduled at 2020-03-31 09:26:51.440707+00:00) 2020-03-31 09:26:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:26:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:01.440707+00:00 (in 9.998149 seconds) repositoryactioncounter stdout | 2020-03-31 09:26:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:01 UTC)" (scheduled at 2020-03-31 09:26:51.440707+00:00) 2020-03-31 09:26:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:26:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:26:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:26:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:26:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:01 UTC)" executed successfully 2020-03-31 09:26:52,131 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:52,131 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:22.113221+00:00 (in 29.981741 seconds) 2020-03-31 09:26:52,138 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:22 UTC)" (scheduled at 2020-03-31 09:26:52.113221+00:00) 2020-03-31 09:26:52,151 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:26:52,131 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:52,131 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:22.113221+00:00 (in 29.981741 seconds) 2020-03-31 09:26:52,138 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:22 UTC)" (scheduled at 2020-03-31 09:26:52.113221+00:00) 2020-03-31 09:26:52,151 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:26:52,215 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:26:52,215 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:26:52,231 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:26:52,231 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:52,231 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:26:52,231 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:26:52,231 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:52,231 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:22 UTC)" executed successfully 2020-03-31 09:26:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:54,347 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:04.323352+00:00 (in 9.975667 seconds) 2020-03-31 09:26:54,348 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:04 UTC)" (scheduled at 2020-03-31 09:26:54.323352+00:00) 2020-03-31 09:26:54,348 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:54,351 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 54, 348473), True, datetime.datetime(2020, 3, 31, 9, 26, 54, 348473), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:26:54,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:54,347 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:04.323352+00:00 (in 9.975667 seconds) 2020-03-31 09:26:54,348 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:04 UTC)" (scheduled at 2020-03-31 09:26:54.323352+00:00) 2020-03-31 09:26:54,348 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:26:54,351 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 26, 54, 348473), True, datetime.datetime(2020, 3, 31, 9, 26, 54, 348473), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:26:54,390 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:26:54,390 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:26:54,390 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:54,390 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:26:54,390 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:54,390 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:04 UTC)" executed successfully 2020-03-31 09:26:54,447 [518] [DEBUG] [app] Starting request: urn:request:2365a0a7-34d8-44b0-b6d4-76aba1abfca4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:54,447 [518] [DEBUG] [app] Starting request: urn:request:2365a0a7-34d8-44b0-b6d4-76aba1abfca4 (/health/instance) 2020-03-31 09:26:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:54,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:54,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:54,455 [555] [DEBUG] [app] Starting request: urn:request:9711d2a6-a58b-46f0-bd0b-2fd3fbd61f38 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:54,455 [555] [DEBUG] [app] Starting request: urn:request:9711d2a6-a58b-46f0-bd0b-2fd3fbd61f38 (/v1/_internal_ping) 2020-03-31 09:26:54,456 [555] [DEBUG] [app] Ending request: urn:request:9711d2a6-a58b-46f0-bd0b-2fd3fbd61f38 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:54,456 [555] [DEBUG] [app] Ending request: urn:request:9711d2a6-a58b-46f0-bd0b-2fd3fbd61f38 (/v1/_internal_ping) 2020-03-31 09:26:54,457 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:26:54,457 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:54,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:26:54,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:54,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:26:54,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:54,462 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:54,462 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:54,464 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:54,464 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:54,465 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:54,465 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:54,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:54,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:54,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:54,488 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:54,494 [518] [DEBUG] [app] Starting request: urn:request:8ddf6f8e-cb82-42be-82dc-2e499fadf509 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:54,494 [518] [DEBUG] [app] Starting request: urn:request:8ddf6f8e-cb82-42be-82dc-2e499fadf509 (/_internal_ping) 2020-03-31 09:26:54,494 [518] [DEBUG] [app] Ending request: urn:request:8ddf6f8e-cb82-42be-82dc-2e499fadf509 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:54,494 [518] [DEBUG] [app] Ending request: urn:request:8ddf6f8e-cb82-42be-82dc-2e499fadf509 (/_internal_ping) 2020-03-31 09:26:54,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:26:54,495 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:26:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:26:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:26:54,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:54,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:26:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:54,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:54,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:54,512 [543] [DEBUG] [app] Starting request: urn:request:3d79b6e3-5a5c-4adb-829f-ee63415a12e4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:54,512 [543] [DEBUG] [app] Starting request: urn:request:3d79b6e3-5a5c-4adb-829f-ee63415a12e4 (/c1/_internal_ping) 2020-03-31 09:26:54,513 [543] [DEBUG] [app] Ending request: urn:request:3d79b6e3-5a5c-4adb-829f-ee63415a12e4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:54,513 [543] [DEBUG] [app] Ending request: urn:request:3d79b6e3-5a5c-4adb-829f-ee63415a12e4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:26:54,514 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:26:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:26:54,515 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:54,516 [518] [DEBUG] [app] Ending request: urn:request:2365a0a7-34d8-44b0-b6d4-76aba1abfca4 (/health/instance) 2020-03-31 09:26:54,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:54,517 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:26:54,514 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:54,515 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:54,516 [518] [DEBUG] [app] Ending request: urn:request:2365a0a7-34d8-44b0-b6d4-76aba1abfca4 (/health/instance) 2020-03-31 09:26:54,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:54,517 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:26:55,240 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c31f0834-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:55 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:26:55,240 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c31f0834-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:26:55 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:26:55,240 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c31f0834-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:occkWtz30SeCri7IbpdSo+t2SzShCcTw+YfT5dvWTpQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:55 GMT'}. storagereplication stdout | 2020-03-31 09:26:55,240 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c31f0834-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:occkWtz30SeCri7IbpdSo+t2SzShCcTw+YfT5dvWTpQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:55 GMT'}. 2020-03-31 09:26:55,321 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:26:55,321 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:26:55,322 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:55 GMT, Server-Request-ID=d724133a-e01e-0058-0d3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c31f0834-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:occkWtz30SeCri7IbpdSo+t2SzShCcTw+YfT5dvWTpQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:55 GMT'}. 2020-03-31 09:26:55,323 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:55 GMT, Server-Request-ID=d724133a-e01e-0058-0d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:55,323 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:55 GMT, Server-Request-ID=d724133a-e01e-0058-0d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:26:55,322 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:26:55 GMT, Server-Request-ID=d724133a-e01e-0058-0d3e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c31f0834-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:occkWtz30SeCri7IbpdSo+t2SzShCcTw+YfT5dvWTpQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:26:55 GMT'}. 2020-03-31 09:26:55,323 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:26:55 GMT, Server-Request-ID=d724133a-e01e-0058-0d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:55,323 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c31f0834-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:26:55 GMT, Server-Request-ID=d724133a-e01e-0058-0d3e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:26:55,323 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:26:55,323 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:26:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:26:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:03.010318+00:00 (in 6.085166 seconds) storagereplication stdout | 2020-03-31 09:26:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:26:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:26:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:03.010318+00:00 (in 6.085166 seconds) 2020-03-31 09:26:58,273 [518] [DEBUG] [app] Starting request: urn:request:84bba98c-46b9-4e34-8db4-38c3ec0a5ec6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:26:58,273 [518] [DEBUG] [app] Starting request: urn:request:84bba98c-46b9-4e34-8db4-38c3ec0a5ec6 (/health/instance) 2020-03-31 09:26:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:58,278 [545] [DEBUG] [app] Starting request: urn:request:3802269b-bf7c-4ac1-850a-74a5d25d5f97 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:58,278 [545] [DEBUG] [app] Starting request: urn:request:3802269b-bf7c-4ac1-850a-74a5d25d5f97 (/v1/_internal_ping) 2020-03-31 09:26:58,279 [545] [DEBUG] [app] Ending request: urn:request:3802269b-bf7c-4ac1-850a-74a5d25d5f97 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:26:58,279 [545] [DEBUG] [app] Ending request: urn:request:3802269b-bf7c-4ac1-850a-74a5d25d5f97 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:58,280 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:26:58,280 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:26:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:26:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:26:58,282 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:26:58,282 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:26:58,284 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:26:58,284 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:26:58,284 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:26:58,284 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:26:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:26:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:26:58,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:26:58,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:26:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:58,311 [518] [DEBUG] [app] Starting request: urn:request:c8254c11-bb26-4c25-9251-999493221877 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:58,311 [518] [DEBUG] [app] Starting request: urn:request:c8254c11-bb26-4c25-9251-999493221877 (/_internal_ping) 2020-03-31 09:26:58,311 [518] [DEBUG] [app] Ending request: urn:request:c8254c11-bb26-4c25-9251-999493221877 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:26:58,311 [518] [DEBUG] [app] Ending request: urn:request:c8254c11-bb26-4c25-9251-999493221877 (/_internal_ping) 2020-03-31 09:26:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:26:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:26:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:26:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:26:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:26:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:26:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:26:58,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:26:58,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:26:58,325 [543] [DEBUG] [app] Starting request: urn:request:19cb967d-445a-45ab-a4cc-2993464137e5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:58,325 [543] [DEBUG] [app] Starting request: urn:request:19cb967d-445a-45ab-a4cc-2993464137e5 (/c1/_internal_ping) 2020-03-31 09:26:58,326 [543] [DEBUG] [app] Ending request: urn:request:19cb967d-445a-45ab-a4cc-2993464137e5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:26:58,326 [543] [DEBUG] [app] Ending request: urn:request:19cb967d-445a-45ab-a4cc-2993464137e5 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:26:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:26:58,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:26:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:26:58,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:26:58,328 [518] [DEBUG] [app] Ending request: urn:request:84bba98c-46b9-4e34-8db4-38c3ec0a5ec6 (/health/instance) 2020-03-31 09:26:58,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:26:58,328 [518] [DEBUG] [app] Ending request: urn:request:84bba98c-46b9-4e34-8db4-38c3ec0a5ec6 (/health/instance) 2020-03-31 09:26:58,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:26:58,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:26:58,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:26:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:26:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:26:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:26:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:26:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:51.110610+00:00 (in 51.549535 seconds) 2020-03-31 09:26:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:59 UTC)" (scheduled at 2020-03-31 09:26:59.560326+00:00) 2020-03-31 09:26:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:59,561 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:26:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:51.110610+00:00 (in 51.549535 seconds) 2020-03-31 09:26:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:59 UTC)" (scheduled at 2020-03-31 09:26:59.560326+00:00) 2020-03-31 09:26:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:26:59,561 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:27:59 UTC)" executed successfully 2020-03-31 09:27:00,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:27:00,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:30.337699+00:00 (in 29.997295 seconds) 2020-03-31 09:27:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:30 UTC)" (scheduled at 2020-03-31 09:27:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:27:00,340 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:30.337699+00:00 (in 29.997295 seconds) 2020-03-31 09:27:00,340 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:30 UTC)" (scheduled at 2020-03-31 09:27:00.337699+00:00) 2020-03-31 09:27:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 27, 0, 340996), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:27:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 27, 0, 340996), False, 50, 1, 0]) 2020-03-31 09:27:00,360 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:27:00,360 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:27:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:27:00,360 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:00,360 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:30 UTC)" executed successfully 2020-03-31 09:27:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:27:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:11.440707+00:00 (in 9.999153 seconds) 2020-03-31 09:27:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:11 UTC)" (scheduled at 2020-03-31 09:27:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:27:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:11.440707+00:00 (in 9.999153 seconds) 2020-03-31 09:27:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:11 UTC)" (scheduled at 2020-03-31 09:27:01.440707+00:00) 2020-03-31 09:27:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:27:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:27:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:27:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:27:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:27:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:11 UTC)" executed successfully 2020-03-31 09:27:03,054 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:03,054 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:06.924497+00:00 (in 3.869819 seconds) 2020-03-31 09:27:03,055 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:03 UTC)" (scheduled at 2020-03-31 09:27:03.010318+00:00) storagereplication stdout | 2020-03-31 09:27:03,054 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:03,054 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:06.924497+00:00 (in 3.869819 seconds) 2020-03-31 09:27:03,057 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:27:03,055 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:03 UTC)" (scheduled at 2020-03-31 09:27:03.010318+00:00) storagereplication stdout | 2020-03-31 09:27:03,057 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), u'imagestoragereplication/%']) 2020-03-31 09:27:03,122 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), True, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:27:03,122 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), True, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), 0, u'imagestoragereplication/%']) 2020-03-31 09:27:03,131 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), True, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:27:03,131 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), True, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 27, 3, 55882), u'imagestoragereplication/%']) 2020-03-31 09:27:03,148 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:27:03,148 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:03,149 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:03,150 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:27:03,149 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:03,150 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:03,150 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:03 UTC)" executed successfully 2020-03-31 09:27:03,151 [191] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics storagereplication stdout | 2020-03-31 09:27:03,150 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:27:03,151 [191] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:27:03,158 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:27:03,158 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:03,160 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:03,161 [191] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics storagereplication stdout | 2020-03-31 09:27:03,160 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:03,161 [191] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:03,325 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c7f0b1c8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:27:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:27:03,325 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c7f0b1c8-7331-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:27:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:27:03,325 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c7f0b1c8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cn1pWshff1apmuhS9a03Px/XNl7gCpAp7E9Wo2kYSLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:27:03 GMT'}. storagereplication stdout | 2020-03-31 09:27:03,325 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c7f0b1c8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cn1pWshff1apmuhS9a03Px/XNl7gCpAp7E9Wo2kYSLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:27:03 GMT'}. 2020-03-31 09:27:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:27:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:24.410927+00:00 (in 21.079960 seconds) 2020-03-31 09:27:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:33 UTC)" (scheduled at 2020-03-31 09:27:03.330112+00:00) 2020-03-31 09:27:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:27:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:24.410927+00:00 (in 21.079960 seconds) 2020-03-31 09:27:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:33 UTC)" (scheduled at 2020-03-31 09:27:03.330112+00:00) 2020-03-31 09:27:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 3, 331711), True, datetime.datetime(2020, 3, 31, 9, 27, 3, 331711), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:27:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 3, 331711), True, datetime.datetime(2020, 3, 31, 9, 27, 3, 331711), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:27:03,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:27:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:27:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:33 UTC)" executed successfully 2020-03-31 09:27:03,408 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:27:03,408 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:27:03,409 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:27:03 GMT, Server-Request-ID=d724283e-e01e-0058-683e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c7f0b1c8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cn1pWshff1apmuhS9a03Px/XNl7gCpAp7E9Wo2kYSLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:27:03 GMT'}. 2020-03-31 09:27:03,410 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:27:03 GMT, Server-Request-ID=d724283e-e01e-0058-683e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:27:03,410 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:27:03 GMT, Server-Request-ID=d724283e-e01e-0058-683e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:27:03,411 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:27:03,409 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:27:03 GMT, Server-Request-ID=d724283e-e01e-0058-683e-07ac3e000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c7f0b1c8-7331-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:cn1pWshff1apmuhS9a03Px/XNl7gCpAp7E9Wo2kYSLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:27:03 GMT'}. 2020-03-31 09:27:03,410 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:27:03 GMT, Server-Request-ID=d724283e-e01e-0058-683e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:27:03,410 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c7f0b1c8-7331-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:27:03 GMT, Server-Request-ID=d724283e-e01e-0058-683e-07ac3e000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:27:03,411 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:27:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:27:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:14.323352+00:00 (in 9.998940 seconds) 2020-03-31 09:27:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:14 UTC)" (scheduled at 2020-03-31 09:27:04.323352+00:00) 2020-03-31 09:27:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:27:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:14.323352+00:00 (in 9.998940 seconds) 2020-03-31 09:27:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:14 UTC)" (scheduled at 2020-03-31 09:27:04.323352+00:00) 2020-03-31 09:27:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 4, 325252), True, datetime.datetime(2020, 3, 31, 9, 27, 4, 325252), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:27:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 4, 325252), True, datetime.datetime(2020, 3, 31, 9, 27, 4, 325252), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:27:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:27:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:27:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:14 UTC)" executed successfully 2020-03-31 09:27:04,446 [527] [DEBUG] [app] Starting request: urn:request:fdc92abc-525c-46eb-ab2f-fb21b357118f (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:04,446 [527] [DEBUG] [app] Starting request: urn:request:fdc92abc-525c-46eb-ab2f-fb21b357118f (/health/instance) 2020-03-31 09:27:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:04,454 [553] [DEBUG] [app] Starting request: urn:request:a435a93b-fe3d-4fc7-b53e-ff9b207b2735 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:04,454 [553] [DEBUG] [app] Starting request: urn:request:a435a93b-fe3d-4fc7-b53e-ff9b207b2735 (/v1/_internal_ping) 2020-03-31 09:27:04,456 [553] [DEBUG] [app] Ending request: urn:request:a435a93b-fe3d-4fc7-b53e-ff9b207b2735 (/v1/_internal_ping) 2020-03-31 09:27:04,457 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:27:04,456 [553] [DEBUG] [app] Ending request: urn:request:a435a93b-fe3d-4fc7-b53e-ff9b207b2735 (/v1/_internal_ping) 2020-03-31 09:27:04,457 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:04,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:04,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:04,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:27:04,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:04,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:27:04,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:04,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:27:04,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:04,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:04,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:04,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:04,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:04,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:04,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:04,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:04,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:04,489 [527] [DEBUG] [app] Starting request: urn:request:f770e2a1-2d6c-489d-8cd9-999bb276e23b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:04,489 [527] [DEBUG] [app] Starting request: urn:request:f770e2a1-2d6c-489d-8cd9-999bb276e23b (/_internal_ping) 2020-03-31 09:27:04,491 [527] [DEBUG] [app] Ending request: urn:request:f770e2a1-2d6c-489d-8cd9-999bb276e23b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 159 0.005) gunicorn-web stdout | 2020-03-31 09:27:04,491 [527] [DEBUG] [app] Ending request: urn:request:f770e2a1-2d6c-489d-8cd9-999bb276e23b (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 159 0.005) 2020-03-31 09:27:04,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:27:04,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:04,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:27:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:27:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:27:04,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:04,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:04,509 [543] [DEBUG] [app] Starting request: urn:request:8173a82a-4bfd-4b6e-bd5a-889449e0bb3a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:04,509 [543] [DEBUG] [app] Starting request: urn:request:8173a82a-4bfd-4b6e-bd5a-889449e0bb3a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:04,510 [543] [DEBUG] [app] Ending request: urn:request:8173a82a-4bfd-4b6e-bd5a-889449e0bb3a (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:04,511 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:04,511 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:27:04,510 [543] [DEBUG] [app] Ending request: urn:request:8173a82a-4bfd-4b6e-bd5a-889449e0bb3a (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:04,511 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:04,512 [527] [DEBUG] [app] Ending request: urn:request:fdc92abc-525c-46eb-ab2f-fb21b357118f (/health/instance) 2020-03-31 09:27:04,512 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:27:04,511 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:04,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:27:04,512 [527] [DEBUG] [app] Ending request: urn:request:fdc92abc-525c-46eb-ab2f-fb21b357118f (/health/instance) 2020-03-31 09:27:04,512 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:27:04,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:27:06,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:06,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:27:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:16.924497+00:00 (in 9.998089 seconds) storagereplication stdout | 2020-03-31 09:27:06,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:06,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:27:06,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:16.924497+00:00 (in 9.998089 seconds) 2020-03-31 09:27:08,273 [518] [DEBUG] [app] Starting request: urn:request:ef035209-25d6-4299-bd99-c681728d499c (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:08,273 [518] [DEBUG] [app] Starting request: urn:request:ef035209-25d6-4299-bd99-c681728d499c (/health/instance) 2020-03-31 09:27:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:08,281 [550] [DEBUG] [app] Starting request: urn:request:f73a3aba-baa1-4dcf-b251-906e6cde234b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:08,281 [550] [DEBUG] [app] Starting request: urn:request:f73a3aba-baa1-4dcf-b251-906e6cde234b (/v1/_internal_ping) 2020-03-31 09:27:08,282 [550] [DEBUG] [app] Ending request: urn:request:f73a3aba-baa1-4dcf-b251-906e6cde234b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:08,282 [550] [DEBUG] [app] Ending request: urn:request:f73a3aba-baa1-4dcf-b251-906e6cde234b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:08,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:27:08,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:08,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:27:08,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:27:08,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:27:08,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:08,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:08,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:08,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:08,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:08,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:08,314 [518] [DEBUG] [app] Starting request: urn:request:34d267ed-156a-4790-97d5-d27f6ce2c52b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:08,314 [518] [DEBUG] [app] Starting request: urn:request:34d267ed-156a-4790-97d5-d27f6ce2c52b (/_internal_ping) 2020-03-31 09:27:08,315 [518] [DEBUG] [app] Ending request: urn:request:34d267ed-156a-4790-97d5-d27f6ce2c52b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:08,315 [518] [DEBUG] [app] Ending request: urn:request:34d267ed-156a-4790-97d5-d27f6ce2c52b (/_internal_ping) 2020-03-31 09:27:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:27:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:27:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:27:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:27:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:27:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:27:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:27:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:27:08,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:08,334 [532] [DEBUG] [app] Starting request: urn:request:a1fbbbef-e204-46e5-b77c-b92d57a39888 (/c1/_internal_ping) 2020-03-31 09:27:08,335 [532] [DEBUG] [app] Ending request: urn:request:a1fbbbef-e204-46e5-b77c-b92d57a39888 (/c1/_internal_ping) 2020-03-31 09:27:08,335 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:08,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:27:08,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:27:08,334 [532] [DEBUG] [app] Starting request: urn:request:a1fbbbef-e204-46e5-b77c-b92d57a39888 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:08,335 [532] [DEBUG] [app] Ending request: urn:request:a1fbbbef-e204-46e5-b77c-b92d57a39888 (/c1/_internal_ping) 2020-03-31 09:27:08,338 [518] [DEBUG] [app] Ending request: urn:request:ef035209-25d6-4299-bd99-c681728d499c (/health/instance) 2020-03-31 09:27:08,338 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:27:08,335 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:08,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:27:08,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:27:08,338 [518] [DEBUG] [app] Ending request: urn:request:ef035209-25d6-4299-bd99-c681728d499c (/health/instance) 2020-03-31 09:27:08,338 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:27:08,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:27:11,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:27:11,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:11,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:21 UTC)" (scheduled at 2020-03-31 09:27:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:27:11,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:21 UTC)" (scheduled at 2020-03-31 09:27:11.440707+00:00) 2020-03-31 09:27:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:21.440707+00:00 (in 9.997599 seconds) repositoryactioncounter stdout | 2020-03-31 09:27:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:21.440707+00:00 (in 9.997599 seconds) 2020-03-31 09:27:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:27:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:27:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:27:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:27:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:27:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:21 UTC)" executed successfully 2020-03-31 09:27:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:27:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:24.323352+00:00 (in 9.999231 seconds) 2020-03-31 09:27:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:24 UTC)" (scheduled at 2020-03-31 09:27:14.323352+00:00) 2020-03-31 09:27:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:27:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:24.323352+00:00 (in 9.999231 seconds) 2020-03-31 09:27:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:24 UTC)" (scheduled at 2020-03-31 09:27:14.323352+00:00) 2020-03-31 09:27:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 14, 325032), True, datetime.datetime(2020, 3, 31, 9, 27, 14, 325032), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:27:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 14, 325032), True, datetime.datetime(2020, 3, 31, 9, 27, 14, 325032), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:27:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:27:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:27:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:24 UTC)" executed successfully 2020-03-31 09:27:14,445 [518] [DEBUG] [app] Starting request: urn:request:3b1c6fdc-b86e-4a17-91ca-86d28174c1c9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:14,445 [518] [DEBUG] [app] Starting request: urn:request:3b1c6fdc-b86e-4a17-91ca-86d28174c1c9 (/health/instance) 2020-03-31 09:27:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:14,453 [555] [DEBUG] [app] Starting request: urn:request:1e1670aa-0e96-4d9c-a3a6-b1b7fb3479c0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:14,453 [555] [DEBUG] [app] Starting request: urn:request:1e1670aa-0e96-4d9c-a3a6-b1b7fb3479c0 (/v1/_internal_ping) 2020-03-31 09:27:14,455 [555] [DEBUG] [app] Ending request: urn:request:1e1670aa-0e96-4d9c-a3a6-b1b7fb3479c0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:14,455 [555] [DEBUG] [app] Ending request: urn:request:1e1670aa-0e96-4d9c-a3a6-b1b7fb3479c0 (/v1/_internal_ping) 2020-03-31 09:27:14,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:27:14,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:14,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:27:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:27:14,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:14,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:14,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:14,488 [518] [DEBUG] [app] Starting request: urn:request:a0e4fa99-9840-4f21-8f40-b662a25ac662 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:14,488 [518] [DEBUG] [app] Starting request: urn:request:a0e4fa99-9840-4f21-8f40-b662a25ac662 (/_internal_ping) 2020-03-31 09:27:14,489 [518] [DEBUG] [app] Ending request: urn:request:a0e4fa99-9840-4f21-8f40-b662a25ac662 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:27:14,489 [518] [DEBUG] [app] Ending request: urn:request:a0e4fa99-9840-4f21-8f40-b662a25ac662 (/_internal_ping) 2020-03-31 09:27:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:27:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:27:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:27:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:27:14,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:14,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:14,504 [541] [DEBUG] [app] Starting request: urn:request:756304c0-46c4-4cae-b976-5a3c859d5170 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:14,504 [541] [DEBUG] [app] Starting request: urn:request:756304c0-46c4-4cae-b976-5a3c859d5170 (/c1/_internal_ping) 2020-03-31 09:27:14,505 [541] [DEBUG] [app] Ending request: urn:request:756304c0-46c4-4cae-b976-5a3c859d5170 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:14,505 [541] [DEBUG] [app] Ending request: urn:request:756304c0-46c4-4cae-b976-5a3c859d5170 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:14,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:27:14,506 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:14,508 [518] [DEBUG] [app] Ending request: urn:request:3b1c6fdc-b86e-4a17-91ca-86d28174c1c9 (/health/instance) 2020-03-31 09:27:14,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:27:14,508 [518] [DEBUG] [app] Ending request: urn:request:3b1c6fdc-b86e-4a17-91ca-86d28174c1c9 (/health/instance) 2020-03-31 09:27:14,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:14,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:27:14,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:27:15,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:27:15,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:45.064514+00:00 (in 29.996227 seconds) 2020-03-31 09:27:15,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:45 UTC)" (scheduled at 2020-03-31 09:27:15.064514+00:00) gcworker stdout | 2020-03-31 09:27:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:45.064514+00:00 (in 29.996227 seconds) 2020-03-31 09:27:15,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:45 UTC)" (scheduled at 2020-03-31 09:27:15.064514+00:00) 2020-03-31 09:27:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437235, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:27:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437235, 1209600, 500, 1, 0]) 2020-03-31 09:27:15,099 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:27:15,099 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:27:15,099 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:15,099 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:27:15,099 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:15,099 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:45 UTC)" executed successfully 2020-03-31 09:27:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:27:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:27:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:26.924497+00:00 (in 9.999138 seconds) storagereplication stdout | 2020-03-31 09:27:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:27:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:26.924497+00:00 (in 9.999138 seconds) 2020-03-31 09:27:18,273 [518] [DEBUG] [app] Starting request: urn:request:a06e02f1-ed92-4da7-a7ce-891fa92f2db4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:18,273 [518] [DEBUG] [app] Starting request: urn:request:a06e02f1-ed92-4da7-a7ce-891fa92f2db4 (/health/instance) 2020-03-31 09:27:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:18,278 [546] [DEBUG] [app] Starting request: urn:request:074d2217-c6a0-4f68-804c-64de82b6c697 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:18,278 [546] [DEBUG] [app] Starting request: urn:request:074d2217-c6a0-4f68-804c-64de82b6c697 (/v1/_internal_ping) 2020-03-31 09:27:18,279 [546] [DEBUG] [app] Ending request: urn:request:074d2217-c6a0-4f68-804c-64de82b6c697 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:18,279 [546] [DEBUG] [app] Ending request: urn:request:074d2217-c6a0-4f68-804c-64de82b6c697 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:18,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:27:18,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:18,282 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:27:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:27:18,282 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:18,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:18,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:18,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:18,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:18,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:18,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:18,319 [518] [DEBUG] [app] Starting request: urn:request:2f0d7707-970d-47cd-8fde-0e1713065d68 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:18,319 [518] [DEBUG] [app] Starting request: urn:request:2f0d7707-970d-47cd-8fde-0e1713065d68 (/_internal_ping) 2020-03-31 09:27:18,320 [518] [DEBUG] [app] Ending request: urn:request:2f0d7707-970d-47cd-8fde-0e1713065d68 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:18,320 [518] [DEBUG] [app] Ending request: urn:request:2f0d7707-970d-47cd-8fde-0e1713065d68 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:27:18,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:27:18,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:27:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:27:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:27:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:18,333 [541] [DEBUG] [app] Starting request: urn:request:6bfbf338-8adb-447f-8b5e-28b97c0013db (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:18,333 [541] [DEBUG] [app] Starting request: urn:request:6bfbf338-8adb-447f-8b5e-28b97c0013db (/c1/_internal_ping) 2020-03-31 09:27:18,334 [541] [DEBUG] [app] Ending request: urn:request:6bfbf338-8adb-447f-8b5e-28b97c0013db (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:18,334 [541] [DEBUG] [app] Ending request: urn:request:6bfbf338-8adb-447f-8b5e-28b97c0013db (/c1/_internal_ping) 2020-03-31 09:27:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:18,334 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:18,336 [518] [DEBUG] [app] Ending request: urn:request:a06e02f1-ed92-4da7-a7ce-891fa92f2db4 (/health/instance) 2020-03-31 09:27:18,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:27:18,334 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:27:18,336 [518] [DEBUG] [app] Ending request: urn:request:a06e02f1-ed92-4da7-a7ce-891fa92f2db4 (/health/instance) 2020-03-31 09:27:18,336 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:27:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:27:19,415 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:27:19,415 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy 2020-03-31 09:27:19,415 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:27:19,415 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:27:19,415 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:27:19,415 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:27:19,428 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) storagereplication stdout | 2020-03-31 09:27:19,428 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) 2020-03-31 09:27:19,433 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 32, 19, 427414), True, u'10381c13-b992-46bd-8bf9-d9ab86951c36', 1]) storagereplication stdout | 2020-03-31 09:27:19,433 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 32, 19, 427414), True, u'10381c13-b992-46bd-8bf9-d9ab86951c36', 1]) 2020-03-31 09:27:19,439 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 19, 437930), True, datetime.datetime(2020, 3, 31, 9, 27, 19, 437930), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:27:19,439 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 19, 437930), True, datetime.datetime(2020, 3, 31, 9, 27, 19, 437930), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:27:19,446 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:27:19,446 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:19,446 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:19,447 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:27:19,446 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:19,447 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:26 UTC)" executed successfully 2020-03-31 09:27:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:31.440707+00:00 (in 9.996401 seconds) 2020-03-31 09:27:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:31 UTC)" (scheduled at 2020-03-31 09:27:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:27:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:27:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:31.440707+00:00 (in 9.996401 seconds) 2020-03-31 09:27:21,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:31 UTC)" (scheduled at 2020-03-31 09:27:21.440707+00:00) 2020-03-31 09:27:21,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:27:21,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:27:21,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:27:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:27:21,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:27:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:31 UTC)" executed successfully 2020-03-31 09:27:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:52.113221+00:00 (in 29.999283 seconds) 2020-03-31 09:27:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:52 UTC)" (scheduled at 2020-03-31 09:27:22.113221+00:00) 2020-03-31 09:27:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:27:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:27:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:52.113221+00:00 (in 29.999283 seconds) 2020-03-31 09:27:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:52 UTC)" (scheduled at 2020-03-31 09:27:22.113221+00:00) securityworker stdout | 2020-03-31 09:27:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:27:22,141 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:27:22,141 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:27:22,144 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:27:22,144 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:27:22,145 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:22,145 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:27:22,145 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:22,145 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:27:52 UTC)" executed successfully 2020-03-31 09:27:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:27:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:31.498397+00:00 (in 7.174140 seconds) 2020-03-31 09:27:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:34 UTC)" (scheduled at 2020-03-31 09:27:24.323352+00:00) 2020-03-31 09:27:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:27:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:31.498397+00:00 (in 7.174140 seconds) 2020-03-31 09:27:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:34 UTC)" (scheduled at 2020-03-31 09:27:24.323352+00:00) 2020-03-31 09:27:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 24, 325024), True, datetime.datetime(2020, 3, 31, 9, 27, 24, 325024), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:27:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 24, 325024), True, datetime.datetime(2020, 3, 31, 9, 27, 24, 325024), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:27:24,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:27:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:27:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:34 UTC)" executed successfully 2020-03-31 09:27:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:27:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:33.330112+00:00 (in 8.918411 seconds) 2020-03-31 09:27:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:24 UTC)" (scheduled at 2020-03-31 09:27:24.410927+00:00) security_notification_worker stdout | 2020-03-31 09:27:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:33.330112+00:00 (in 8.918411 seconds) 2020-03-31 09:27:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:24 UTC)" (scheduled at 2020-03-31 09:27:24.410927+00:00) 2020-03-31 09:27:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:27:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:24 UTC)" executed successfully 2020-03-31 09:27:24,447 [518] [DEBUG] [app] Starting request: urn:request:5b5218ee-2a6c-4c34-887e-8eb1e247b860 (/health/instance) 2020-03-31 09:27:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:24,447 [518] [DEBUG] [app] Starting request: urn:request:5b5218ee-2a6c-4c34-887e-8eb1e247b860 (/health/instance) 2020-03-31 09:27:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:24,455 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:24,455 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:24,457 [550] [DEBUG] [app] Starting request: urn:request:c5f79dce-8851-468f-bde5-434009f0aadb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:24,457 [550] [DEBUG] [app] Starting request: urn:request:c5f79dce-8851-468f-bde5-434009f0aadb (/v1/_internal_ping) 2020-03-31 09:27:24,458 [550] [DEBUG] [app] Ending request: urn:request:c5f79dce-8851-468f-bde5-434009f0aadb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:24,458 [550] [DEBUG] [app] Ending request: urn:request:c5f79dce-8851-468f-bde5-434009f0aadb (/v1/_internal_ping) 2020-03-31 09:27:24,459 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:27:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:27:24,459 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:24,461 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:27:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:27:24,463 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:24,461 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:24,464 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:24,463 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:24,464 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:24,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:24,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:24,493 [527] [DEBUG] [app] Starting request: urn:request:9fef63c1-a3b1-4e71-a875-97891509652e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:24,493 [527] [DEBUG] [app] Starting request: urn:request:9fef63c1-a3b1-4e71-a875-97891509652e (/_internal_ping) 2020-03-31 09:27:24,494 [527] [DEBUG] [app] Ending request: urn:request:9fef63c1-a3b1-4e71-a875-97891509652e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:24,494 [527] [DEBUG] [app] Ending request: urn:request:9fef63c1-a3b1-4e71-a875-97891509652e (/_internal_ping) 2020-03-31 09:27:24,495 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:27:24,495 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:27:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.004 153 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.004 153 0.004) 2020-03-31 09:27:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:27:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:27:24,514 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:24,514 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:24,515 [536] [DEBUG] [app] Starting request: urn:request:4b3ac777-0cdb-426f-a30f-c38e0cd5fa7e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:24,515 [536] [DEBUG] [app] Starting request: urn:request:4b3ac777-0cdb-426f-a30f-c38e0cd5fa7e (/c1/_internal_ping) 2020-03-31 09:27:24,517 [536] [DEBUG] [app] Ending request: urn:request:4b3ac777-0cdb-426f-a30f-c38e0cd5fa7e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:24,517 [536] [DEBUG] [app] Ending request: urn:request:4b3ac777-0cdb-426f-a30f-c38e0cd5fa7e (/c1/_internal_ping) 2020-03-31 09:27:24,518 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:24,517 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:24,518 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:27:24,517 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:24,520 [518] [DEBUG] [app] Ending request: urn:request:5b5218ee-2a6c-4c34-887e-8eb1e247b860 (/health/instance) 2020-03-31 09:27:24,520 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:27:24,520 [518] [DEBUG] [app] Ending request: urn:request:5b5218ee-2a6c-4c34-887e-8eb1e247b860 (/health/instance) 2020-03-31 09:27:24,520 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:24,520 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.076 49 0.076) gunicorn-web stdout | 2020-03-31 09:27:24,520 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.076 49 0.076) 2020-03-31 09:27:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:36.924497+00:00 (in 9.999065 seconds) 2020-03-31 09:27:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:36 UTC)" (scheduled at 2020-03-31 09:27:26.924497+00:00) 2020-03-31 09:27:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 26, 926283), True, datetime.datetime(2020, 3, 31, 9, 27, 26, 926283), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:27:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:27:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:36.924497+00:00 (in 9.999065 seconds) 2020-03-31 09:27:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:36 UTC)" (scheduled at 2020-03-31 09:27:26.924497+00:00) 2020-03-31 09:27:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:27:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 26, 926283), True, datetime.datetime(2020, 3, 31, 9, 27, 26, 926283), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:27:26,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:27:26,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:27:26,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:26,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:36 UTC)" executed successfully 2020-03-31 09:27:28,273 [518] [DEBUG] [app] Starting request: urn:request:a8306ec1-542f-4c02-85b1-a487a4c1b652 (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:28,273 [518] [DEBUG] [app] Starting request: urn:request:a8306ec1-542f-4c02-85b1-a487a4c1b652 (/health/instance) 2020-03-31 09:27:28,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:28,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:28,279 [545] [DEBUG] [app] Starting request: urn:request:bbefdaad-48a8-4ff3-b058-136c1f5dd1fe (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:28,279 [545] [DEBUG] [app] Starting request: urn:request:bbefdaad-48a8-4ff3-b058-136c1f5dd1fe (/v1/_internal_ping) 2020-03-31 09:27:28,280 [545] [DEBUG] [app] Ending request: urn:request:bbefdaad-48a8-4ff3-b058-136c1f5dd1fe (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:27:28,280 [545] [DEBUG] [app] Ending request: urn:request:bbefdaad-48a8-4ff3-b058-136c1f5dd1fe (/v1/_internal_ping) 2020-03-31 09:27:28,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:27:28,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:27:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:28,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:28,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:27:28,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:27:28,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:28,288 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:28,288 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:28,289 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:28,289 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:28,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:28,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:28,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:28,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:28,319 [518] [DEBUG] [app] Starting request: urn:request:75317d41-615b-4e91-9593-3d47a6ee3a2c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:28,319 [518] [DEBUG] [app] Starting request: urn:request:75317d41-615b-4e91-9593-3d47a6ee3a2c (/_internal_ping) 2020-03-31 09:27:28,321 [518] [DEBUG] [app] Ending request: urn:request:75317d41-615b-4e91-9593-3d47a6ee3a2c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:28,321 [518] [DEBUG] [app] Ending request: urn:request:75317d41-615b-4e91-9593-3d47a6ee3a2c (/_internal_ping) 2020-03-31 09:27:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:27:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:27:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:28,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:27:28,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:27:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:28,337 [532] [DEBUG] [app] Starting request: urn:request:9900eabd-a3ad-420b-8a5c-88e984124633 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:28,337 [532] [DEBUG] [app] Starting request: urn:request:9900eabd-a3ad-420b-8a5c-88e984124633 (/c1/_internal_ping) 2020-03-31 09:27:28,338 [532] [DEBUG] [app] Ending request: urn:request:9900eabd-a3ad-420b-8a5c-88e984124633 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:28,338 [532] [DEBUG] [app] Ending request: urn:request:9900eabd-a3ad-420b-8a5c-88e984124633 (/c1/_internal_ping) 2020-03-31 09:27:28,339 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:28,338 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:28,339 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:28,340 [518] [DEBUG] [app] Ending request: urn:request:a8306ec1-542f-4c02-85b1-a487a4c1b652 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:27:28,338 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:28,340 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:28,341 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:27:28,340 [518] [DEBUG] [app] Ending request: urn:request:a8306ec1-542f-4c02-85b1-a487a4c1b652 (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:28,340 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:28,341 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:27:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:27:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:00.337699+00:00 (in 29.999177 seconds) 2020-03-31 09:27:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:00 UTC)" (scheduled at 2020-03-31 09:27:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:27:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:00.337699+00:00 (in 29.999177 seconds) 2020-03-31 09:27:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:00 UTC)" (scheduled at 2020-03-31 09:27:30.337699+00:00) 2020-03-31 09:27:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 27, 30, 339406), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:27:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 27, 30, 339406), False, 50, 1, 0]) 2020-03-31 09:27:30,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:27:30,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:27:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:27:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:00 UTC)" executed successfully 2020-03-31 09:27:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:27:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:41.440707+00:00 (in 9.999184 seconds) 2020-03-31 09:27:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:41 UTC)" (scheduled at 2020-03-31 09:27:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:27:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:41.440707+00:00 (in 9.999184 seconds) 2020-03-31 09:27:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:41 UTC)" (scheduled at 2020-03-31 09:27:31.440707+00:00) 2020-03-31 09:27:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:27:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:27:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:27:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:27:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:27:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:41 UTC)" executed successfully 2020-03-31 09:27:31,503 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:31,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:34.323352+00:00 (in 2.819804 seconds) 2020-03-31 09:27:31,504 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:31 UTC)" (scheduled at 2020-03-31 09:27:31.498397+00:00) 2020-03-31 09:27:31,505 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), u'notification/%']) notificationworker stdout | 2020-03-31 09:27:31,503 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:31,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:34.323352+00:00 (in 2.819804 seconds) 2020-03-31 09:27:31,504 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:31 UTC)" (scheduled at 2020-03-31 09:27:31.498397+00:00) 2020-03-31 09:27:31,505 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), u'notification/%']) 2020-03-31 09:27:31,529 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), True, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:27:31,529 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), True, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), 0, u'notification/%']) 2020-03-31 09:27:31,534 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), True, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), u'notification/%']) notificationworker stdout | 2020-03-31 09:27:31,534 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), True, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 27, 31, 504237), u'notification/%']) 2020-03-31 09:27:31,539 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:27:31,539 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:31,542 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:27:31,542 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:31,543 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:31 UTC)" executed successfully 2020-03-31 09:27:31,543 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:27:31,543 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:31 UTC)" executed successfully 2020-03-31 09:27:31,543 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:31,544 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 09:27:31,544 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:27:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:27:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:03.330112+00:00 (in 29.999166 seconds) 2020-03-31 09:27:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:03 UTC)" (scheduled at 2020-03-31 09:27:33.330112+00:00) 2020-03-31 09:27:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:27:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:03.330112+00:00 (in 29.999166 seconds) 2020-03-31 09:27:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:03 UTC)" (scheduled at 2020-03-31 09:27:33.330112+00:00) 2020-03-31 09:27:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 33, 331793), True, datetime.datetime(2020, 3, 31, 9, 27, 33, 331793), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:27:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 33, 331793), True, datetime.datetime(2020, 3, 31, 9, 27, 33, 331793), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:27:33,355 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:27:33,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:33,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:33,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:27:33,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:33,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:03 UTC)" executed successfully 2020-03-31 09:27:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:27:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:33,369 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:43.539999+00:00 (in 10.171198 seconds) exportactionlogsworker stdout | 2020-03-31 09:27:33,369 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:43.539999+00:00 (in 10.171198 seconds) 2020-03-31 09:27:33,369 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:33 UTC)" (scheduled at 2020-03-31 09:27:33.366791+00:00) 2020-03-31 09:27:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:27:33,369 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:33 UTC)" (scheduled at 2020-03-31 09:27:33.366791+00:00) 2020-03-31 09:27:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 33, 369639), True, datetime.datetime(2020, 3, 31, 9, 27, 33, 369639), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:27:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 33, 369639), True, datetime.datetime(2020, 3, 31, 9, 27, 33, 369639), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:27:33,396 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:27:33,396 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:33,396 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:33,396 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:27:33,396 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:33,396 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:33 UTC)" executed successfully 2020-03-31 09:27:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:27:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:44.323352+00:00 (in 9.999110 seconds) 2020-03-31 09:27:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:44 UTC)" (scheduled at 2020-03-31 09:27:34.323352+00:00) 2020-03-31 09:27:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:27:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:44.323352+00:00 (in 9.999110 seconds) 2020-03-31 09:27:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:44 UTC)" (scheduled at 2020-03-31 09:27:34.323352+00:00) 2020-03-31 09:27:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 34, 325369), True, datetime.datetime(2020, 3, 31, 9, 27, 34, 325369), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:27:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 34, 325369), True, datetime.datetime(2020, 3, 31, 9, 27, 34, 325369), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:27:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:27:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:27:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:44 UTC)" executed successfully 2020-03-31 09:27:34,446 [527] [DEBUG] [app] Starting request: urn:request:c7c8055a-66df-405f-9f63-065834240dfc (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:34,446 [527] [DEBUG] [app] Starting request: urn:request:c7c8055a-66df-405f-9f63-065834240dfc (/health/instance) 2020-03-31 09:27:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:34,453 [552] [DEBUG] [app] Starting request: urn:request:93a35e4f-89e3-41a7-b1e5-3cf81d33a5de (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:34,453 [552] [DEBUG] [app] Starting request: urn:request:93a35e4f-89e3-41a7-b1e5-3cf81d33a5de (/v1/_internal_ping) 2020-03-31 09:27:34,458 [552] [DEBUG] [app] Ending request: urn:request:93a35e4f-89e3-41a7-b1e5-3cf81d33a5de (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:34,458 [552] [DEBUG] [app] Ending request: urn:request:93a35e4f-89e3-41a7-b1e5-3cf81d33a5de (/v1/_internal_ping) 2020-03-31 09:27:34,459 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.007 162 0.007) gunicorn-registry stdout | 2020-03-31 09:27:34,459 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:34,459 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.007 162 0.007) 2020-03-31 09:27:34,460 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:34,459 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:34,461 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:27:34,460 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:27:34,461 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:34,463 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:34,463 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:34,465 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:34,465 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:34,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:34,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:34,492 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:34,492 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:34,497 [518] [DEBUG] [app] Starting request: urn:request:e460ee88-2fe8-48be-84b5-b57126c5be35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:34,497 [518] [DEBUG] [app] Starting request: urn:request:e460ee88-2fe8-48be-84b5-b57126c5be35 (/_internal_ping) 2020-03-31 09:27:34,498 [518] [DEBUG] [app] Ending request: urn:request:e460ee88-2fe8-48be-84b5-b57126c5be35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:34,498 [518] [DEBUG] [app] Ending request: urn:request:e460ee88-2fe8-48be-84b5-b57126c5be35 (/_internal_ping) 2020-03-31 09:27:34,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:27:34,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:27:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:34,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:34,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:34,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:34,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:27:34,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:27:34,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:34,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:34,514 [543] [DEBUG] [app] Starting request: urn:request:babb0c44-e4f7-4d4d-b42f-711686f5e07c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:34,514 [543] [DEBUG] [app] Starting request: urn:request:babb0c44-e4f7-4d4d-b42f-711686f5e07c (/c1/_internal_ping) 2020-03-31 09:27:34,515 [543] [DEBUG] [app] Ending request: urn:request:babb0c44-e4f7-4d4d-b42f-711686f5e07c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:34,515 [543] [DEBUG] [app] Ending request: urn:request:babb0c44-e4f7-4d4d-b42f-711686f5e07c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:34,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:34,515 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:34,515 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:27:34,515 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:34,517 [527] [DEBUG] [app] Ending request: urn:request:c7c8055a-66df-405f-9f63-065834240dfc (/health/instance) 2020-03-31 09:27:34,517 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:27:34,517 [527] [DEBUG] [app] Ending request: urn:request:c7c8055a-66df-405f-9f63-065834240dfc (/health/instance) 2020-03-31 09:27:34,517 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:34,517 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:27:34,517 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:27:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:27:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:38.010934+00:00 (in 1.085602 seconds) 2020-03-31 09:27:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:46 UTC)" (scheduled at 2020-03-31 09:27:36.924497+00:00) 2020-03-31 09:27:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:27:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:38.010934+00:00 (in 1.085602 seconds) 2020-03-31 09:27:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:46 UTC)" (scheduled at 2020-03-31 09:27:36.924497+00:00) 2020-03-31 09:27:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 36, 926102), True, datetime.datetime(2020, 3, 31, 9, 27, 36, 926102), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:27:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 36, 926102), True, datetime.datetime(2020, 3, 31, 9, 27, 36, 926102), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:27:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:27:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:27:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:46 UTC)" executed successfully 2020-03-31 09:27:38,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:38,013 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:46.924497+00:00 (in 8.910965 seconds) 2020-03-31 09:27:38,021 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" (scheduled at 2020-03-31 09:27:38.010934+00:00) 2020-03-31 09:27:38,021 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:38,021 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:27:38,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:38,013 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:46.924497+00:00 (in 8.910965 seconds) 2020-03-31 09:27:38,021 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" (scheduled at 2020-03-31 09:27:38.010934+00:00) 2020-03-31 09:27:38,021 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:38,021 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" executed successfully 2020-03-31 09:27:38,331 [518] [DEBUG] [app] Starting request: urn:request:205485a5-5ee3-4dce-8462-a6e7d19c3fcc (/health/instance) 2020-03-31 09:27:38,334 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:38,331 [518] [DEBUG] [app] Starting request: urn:request:205485a5-5ee3-4dce-8462-a6e7d19c3fcc (/health/instance) 2020-03-31 09:27:38,334 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:38,375 [555] [DEBUG] [app] Starting request: urn:request:40366d56-c18b-45f4-a2fa-e7fe8310fe1c (/v1/_internal_ping) 2020-03-31 09:27:38,377 [555] [DEBUG] [app] Ending request: urn:request:40366d56-c18b-45f4-a2fa-e7fe8310fe1c (/v1/_internal_ping) 2020-03-31 09:27:38,378 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:38,373 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:38,378 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:38,379 [518] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:27:38,384 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:38,387 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:38,388 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:38,391 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-registry stdout | 2020-03-31 09:27:38,375 [555] [DEBUG] [app] Starting request: urn:request:40366d56-c18b-45f4-a2fa-e7fe8310fe1c (/v1/_internal_ping) 2020-03-31 09:27:38,377 [555] [DEBUG] [app] Ending request: urn:request:40366d56-c18b-45f4-a2fa-e7fe8310fe1c (/v1/_internal_ping) 2020-03-31 09:27:38,378 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:38,373 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:38,378 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:38,379 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:27:38,384 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:27:38,387 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:38,388 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:27:38,391 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:38,405 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:38,405 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:38,417 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:38,419 [521] [DEBUG] [app] Starting request: urn:request:d669711e-c001-42b5-90db-9ba337b0e165 (/_internal_ping) 2020-03-31 09:27:38,420 [521] [DEBUG] [app] Ending request: urn:request:d669711e-c001-42b5-90db-9ba337b0e165 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:38,417 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:38,421 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:38,421 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:38,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:38,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:38,529 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:38,419 [521] [DEBUG] [app] Starting request: urn:request:d669711e-c001-42b5-90db-9ba337b0e165 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.002 153 0.002) gunicorn-web stdout | 2020-03-31 09:27:38,420 [521] [DEBUG] [app] Ending request: urn:request:d669711e-c001-42b5-90db-9ba337b0e165 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:38,421 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:38,421 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:38,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:38,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:27:38,535 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:38,529 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.002 153 0.002) gunicorn-web stdout | 2020-03-31 09:27:38,535 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:27:38,543 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:38,543 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:38,547 [536] [DEBUG] [app] Starting request: urn:request:44042d68-7b83-4420-beb8-34b21ca71d11 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:38,547 [536] [DEBUG] [app] Starting request: urn:request:44042d68-7b83-4420-beb8-34b21ca71d11 (/c1/_internal_ping) 2020-03-31 09:27:38,550 [536] [DEBUG] [app] Ending request: urn:request:44042d68-7b83-4420-beb8-34b21ca71d11 (/c1/_internal_ping) 2020-03-31 09:27:38,550 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:27:38,550 [536] [DEBUG] [app] Ending request: urn:request:44042d68-7b83-4420-beb8-34b21ca71d11 (/c1/_internal_ping) 2020-03-31 09:27:38,550 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:38,552 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-web stdout | 2020-03-31 09:27:38,552 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) 2020-03-31 09:27:38,557 [518] [DEBUG] [app] Ending request: urn:request:205485a5-5ee3-4dce-8462-a6e7d19c3fcc (/health/instance) 2020-03-31 09:27:38,558 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:27:38,557 [518] [DEBUG] [app] Ending request: urn:request:205485a5-5ee3-4dce-8462-a6e7d19c3fcc (/health/instance) 2020-03-31 09:27:38,558 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:38,560 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.231 49 0.231) gunicorn-web stdout | 2020-03-31 09:27:38,560 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.231 49 0.231) 2020-03-31 09:27:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:27:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:51.440707+00:00 (in 9.998979 seconds) 2020-03-31 09:27:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:51 UTC)" (scheduled at 2020-03-31 09:27:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:27:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:51.440707+00:00 (in 9.998979 seconds) 2020-03-31 09:27:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:51 UTC)" (scheduled at 2020-03-31 09:27:41.440707+00:00) 2020-03-31 09:27:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:27:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:27:41,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:27:41,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:27:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:27:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:51 UTC)" executed successfully 2020-03-31 09:27:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:27:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:43 UTC)" (scheduled at 2020-03-31 09:27:43.539999+00:00) 2020-03-31 09:27:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:33.366791+00:00 (in 49.825792 seconds) 2020-03-31 09:27:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:27:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:43 UTC)" (scheduled at 2020-03-31 09:27:43.539999+00:00) 2020-03-31 09:27:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:33.366791+00:00 (in 49.825792 seconds) 2020-03-31 09:27:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:43 UTC)" executed successfully 2020-03-31 09:27:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:49.498966+00:00 (in 5.173474 seconds) 2020-03-31 09:27:44,329 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:54 UTC)" (scheduled at 2020-03-31 09:27:44.323352+00:00) 2020-03-31 09:27:44,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:27:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:49.498966+00:00 (in 5.173474 seconds) 2020-03-31 09:27:44,329 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:54 UTC)" (scheduled at 2020-03-31 09:27:44.323352+00:00) 2020-03-31 09:27:44,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:44,422 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 44, 329737), True, datetime.datetime(2020, 3, 31, 9, 27, 44, 329737), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:27:44,422 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 44, 329737), True, datetime.datetime(2020, 3, 31, 9, 27, 44, 329737), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:27:44,481 [548] [DEBUG] [app] Starting request: urn:request:b3893e07-48e0-437e-a47b-f823e6f607a3 (/v1/_internal_ping) 2020-03-31 09:27:44,470 [525] [DEBUG] [app] Starting request: urn:request:a46a34ad-cea4-461f-8ac9-9d9c48ae7064 (/health/instance) 2020-03-31 09:27:44,473 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:44,479 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-registry stdout | 2020-03-31 09:27:44,481 [548] [DEBUG] [app] Starting request: urn:request:b3893e07-48e0-437e-a47b-f823e6f607a3 (/v1/_internal_ping) 2020-03-31 09:27:44,484 [548] [DEBUG] [app] Ending request: urn:request:b3893e07-48e0-437e-a47b-f823e6f607a3 (/v1/_internal_ping) 2020-03-31 09:27:44,485 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:44,490 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:44,493 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:44,494 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:27:44,470 [525] [DEBUG] [app] Starting request: urn:request:a46a34ad-cea4-461f-8ac9-9d9c48ae7064 (/health/instance) 2020-03-31 09:27:44,473 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:44,479 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:44,496 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:44,498 [525] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.016 162 0.016) gunicorn-registry stdout | 2020-03-31 09:27:44,484 [548] [DEBUG] [app] Ending request: urn:request:b3893e07-48e0-437e-a47b-f823e6f607a3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:44,485 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:27:44,490 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:27:44,493 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:44,494 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:27:44,496 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:44,498 [525] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.016 162 0.016) 2020-03-31 09:27:44,511 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:44,511 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:44,537 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:44,542 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:44,544 [518] [DEBUG] [app] Starting request: urn:request:6a7a6a6b-ede7-417e-ad79-05dc9d00ca2c (/_internal_ping) 2020-03-31 09:27:44,545 [518] [DEBUG] [app] Ending request: urn:request:6a7a6a6b-ede7-417e-ad79-05dc9d00ca2c (/_internal_ping) 2020-03-31 09:27:44,546 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:27:44,537 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:44,542 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:44,544 [518] [DEBUG] [app] Starting request: urn:request:6a7a6a6b-ede7-417e-ad79-05dc9d00ca2c (/_internal_ping) 2020-03-31 09:27:44,545 [518] [DEBUG] [app] Ending request: urn:request:6a7a6a6b-ede7-417e-ad79-05dc9d00ca2c (/_internal_ping) 2020-03-31 09:27:44,546 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:44,547 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:27:44,547 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:44,555 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:44,555 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:44,556 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:44,556 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:44,562 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:44,562 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:44,563 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:27:44,563 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:27:44,567 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:44,567 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:44,570 [536] [DEBUG] [app] Starting request: urn:request:d7c76847-54c5-414e-b1f5-ec26d839ca80 (/c1/_internal_ping) 2020-03-31 09:27:44,572 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:44,572 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:44,572 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:54 UTC)" executed successfully 2020-03-31 09:27:44,572 [536] [DEBUG] [app] Ending request: urn:request:d7c76847-54c5-414e-b1f5-ec26d839ca80 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:44,570 [536] [DEBUG] [app] Starting request: urn:request:d7c76847-54c5-414e-b1f5-ec26d839ca80 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:27:44,572 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" notificationworker stdout | 2020-03-31 09:27:44,572 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:44,572 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:44,572 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:54 UTC)" executed successfully gunicorn-verbs stdout | 2020-03-31 09:27:44,572 [536] [DEBUG] [app] Ending request: urn:request:d7c76847-54c5-414e-b1f5-ec26d839ca80 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-verbs stdout | 2020-03-31 09:27:44,572 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:44,577 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:27:44,577 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:44,578 [525] [DEBUG] [app] Ending request: urn:request:a46a34ad-cea4-461f-8ac9-9d9c48ae7064 (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:44,578 [525] [DEBUG] [app] Ending request: urn:request:a46a34ad-cea4-461f-8ac9-9d9c48ae7064 (/health/instance) 2020-03-31 09:27:44,578 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:27:44,578 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:44,579 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.111 49 0.111) gunicorn-web stdout | 2020-03-31 09:27:44,579 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.111 49 0.111) 2020-03-31 09:27:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:27:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:15.064514+00:00 (in 29.999165 seconds) 2020-03-31 09:27:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:15 UTC)" (scheduled at 2020-03-31 09:27:45.064514+00:00) gcworker stdout | 2020-03-31 09:27:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:15.064514+00:00 (in 29.999165 seconds) 2020-03-31 09:27:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:15 UTC)" (scheduled at 2020-03-31 09:27:45.064514+00:00) 2020-03-31 09:27:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437265, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:27:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437265, 1209600, 500, 1, 0]) 2020-03-31 09:27:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:27:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:27:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:27:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:45,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:15 UTC)" executed successfully 2020-03-31 09:27:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:56.924497+00:00 (in 9.999108 seconds) 2020-03-31 09:27:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:56 UTC)" (scheduled at 2020-03-31 09:27:46.924497+00:00) 2020-03-31 09:27:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:46,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 46, 926699), True, datetime.datetime(2020, 3, 31, 9, 27, 46, 926699), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:27:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:56.924497+00:00 (in 9.999108 seconds) 2020-03-31 09:27:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:56 UTC)" (scheduled at 2020-03-31 09:27:46.924497+00:00) 2020-03-31 09:27:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:46,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 46, 926699), True, datetime.datetime(2020, 3, 31, 9, 27, 46, 926699), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:27:46,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:27:46,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:46,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:46,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:27:46,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:46,951 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:27:56 UTC)" executed successfully 2020-03-31 09:27:48,273 [526] [DEBUG] [app] Starting request: urn:request:fa7441c5-b4ca-41fc-953e-324626fd0f5a (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:48,273 [526] [DEBUG] [app] Starting request: urn:request:fa7441c5-b4ca-41fc-953e-324626fd0f5a (/health/instance) 2020-03-31 09:27:48,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:48,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:48,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:48,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:48,282 [551] [DEBUG] [app] Starting request: urn:request:025de1a1-1a94-44d8-b9f6-d154d03ddbdb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:48,282 [551] [DEBUG] [app] Starting request: urn:request:025de1a1-1a94-44d8-b9f6-d154d03ddbdb (/v1/_internal_ping) 2020-03-31 09:27:48,283 [551] [DEBUG] [app] Ending request: urn:request:025de1a1-1a94-44d8-b9f6-d154d03ddbdb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:48,283 [551] [DEBUG] [app] Ending request: urn:request:025de1a1-1a94-44d8-b9f6-d154d03ddbdb (/v1/_internal_ping) 2020-03-31 09:27:48,284 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:48,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:27:48,284 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:48,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:48,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:48,287 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:27:48,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:27:48,287 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:48,289 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:48,289 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:48,289 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:48,289 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:48,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:48,299 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:48,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:48,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:48,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:48,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:48,321 [526] [DEBUG] [app] Starting request: urn:request:703089fc-fa15-42b6-a603-663e1bd87566 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:48,321 [526] [DEBUG] [app] Starting request: urn:request:703089fc-fa15-42b6-a603-663e1bd87566 (/_internal_ping) 2020-03-31 09:27:48,322 [526] [DEBUG] [app] Ending request: urn:request:703089fc-fa15-42b6-a603-663e1bd87566 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:48,322 [526] [DEBUG] [app] Ending request: urn:request:703089fc-fa15-42b6-a603-663e1bd87566 (/_internal_ping) 2020-03-31 09:27:48,322 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:48,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:27:48,322 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:48,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:27:48,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:48,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:48,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:48,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:48,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:48,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:27:48,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:27:48,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:27:48,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:48,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:48,334 [541] [DEBUG] [app] Starting request: urn:request:550f347e-1692-4cda-94e1-bf75049cc4ab (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:48,334 [541] [DEBUG] [app] Starting request: urn:request:550f347e-1692-4cda-94e1-bf75049cc4ab (/c1/_internal_ping) 2020-03-31 09:27:48,334 [541] [DEBUG] [app] Ending request: urn:request:550f347e-1692-4cda-94e1-bf75049cc4ab (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:48,334 [541] [DEBUG] [app] Ending request: urn:request:550f347e-1692-4cda-94e1-bf75049cc4ab (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:48,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:27:48,335 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:27:48,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:48,336 [526] [DEBUG] [app] Ending request: urn:request:fa7441c5-b4ca-41fc-953e-324626fd0f5a (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:48,335 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:48,336 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:48,337 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:27:48,336 [526] [DEBUG] [app] Ending request: urn:request:fa7441c5-b4ca-41fc-953e-324626fd0f5a (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:48,336 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:48,337 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:27:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:27:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:49,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:54.323352+00:00 (in 4.823729 seconds) 2020-03-31 09:27:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:49 UTC)" (scheduled at 2020-03-31 09:27:49.498966+00:00) 2020-03-31 09:27:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:27:49,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:54.323352+00:00 (in 4.823729 seconds) 2020-03-31 09:27:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:49 UTC)" (scheduled at 2020-03-31 09:27:49.498966+00:00) 2020-03-31 09:27:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:27:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:49 UTC)" executed successfully 2020-03-31 09:27:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:27:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:59.560326+00:00 (in 8.448778 seconds) 2020-03-31 09:27:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:51 UTC)" (scheduled at 2020-03-31 09:27:51.110610+00:00) 2020-03-31 09:27:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:27:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:27:59.560326+00:00 (in 8.448778 seconds) 2020-03-31 09:27:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:51 UTC)" (scheduled at 2020-03-31 09:27:51.110610+00:00) 2020-03-31 09:27:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 51, 112543), True, datetime.datetime(2020, 3, 31, 9, 27, 51, 112543), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:27:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 51, 112543), True, datetime.datetime(2020, 3, 31, 9, 27, 51, 112543), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:27:51,132 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:27:51,132 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:27:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:51 UTC)" executed successfully 2020-03-31 09:27:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:27:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:01.440707+00:00 (in 9.998982 seconds) 2020-03-31 09:27:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:01 UTC)" (scheduled at 2020-03-31 09:27:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:27:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:01.440707+00:00 (in 9.998982 seconds) 2020-03-31 09:27:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:01 UTC)" (scheduled at 2020-03-31 09:27:51.440707+00:00) 2020-03-31 09:27:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:27:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:27:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:27:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:27:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:27:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:01 UTC)" executed successfully 2020-03-31 09:27:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:27:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:22.113221+00:00 (in 29.999301 seconds) 2020-03-31 09:27:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:22 UTC)" (scheduled at 2020-03-31 09:27:52.113221+00:00) securityworker stdout | 2020-03-31 09:27:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:22.113221+00:00 (in 29.999301 seconds) 2020-03-31 09:27:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:22 UTC)" (scheduled at 2020-03-31 09:27:52.113221+00:00) 2020-03-31 09:27:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:27:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:27:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:27:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:27:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:27:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:27:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:27:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:22 UTC)" executed successfully 2020-03-31 09:27:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:27:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:04.323352+00:00 (in 9.998764 seconds) 2020-03-31 09:27:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:04 UTC)" (scheduled at 2020-03-31 09:27:54.323352+00:00) 2020-03-31 09:27:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:27:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:04.323352+00:00 (in 9.998764 seconds) 2020-03-31 09:27:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:04 UTC)" (scheduled at 2020-03-31 09:27:54.323352+00:00) 2020-03-31 09:27:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 54, 325529), True, datetime.datetime(2020, 3, 31, 9, 27, 54, 325529), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:27:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 54, 325529), True, datetime.datetime(2020, 3, 31, 9, 27, 54, 325529), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:27:54,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:27:54,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:54,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:27:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:54,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:04 UTC)" executed successfully 2020-03-31 09:27:54,446 [527] [DEBUG] [app] Starting request: urn:request:ba6a2227-782d-403d-84fc-27a562f3347b (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:54,446 [527] [DEBUG] [app] Starting request: urn:request:ba6a2227-782d-403d-84fc-27a562f3347b (/health/instance) 2020-03-31 09:27:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:54,454 [551] [DEBUG] [app] Starting request: urn:request:76e4a91c-baa1-477e-8013-2ace9a65da2b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:54,454 [551] [DEBUG] [app] Starting request: urn:request:76e4a91c-baa1-477e-8013-2ace9a65da2b (/v1/_internal_ping) 2020-03-31 09:27:54,455 [551] [DEBUG] [app] Ending request: urn:request:76e4a91c-baa1-477e-8013-2ace9a65da2b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:54,455 [551] [DEBUG] [app] Ending request: urn:request:76e4a91c-baa1-477e-8013-2ace9a65da2b (/v1/_internal_ping) 2020-03-31 09:27:54,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:27:54,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:54,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:27:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:27:54,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:54,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:54,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:27:54,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:54,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:54,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:54,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:54,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:54,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:54,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:54,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:54,490 [518] [DEBUG] [app] Starting request: urn:request:e2a8550f-4c02-4ad0-a4f0-42fe37617297 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:54,490 [518] [DEBUG] [app] Starting request: urn:request:e2a8550f-4c02-4ad0-a4f0-42fe37617297 (/_internal_ping) 2020-03-31 09:27:54,491 [518] [DEBUG] [app] Ending request: urn:request:e2a8550f-4c02-4ad0-a4f0-42fe37617297 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:54,491 [518] [DEBUG] [app] Ending request: urn:request:e2a8550f-4c02-4ad0-a4f0-42fe37617297 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:27:54,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:27:54,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:27:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:54,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:27:54,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:27:54,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:54,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:54,505 [543] [DEBUG] [app] Starting request: urn:request:85a1fd9c-bc62-44aa-b203-6321dd707915 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:54,505 [543] [DEBUG] [app] Starting request: urn:request:85a1fd9c-bc62-44aa-b203-6321dd707915 (/c1/_internal_ping) 2020-03-31 09:27:54,506 [543] [DEBUG] [app] Ending request: urn:request:85a1fd9c-bc62-44aa-b203-6321dd707915 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:54,506 [543] [DEBUG] [app] Ending request: urn:request:85a1fd9c-bc62-44aa-b203-6321dd707915 (/c1/_internal_ping) 2020-03-31 09:27:54,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:54,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:27:54,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:54,508 [527] [DEBUG] [app] Ending request: urn:request:ba6a2227-782d-403d-84fc-27a562f3347b (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:27:54,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:54,508 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:27:54,508 [527] [DEBUG] [app] Ending request: urn:request:ba6a2227-782d-403d-84fc-27a562f3347b (/health/instance) 2020-03-31 09:27:54,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:27:54,508 [527] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:27:54,509 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:27:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:56,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:06.924497+00:00 (in 9.998864 seconds) 2020-03-31 09:27:56,930 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:06 UTC)" (scheduled at 2020-03-31 09:27:56.924497+00:00) 2020-03-31 09:27:56,930 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:27:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:56,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:06.924497+00:00 (in 9.998864 seconds) 2020-03-31 09:27:56,930 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:06 UTC)" (scheduled at 2020-03-31 09:27:56.924497+00:00) 2020-03-31 09:27:56,930 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:27:56,994 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 56, 930602), True, datetime.datetime(2020, 3, 31, 9, 27, 56, 930602), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:27:56,994 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 27, 56, 930602), True, datetime.datetime(2020, 3, 31, 9, 27, 56, 930602), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:27:57,036 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:27:57,036 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:57,036 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:27:57,036 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:27:57,036 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:57,036 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:06 UTC)" executed successfully 2020-03-31 09:27:58,272 [527] [DEBUG] [app] Starting request: urn:request:d700248f-cca1-4b21-b0ad-1ffb8eb4bdca (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:58,272 [527] [DEBUG] [app] Starting request: urn:request:d700248f-cca1-4b21-b0ad-1ffb8eb4bdca (/health/instance) 2020-03-31 09:27:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:58,278 [556] [DEBUG] [app] Starting request: urn:request:db1bac88-e6b9-4a4d-a992-25effbb6d8aa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:58,278 [556] [DEBUG] [app] Starting request: urn:request:db1bac88-e6b9-4a4d-a992-25effbb6d8aa (/v1/_internal_ping) 2020-03-31 09:27:58,279 [556] [DEBUG] [app] Ending request: urn:request:db1bac88-e6b9-4a4d-a992-25effbb6d8aa (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:27:58,279 [556] [DEBUG] [app] Ending request: urn:request:db1bac88-e6b9-4a4d-a992-25effbb6d8aa (/v1/_internal_ping) 2020-03-31 09:27:58,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:27:58,279 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:27:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:58,281 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:27:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:27:58,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:58,281 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:27:58,283 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:27:58,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:27:58,283 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:27:58,309 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:27:58,309 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:27:58,323 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:27:58,323 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:27:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:58,329 [526] [DEBUG] [app] Starting request: urn:request:c3952d45-01f5-4d25-91e2-28d9d940c9f7 (/_internal_ping) 2020-03-31 09:27:58,330 [526] [DEBUG] [app] Ending request: urn:request:c3952d45-01f5-4d25-91e2-28d9d940c9f7 (/_internal_ping) 2020-03-31 09:27:58,331 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:27:58,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:58,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:27:58,329 [526] [DEBUG] [app] Starting request: urn:request:c3952d45-01f5-4d25-91e2-28d9d940c9f7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:58,330 [526] [DEBUG] [app] Ending request: urn:request:c3952d45-01f5-4d25-91e2-28d9d940c9f7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:27:58,331 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:27:58,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:58,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:58,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:58,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:27:58,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:27:58,338 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:58,338 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:58,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:27:58,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:27:58,344 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:27:58,344 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:27:58,345 [534] [DEBUG] [app] Starting request: urn:request:7d6b17ef-c6d1-4fa6-a420-95269ff77928 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:58,345 [534] [DEBUG] [app] Starting request: urn:request:7d6b17ef-c6d1-4fa6-a420-95269ff77928 (/c1/_internal_ping) 2020-03-31 09:27:58,346 [534] [DEBUG] [app] Ending request: urn:request:7d6b17ef-c6d1-4fa6-a420-95269ff77928 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:27:58,346 [534] [DEBUG] [app] Ending request: urn:request:7d6b17ef-c6d1-4fa6-a420-95269ff77928 (/c1/_internal_ping) 2020-03-31 09:27:58,347 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:27:58,347 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:27:58,347 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:27:58,349 [527] [DEBUG] [app] Ending request: urn:request:d700248f-cca1-4b21-b0ad-1ffb8eb4bdca (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:27:58,347 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:27:58,349 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:58,350 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.079 49 0.079) gunicorn-web stdout | 2020-03-31 09:27:58,349 [527] [DEBUG] [app] Ending request: urn:request:d700248f-cca1-4b21-b0ad-1ffb8eb4bdca (/health/instance) gunicorn-web stdout | 2020-03-31 09:27:58,349 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:27:58,350 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:27:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:27:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.079 49 0.079) 2020-03-31 09:27:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:27:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:27:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:51.110610+00:00 (in 51.548654 seconds) 2020-03-31 09:27:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:59 UTC)" (scheduled at 2020-03-31 09:27:59.560326+00:00) 2020-03-31 09:27:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:27:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:51.110610+00:00 (in 51.548654 seconds) 2020-03-31 09:27:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:59 UTC)" (scheduled at 2020-03-31 09:27:59.560326+00:00) 2020-03-31 09:27:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:27:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:59 UTC)" executed successfully 2020-03-31 09:28:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:28:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:30.337699+00:00 (in 29.999190 seconds) 2020-03-31 09:28:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:30 UTC)" (scheduled at 2020-03-31 09:28:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:28:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:30.337699+00:00 (in 29.999190 seconds) 2020-03-31 09:28:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:30 UTC)" (scheduled at 2020-03-31 09:28:00.337699+00:00) 2020-03-31 09:28:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 28, 0, 339329), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:28:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 28, 0, 339329), False, 50, 1, 0]) 2020-03-31 09:28:00,391 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:28:00,391 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:28:00,391 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:00,391 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:28:00,391 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:00,391 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:30 UTC)" executed successfully 2020-03-31 09:28:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:28:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:11.440707+00:00 (in 9.998683 seconds) 2020-03-31 09:28:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:11 UTC)" (scheduled at 2020-03-31 09:28:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:28:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:11.440707+00:00 (in 9.998683 seconds) 2020-03-31 09:28:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:11 UTC)" (scheduled at 2020-03-31 09:28:01.440707+00:00) 2020-03-31 09:28:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:28:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:28:01,469 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:28:01,469 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:01,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:01,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:28:01,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:01,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:11 UTC)" executed successfully 2020-03-31 09:28:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:28:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:24.410927+00:00 (in 21.079928 seconds) 2020-03-31 09:28:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:33 UTC)" (scheduled at 2020-03-31 09:28:03.330112+00:00) 2020-03-31 09:28:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:28:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:24.410927+00:00 (in 21.079928 seconds) 2020-03-31 09:28:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:33 UTC)" (scheduled at 2020-03-31 09:28:03.330112+00:00) 2020-03-31 09:28:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 3, 331944), True, datetime.datetime(2020, 3, 31, 9, 28, 3, 331944), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:28:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 3, 331944), True, datetime.datetime(2020, 3, 31, 9, 28, 3, 331944), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:28:03,353 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:28:03,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:28:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:33 UTC)" executed successfully 2020-03-31 09:28:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:28:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:14 UTC)" (scheduled at 2020-03-31 09:28:04.323352+00:00) 2020-03-31 09:28:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:28:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:14 UTC)" (scheduled at 2020-03-31 09:28:04.323352+00:00) 2020-03-31 09:28:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:14.323352+00:00 (in 9.998919 seconds) notificationworker stdout | 2020-03-31 09:28:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:14.323352+00:00 (in 9.998919 seconds) 2020-03-31 09:28:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 4, 325257), True, datetime.datetime(2020, 3, 31, 9, 28, 4, 325257), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:28:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 4, 325257), True, datetime.datetime(2020, 3, 31, 9, 28, 4, 325257), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:28:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:28:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:28:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:14 UTC)" executed successfully 2020-03-31 09:28:04,446 [527] [DEBUG] [app] Starting request: urn:request:76c386b8-349f-465a-9121-317a8c13d294 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:04,446 [527] [DEBUG] [app] Starting request: urn:request:76c386b8-349f-465a-9121-317a8c13d294 (/health/instance) 2020-03-31 09:28:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:04,454 [553] [DEBUG] [app] Starting request: urn:request:072feeac-e651-43a0-8ad0-ad2fe45956dd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:04,454 [553] [DEBUG] [app] Starting request: urn:request:072feeac-e651-43a0-8ad0-ad2fe45956dd (/v1/_internal_ping) 2020-03-31 09:28:04,455 [553] [DEBUG] [app] Ending request: urn:request:072feeac-e651-43a0-8ad0-ad2fe45956dd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:04,455 [553] [DEBUG] [app] Ending request: urn:request:072feeac-e651-43a0-8ad0-ad2fe45956dd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:04,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:28:04,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:28:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:28:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:04,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:04,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:04,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:04,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:04,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:04,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:04,523 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:04,523 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:04,528 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:04,528 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:04,530 [518] [DEBUG] [app] Starting request: urn:request:41bca388-7da3-4f39-a491-77d78d3604b8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:04,530 [518] [DEBUG] [app] Starting request: urn:request:41bca388-7da3-4f39-a491-77d78d3604b8 (/_internal_ping) 2020-03-31 09:28:04,531 [518] [DEBUG] [app] Ending request: urn:request:41bca388-7da3-4f39-a491-77d78d3604b8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:04,531 [518] [DEBUG] [app] Ending request: urn:request:41bca388-7da3-4f39-a491-77d78d3604b8 (/_internal_ping) 2020-03-31 09:28:04,532 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:04,532 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:04,532 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:04,532 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:04,536 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:04,536 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:04,537 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:28:04,537 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:04,540 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:04,540 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:04,541 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:28:04,541 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:28:04,544 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:04,544 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:04,546 [534] [DEBUG] [app] Starting request: urn:request:636af01b-d635-4d77-9d8e-5836e6b306a2 (/c1/_internal_ping) 2020-03-31 09:28:04,547 [534] [DEBUG] [app] Ending request: urn:request:636af01b-d635-4d77-9d8e-5836e6b306a2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:04,546 [534] [DEBUG] [app] Starting request: urn:request:636af01b-d635-4d77-9d8e-5836e6b306a2 (/c1/_internal_ping) 2020-03-31 09:28:04,547 [534] [DEBUG] [app] Ending request: urn:request:636af01b-d635-4d77-9d8e-5836e6b306a2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:04,548 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:04,548 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:28:04,548 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:04,548 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:04,549 [527] [DEBUG] [app] Ending request: urn:request:76c386b8-349f-465a-9121-317a8c13d294 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:04,549 [527] [DEBUG] [app] Ending request: urn:request:76c386b8-349f-465a-9121-317a8c13d294 (/health/instance) 2020-03-31 09:28:04,550 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.105 49 0.105) gunicorn-web stdout | 2020-03-31 09:28:04,550 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:04,550 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.105 49 0.105) gunicorn-web stdout | 2020-03-31 09:28:04,550 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:28:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:28:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:16.924497+00:00 (in 9.999091 seconds) 2020-03-31 09:28:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:16 UTC)" (scheduled at 2020-03-31 09:28:06.924497+00:00) 2020-03-31 09:28:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:28:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:16.924497+00:00 (in 9.999091 seconds) 2020-03-31 09:28:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:16 UTC)" (scheduled at 2020-03-31 09:28:06.924497+00:00) 2020-03-31 09:28:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 6, 926380), True, datetime.datetime(2020, 3, 31, 9, 28, 6, 926380), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:28:06,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 6, 926380), True, datetime.datetime(2020, 3, 31, 9, 28, 6, 926380), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:28:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:28:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:28:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:16 UTC)" executed successfully 2020-03-31 09:28:08,273 [527] [DEBUG] [app] Starting request: urn:request:0506618c-1380-4262-8a4a-eec87c3b5b0f (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:08,273 [527] [DEBUG] [app] Starting request: urn:request:0506618c-1380-4262-8a4a-eec87c3b5b0f (/health/instance) 2020-03-31 09:28:08,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:08,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:08,278 [538] [DEBUG] [app] Starting request: urn:request:d480eb25-b8d2-45a7-9894-22984a37c975 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:08,278 [538] [DEBUG] [app] Starting request: urn:request:d480eb25-b8d2-45a7-9894-22984a37c975 (/v1/_internal_ping) 2020-03-31 09:28:08,279 [538] [DEBUG] [app] Ending request: urn:request:d480eb25-b8d2-45a7-9894-22984a37c975 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:08,279 [538] [DEBUG] [app] Ending request: urn:request:d480eb25-b8d2-45a7-9894-22984a37c975 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:08,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:08,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:28:08,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:08,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:08,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:08,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:28:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:08,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:08,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:08,299 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:08,299 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:08,323 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:08,323 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:08,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:08,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:08,329 [518] [DEBUG] [app] Starting request: urn:request:62a8e404-3b5a-403f-9f4d-55123aafac79 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:08,329 [518] [DEBUG] [app] Starting request: urn:request:62a8e404-3b5a-403f-9f4d-55123aafac79 (/_internal_ping) 2020-03-31 09:28:08,330 [518] [DEBUG] [app] Ending request: urn:request:62a8e404-3b5a-403f-9f4d-55123aafac79 (/_internal_ping) 2020-03-31 09:28:08,331 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:08,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:08,330 [518] [DEBUG] [app] Ending request: urn:request:62a8e404-3b5a-403f-9f4d-55123aafac79 (/_internal_ping) 2020-03-31 09:28:08,331 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:08,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:28:08,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:08,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:08,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:28:08,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:28:08,339 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:08,339 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:28:08,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:08,340 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:08,343 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:08,343 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:08,345 [534] [DEBUG] [app] Starting request: urn:request:9403183d-b04f-4c8c-ab51-cc2216799abe (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:08,345 [534] [DEBUG] [app] Starting request: urn:request:9403183d-b04f-4c8c-ab51-cc2216799abe (/c1/_internal_ping) 2020-03-31 09:28:08,346 [534] [DEBUG] [app] Ending request: urn:request:9403183d-b04f-4c8c-ab51-cc2216799abe (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:08,346 [534] [DEBUG] [app] Ending request: urn:request:9403183d-b04f-4c8c-ab51-cc2216799abe (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:08,346 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:08,346 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:28:08,346 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:08,346 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:08,348 [527] [DEBUG] [app] Ending request: urn:request:0506618c-1380-4262-8a4a-eec87c3b5b0f (/health/instance) 2020-03-31 09:28:08,348 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:08,348 [527] [DEBUG] [app] Ending request: urn:request:0506618c-1380-4262-8a4a-eec87c3b5b0f (/health/instance) 2020-03-31 09:28:08,348 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:08,349 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) gunicorn-web stdout | 2020-03-31 09:28:08,349 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) 2020-03-31 09:28:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:28:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:21.440707+00:00 (in 9.998738 seconds) 2020-03-31 09:28:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:21 UTC)" (scheduled at 2020-03-31 09:28:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:28:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:21.440707+00:00 (in 9.998738 seconds) 2020-03-31 09:28:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:21 UTC)" (scheduled at 2020-03-31 09:28:11.440707+00:00) 2020-03-31 09:28:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:28:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:28:11,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:28:11,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:11,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:28:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:11,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:21 UTC)" executed successfully 2020-03-31 09:28:14,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:28:14,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:24.323352+00:00 (in 9.996626 seconds) 2020-03-31 09:28:14,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:24 UTC)" (scheduled at 2020-03-31 09:28:14.323352+00:00) 2020-03-31 09:28:14,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:28:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:24.323352+00:00 (in 9.996626 seconds) 2020-03-31 09:28:14,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:24 UTC)" (scheduled at 2020-03-31 09:28:14.323352+00:00) 2020-03-31 09:28:14,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:14,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 14, 327639), True, datetime.datetime(2020, 3, 31, 9, 28, 14, 327639), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:28:14,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 14, 327639), True, datetime.datetime(2020, 3, 31, 9, 28, 14, 327639), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:28:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:28:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:28:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:24 UTC)" executed successfully 2020-03-31 09:28:14,446 [518] [DEBUG] [app] Starting request: urn:request:243c3796-3857-44c8-8158-f1970fc17a91 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:14,446 [518] [DEBUG] [app] Starting request: urn:request:243c3796-3857-44c8-8158-f1970fc17a91 (/health/instance) 2020-03-31 09:28:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:14,454 [538] [DEBUG] [app] Starting request: urn:request:3ad6381c-9970-4c4f-9a56-eb126344028d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:14,454 [538] [DEBUG] [app] Starting request: urn:request:3ad6381c-9970-4c4f-9a56-eb126344028d (/v1/_internal_ping) 2020-03-31 09:28:14,456 [538] [DEBUG] [app] Ending request: urn:request:3ad6381c-9970-4c4f-9a56-eb126344028d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:14,456 [538] [DEBUG] [app] Ending request: urn:request:3ad6381c-9970-4c4f-9a56-eb126344028d (/v1/_internal_ping) 2020-03-31 09:28:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:28:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:28:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:14,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:28:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:14,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:14,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:14,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:14,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:14,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:14,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:14,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:14,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:14,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:14,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:14,487 [527] [DEBUG] [app] Starting request: urn:request:1f444f37-d855-47a2-b040-b2b5f816958f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:14,487 [527] [DEBUG] [app] Starting request: urn:request:1f444f37-d855-47a2-b040-b2b5f816958f (/_internal_ping) 2020-03-31 09:28:14,489 [527] [DEBUG] [app] Ending request: urn:request:1f444f37-d855-47a2-b040-b2b5f816958f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:14,489 [527] [DEBUG] [app] Ending request: urn:request:1f444f37-d855-47a2-b040-b2b5f816958f (/_internal_ping) 2020-03-31 09:28:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:14,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:14,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:28:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:28:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:28:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:14,503 [535] [DEBUG] [app] Starting request: urn:request:741edff6-57b4-4744-8473-ca75eab538cb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:14,503 [535] [DEBUG] [app] Starting request: urn:request:741edff6-57b4-4744-8473-ca75eab538cb (/c1/_internal_ping) 2020-03-31 09:28:14,504 [535] [DEBUG] [app] Ending request: urn:request:741edff6-57b4-4744-8473-ca75eab538cb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:14,504 [535] [DEBUG] [app] Ending request: urn:request:741edff6-57b4-4744-8473-ca75eab538cb (/c1/_internal_ping) 2020-03-31 09:28:14,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:14,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:14,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:14,506 [518] [DEBUG] [app] Ending request: urn:request:243c3796-3857-44c8-8158-f1970fc17a91 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:28:14,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:14,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:14,506 [518] [DEBUG] [app] Ending request: urn:request:243c3796-3857-44c8-8158-f1970fc17a91 (/health/instance) 2020-03-31 09:28:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:28:14,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:28:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:28:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:45.064514+00:00 (in 29.999219 seconds) 2020-03-31 09:28:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:45 UTC)" (scheduled at 2020-03-31 09:28:15.064514+00:00) gcworker stdout | 2020-03-31 09:28:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:45.064514+00:00 (in 29.999219 seconds) 2020-03-31 09:28:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:45 UTC)" (scheduled at 2020-03-31 09:28:15.064514+00:00) 2020-03-31 09:28:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437295, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:28:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437295, 1209600, 500, 1, 0]) 2020-03-31 09:28:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:28:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:28:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:28:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:45 UTC)" executed successfully 2020-03-31 09:28:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:28:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:26.924497+00:00 (in 9.999256 seconds) storagereplication stdout | 2020-03-31 09:28:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:26.924497+00:00 (in 9.999256 seconds) 2020-03-31 09:28:16,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:26 UTC)" (scheduled at 2020-03-31 09:28:16.924497+00:00) storagereplication stdout | 2020-03-31 09:28:16,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:26 UTC)" (scheduled at 2020-03-31 09:28:16.924497+00:00) 2020-03-31 09:28:16,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:28:16,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:16,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 16, 927992), True, datetime.datetime(2020, 3, 31, 9, 28, 16, 927992), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:28:16,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 16, 927992), True, datetime.datetime(2020, 3, 31, 9, 28, 16, 927992), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:28:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:28:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:28:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:26 UTC)" executed successfully 2020-03-31 09:28:18,273 [526] [DEBUG] [app] Starting request: urn:request:bc9c8d88-6009-4994-a075-fadb24732a34 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:18,273 [526] [DEBUG] [app] Starting request: urn:request:bc9c8d88-6009-4994-a075-fadb24732a34 (/health/instance) 2020-03-31 09:28:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:18,280 [549] [DEBUG] [app] Starting request: urn:request:272539e4-cd00-4024-b64e-61c8822d2737 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:18,280 [549] [DEBUG] [app] Starting request: urn:request:272539e4-cd00-4024-b64e-61c8822d2737 (/v1/_internal_ping) 2020-03-31 09:28:18,281 [549] [DEBUG] [app] Ending request: urn:request:272539e4-cd00-4024-b64e-61c8822d2737 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:18,281 [549] [DEBUG] [app] Ending request: urn:request:272539e4-cd00-4024-b64e-61c8822d2737 (/v1/_internal_ping) 2020-03-31 09:28:18,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:28:18,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:18,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:18,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:18,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:18,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:28:18,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:28:18,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:18,287 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:18,287 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:18,288 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:18,288 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:18,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:18,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:18,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:18,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:18,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:18,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:18,315 [520] [DEBUG] [app] Starting request: urn:request:daccfe48-9d9f-4996-9125-610d6c511471 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:18,315 [520] [DEBUG] [app] Starting request: urn:request:daccfe48-9d9f-4996-9125-610d6c511471 (/_internal_ping) 2020-03-31 09:28:18,317 [520] [DEBUG] [app] Ending request: urn:request:daccfe48-9d9f-4996-9125-610d6c511471 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:18,317 [520] [DEBUG] [app] Ending request: urn:request:daccfe48-9d9f-4996-9125-610d6c511471 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:28:18,317 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:18,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:28:18,317 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:18,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:18,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:18,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:18,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:28:18,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:18,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:18,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:28:18,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:18,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:18,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:18,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:18,329 [535] [DEBUG] [app] Starting request: urn:request:a9f319b5-6350-48af-98fd-d34a94448ed4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:18,329 [535] [DEBUG] [app] Starting request: urn:request:a9f319b5-6350-48af-98fd-d34a94448ed4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:18,331 [535] [DEBUG] [app] Ending request: urn:request:a9f319b5-6350-48af-98fd-d34a94448ed4 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:18,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:18,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:28:18,331 [535] [DEBUG] [app] Ending request: urn:request:a9f319b5-6350-48af-98fd-d34a94448ed4 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:18,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:18,333 [526] [DEBUG] [app] Ending request: urn:request:bc9c8d88-6009-4994-a075-fadb24732a34 (/health/instance) 2020-03-31 09:28:18,333 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:28:18,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:18,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:28:18,333 [526] [DEBUG] [app] Ending request: urn:request:bc9c8d88-6009-4994-a075-fadb24732a34 (/health/instance) 2020-03-31 09:28:18,333 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:18,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:28:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:28:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:21,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:31.440707+00:00 (in 9.997634 seconds) 2020-03-31 09:28:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:31 UTC)" (scheduled at 2020-03-31 09:28:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:28:21,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:31.440707+00:00 (in 9.997634 seconds) 2020-03-31 09:28:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:31 UTC)" (scheduled at 2020-03-31 09:28:21.440707+00:00) 2020-03-31 09:28:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:28:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:28:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:21,460 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:28:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:28:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:31 UTC)" executed successfully 2020-03-31 09:28:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:28:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:52.113221+00:00 (in 29.999158 seconds) 2020-03-31 09:28:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:52 UTC)" (scheduled at 2020-03-31 09:28:22.113221+00:00) securityworker stdout | 2020-03-31 09:28:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:52.113221+00:00 (in 29.999158 seconds) 2020-03-31 09:28:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:52 UTC)" (scheduled at 2020-03-31 09:28:22.113221+00:00) 2020-03-31 09:28:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:28:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:28:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:28:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:28:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:28:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:28:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:28:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:28:52 UTC)" executed successfully 2020-03-31 09:28:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:28:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:34.323352+00:00 (in 9.995962 seconds) 2020-03-31 09:28:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:34 UTC)" (scheduled at 2020-03-31 09:28:24.323352+00:00) 2020-03-31 09:28:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:28:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:34.323352+00:00 (in 9.995962 seconds) 2020-03-31 09:28:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:34 UTC)" (scheduled at 2020-03-31 09:28:24.323352+00:00) 2020-03-31 09:28:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:24,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 24, 328392), True, datetime.datetime(2020, 3, 31, 9, 28, 24, 328392), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:28:24,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 24, 328392), True, datetime.datetime(2020, 3, 31, 9, 28, 24, 328392), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:28:24,380 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:24,380 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:28:24,380 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:24,380 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:24,380 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:28:24,380 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:34 UTC)" executed successfully 2020-03-31 09:28:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:28:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:33.330112+00:00 (in 8.918274 seconds) 2020-03-31 09:28:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:24 UTC)" (scheduled at 2020-03-31 09:28:24.410927+00:00) 2020-03-31 09:28:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:28:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:33.330112+00:00 (in 8.918274 seconds) 2020-03-31 09:28:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:24 UTC)" (scheduled at 2020-03-31 09:28:24.410927+00:00) 2020-03-31 09:28:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:28:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:24 UTC)" executed successfully 2020-03-31 09:28:24,445 [527] [DEBUG] [app] Starting request: urn:request:c39d4948-c529-4105-a6c0-80e5196d8d6a (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:24,445 [527] [DEBUG] [app] Starting request: urn:request:c39d4948-c529-4105-a6c0-80e5196d8d6a (/health/instance) 2020-03-31 09:28:24,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:24,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:24,453 [552] [DEBUG] [app] Starting request: urn:request:4c54bece-ecde-409d-8ad7-6dc9004c85f2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:24,453 [552] [DEBUG] [app] Starting request: urn:request:4c54bece-ecde-409d-8ad7-6dc9004c85f2 (/v1/_internal_ping) 2020-03-31 09:28:24,455 [552] [DEBUG] [app] Ending request: urn:request:4c54bece-ecde-409d-8ad7-6dc9004c85f2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:24,455 [552] [DEBUG] [app] Ending request: urn:request:4c54bece-ecde-409d-8ad7-6dc9004c85f2 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:24,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:24,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:28:24,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:24,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:24,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:24,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:24,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:28:24,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:24,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:24,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:24,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:24,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:24,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:24,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:24,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:24,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:24,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:24,489 [518] [DEBUG] [app] Starting request: urn:request:f45e7ce6-4fae-4d3e-a9c0-422671cdaaa5 (/_internal_ping) 2020-03-31 09:28:24,490 [518] [DEBUG] [app] Ending request: urn:request:f45e7ce6-4fae-4d3e-a9c0-422671cdaaa5 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:28:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:28:24,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:24,489 [518] [DEBUG] [app] Starting request: urn:request:f45e7ce6-4fae-4d3e-a9c0-422671cdaaa5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:24,490 [518] [DEBUG] [app] Ending request: urn:request:f45e7ce6-4fae-4d3e-a9c0-422671cdaaa5 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:24,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:24,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:24,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:28:24,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:28:24,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:24,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:28:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:28:24,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:24,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:24,506 [534] [DEBUG] [app] Starting request: urn:request:a1e4f312-52e4-4b8c-9727-97335324756a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:24,506 [534] [DEBUG] [app] Starting request: urn:request:a1e4f312-52e4-4b8c-9727-97335324756a (/c1/_internal_ping) 2020-03-31 09:28:24,507 [534] [DEBUG] [app] Ending request: urn:request:a1e4f312-52e4-4b8c-9727-97335324756a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:24,507 [534] [DEBUG] [app] Ending request: urn:request:a1e4f312-52e4-4b8c-9727-97335324756a (/c1/_internal_ping) 2020-03-31 09:28:24,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:24,508 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:24,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:24,510 [527] [DEBUG] [app] Ending request: urn:request:c39d4948-c529-4105-a6c0-80e5196d8d6a (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:28:24,508 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:24,510 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:28:24,510 [527] [DEBUG] [app] Ending request: urn:request:c39d4948-c529-4105-a6c0-80e5196d8d6a (/health/instance) 2020-03-31 09:28:24,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:28:24,510 [527] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:28:24,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:28:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:28:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:36.924497+00:00 (in 9.999252 seconds) 2020-03-31 09:28:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:36 UTC)" (scheduled at 2020-03-31 09:28:26.924497+00:00) 2020-03-31 09:28:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:28:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:36.924497+00:00 (in 9.999252 seconds) 2020-03-31 09:28:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:36 UTC)" (scheduled at 2020-03-31 09:28:26.924497+00:00) 2020-03-31 09:28:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 26, 925845), True, datetime.datetime(2020, 3, 31, 9, 28, 26, 925845), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:28:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 26, 925845), True, datetime.datetime(2020, 3, 31, 9, 28, 26, 925845), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:28:26,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:28:26,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:28:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:36 UTC)" executed successfully 2020-03-31 09:28:28,273 [526] [DEBUG] [app] Starting request: urn:request:0b277fde-341f-4c5f-a041-310562c33593 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:28,273 [526] [DEBUG] [app] Starting request: urn:request:0b277fde-341f-4c5f-a041-310562c33593 (/health/instance) 2020-03-31 09:28:28,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:28,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:28,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:28,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:28,282 [546] [DEBUG] [app] Starting request: urn:request:bbb32032-f5f9-416b-b393-d2f9e10aaa64 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:28,282 [546] [DEBUG] [app] Starting request: urn:request:bbb32032-f5f9-416b-b393-d2f9e10aaa64 (/v1/_internal_ping) 2020-03-31 09:28:28,283 [546] [DEBUG] [app] Ending request: urn:request:bbb32032-f5f9-416b-b393-d2f9e10aaa64 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:28,283 [546] [DEBUG] [app] Ending request: urn:request:bbb32032-f5f9-416b-b393-d2f9e10aaa64 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:28,284 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:28,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:28:28,284 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:28,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:28,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:28:28,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:28,287 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:28:28,287 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:28,289 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:28,289 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:28,289 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:28,289 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:28,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:28,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:28,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:28,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:28,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:28,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:28,315 [526] [DEBUG] [app] Starting request: urn:request:748a7e78-2279-4465-b160-3560a7a99faf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:28,315 [526] [DEBUG] [app] Starting request: urn:request:748a7e78-2279-4465-b160-3560a7a99faf (/_internal_ping) 2020-03-31 09:28:28,316 [526] [DEBUG] [app] Ending request: urn:request:748a7e78-2279-4465-b160-3560a7a99faf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:28,316 [526] [DEBUG] [app] Ending request: urn:request:748a7e78-2279-4465-b160-3560a7a99faf (/_internal_ping) 2020-03-31 09:28:28,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:28,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:28,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:28:28,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:28:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:28,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:28:28,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:28,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:28:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:28:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:28,330 [534] [DEBUG] [app] Starting request: urn:request:abe2334e-571d-4f14-ae5f-b0c6a5899885 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:28,330 [534] [DEBUG] [app] Starting request: urn:request:abe2334e-571d-4f14-ae5f-b0c6a5899885 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:28,331 [534] [DEBUG] [app] Ending request: urn:request:abe2334e-571d-4f14-ae5f-b0c6a5899885 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:28,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:28,331 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:28:28,331 [534] [DEBUG] [app] Ending request: urn:request:abe2334e-571d-4f14-ae5f-b0c6a5899885 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:28,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:28:28,331 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:28,333 [526] [DEBUG] [app] Ending request: urn:request:0b277fde-341f-4c5f-a041-310562c33593 (/health/instance) 2020-03-31 09:28:28,333 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:28,333 [526] [DEBUG] [app] Ending request: urn:request:0b277fde-341f-4c5f-a041-310562c33593 (/health/instance) 2020-03-31 09:28:28,333 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:28,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:28:28,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:28:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:28:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:00.337699+00:00 (in 29.998923 seconds) 2020-03-31 09:28:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:00 UTC)" (scheduled at 2020-03-31 09:28:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:28:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:00.337699+00:00 (in 29.998923 seconds) 2020-03-31 09:28:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:00 UTC)" (scheduled at 2020-03-31 09:28:30.337699+00:00) 2020-03-31 09:28:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 28, 30, 339509), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:28:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 28, 30, 339509), False, 50, 1, 0]) 2020-03-31 09:28:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:28:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:28:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:28:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:00 UTC)" executed successfully 2020-03-31 09:28:31,447 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:28:31,447 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:31,447 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:41.440707+00:00 (in 9.993125 seconds) repositoryactioncounter stdout | 2020-03-31 09:28:31,447 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:41.440707+00:00 (in 9.993125 seconds) 2020-03-31 09:28:31,448 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:41 UTC)" (scheduled at 2020-03-31 09:28:31.440707+00:00) 2020-03-31 09:28:31,454 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:28:31,448 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:41 UTC)" (scheduled at 2020-03-31 09:28:31.440707+00:00) 2020-03-31 09:28:31,454 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:28:31,508 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:31,508 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:31,509 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:28:31,508 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:31,508 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:31,509 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:41 UTC)" executed successfully 2020-03-31 09:28:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:28:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:03.330112+00:00 (in 29.999120 seconds) 2020-03-31 09:28:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:03 UTC)" (scheduled at 2020-03-31 09:28:33.330112+00:00) 2020-03-31 09:28:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:28:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:03.330112+00:00 (in 29.999120 seconds) 2020-03-31 09:28:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:03 UTC)" (scheduled at 2020-03-31 09:28:33.330112+00:00) 2020-03-31 09:28:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 33, 331888), True, datetime.datetime(2020, 3, 31, 9, 28, 33, 331888), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:28:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 33, 331888), True, datetime.datetime(2020, 3, 31, 9, 28, 33, 331888), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:28:33,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:28:33,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:28:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:03 UTC)" executed successfully 2020-03-31 09:28:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:28:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:43.539999+00:00 (in 10.171968 seconds) 2020-03-31 09:28:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:33 UTC)" (scheduled at 2020-03-31 09:28:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:28:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:43.539999+00:00 (in 10.171968 seconds) 2020-03-31 09:28:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:33 UTC)" (scheduled at 2020-03-31 09:28:33.366791+00:00) 2020-03-31 09:28:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:28:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 33, 368773), True, datetime.datetime(2020, 3, 31, 9, 28, 33, 368773), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:28:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 33, 368773), True, datetime.datetime(2020, 3, 31, 9, 28, 33, 368773), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:28:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:28:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:28:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:33 UTC)" executed successfully 2020-03-31 09:28:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:28:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:44.323352+00:00 (in 9.999020 seconds) 2020-03-31 09:28:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:44 UTC)" (scheduled at 2020-03-31 09:28:34.323352+00:00) 2020-03-31 09:28:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:28:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:44.323352+00:00 (in 9.999020 seconds) 2020-03-31 09:28:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:44 UTC)" (scheduled at 2020-03-31 09:28:34.323352+00:00) 2020-03-31 09:28:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 34, 325389), True, datetime.datetime(2020, 3, 31, 9, 28, 34, 325389), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:28:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 34, 325389), True, datetime.datetime(2020, 3, 31, 9, 28, 34, 325389), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:28:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:28:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:28:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:44 UTC)" executed successfully 2020-03-31 09:28:34,446 [527] [DEBUG] [app] Starting request: urn:request:9fd2af9c-f4a7-4660-8ccd-34b39bb6b451 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:34,446 [527] [DEBUG] [app] Starting request: urn:request:9fd2af9c-f4a7-4660-8ccd-34b39bb6b451 (/health/instance) 2020-03-31 09:28:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:34,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:34,454 [550] [DEBUG] [app] Starting request: urn:request:36588718-d0f0-481c-a30c-c01bc8de4c65 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:34,454 [550] [DEBUG] [app] Starting request: urn:request:36588718-d0f0-481c-a30c-c01bc8de4c65 (/v1/_internal_ping) 2020-03-31 09:28:34,455 [550] [DEBUG] [app] Ending request: urn:request:36588718-d0f0-481c-a30c-c01bc8de4c65 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:34,455 [550] [DEBUG] [app] Ending request: urn:request:36588718-d0f0-481c-a30c-c01bc8de4c65 (/v1/_internal_ping) 2020-03-31 09:28:34,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:28:34,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:34,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:28:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:34,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:34,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:34,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:34,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:34,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:34,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:34,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:34,489 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:34,489 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:34,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:34,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:34,495 [527] [DEBUG] [app] Starting request: urn:request:e289aa56-6606-4196-9824-8fb2cf42b2c1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:34,495 [527] [DEBUG] [app] Starting request: urn:request:e289aa56-6606-4196-9824-8fb2cf42b2c1 (/_internal_ping) 2020-03-31 09:28:34,496 [527] [DEBUG] [app] Ending request: urn:request:e289aa56-6606-4196-9824-8fb2cf42b2c1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:34,496 [527] [DEBUG] [app] Ending request: urn:request:e289aa56-6606-4196-9824-8fb2cf42b2c1 (/_internal_ping) 2020-03-31 09:28:34,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:34,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:28:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:28:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:34,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:34,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:28:34,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:34,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:34,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:34,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:34,512 [535] [DEBUG] [app] Starting request: urn:request:96f4b61a-7c6b-40c5-87dc-e485a889a0bb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:34,512 [535] [DEBUG] [app] Starting request: urn:request:96f4b61a-7c6b-40c5-87dc-e485a889a0bb (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:34,513 [535] [DEBUG] [app] Ending request: urn:request:96f4b61a-7c6b-40c5-87dc-e485a889a0bb (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:34,514 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:34,514 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:28:34,513 [535] [DEBUG] [app] Ending request: urn:request:96f4b61a-7c6b-40c5-87dc-e485a889a0bb (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:34,514 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:28:34,514 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:34,515 [527] [DEBUG] [app] Ending request: urn:request:9fd2af9c-f4a7-4660-8ccd-34b39bb6b451 (/health/instance) 2020-03-31 09:28:34,515 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:34,515 [527] [DEBUG] [app] Ending request: urn:request:9fd2af9c-f4a7-4660-8ccd-34b39bb6b451 (/health/instance) 2020-03-31 09:28:34,515 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:34,516 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:28:34,516 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:28:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:28:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:38.010934+00:00 (in 1.085690 seconds) 2020-03-31 09:28:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:46 UTC)" (scheduled at 2020-03-31 09:28:36.924497+00:00) 2020-03-31 09:28:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:28:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:38.010934+00:00 (in 1.085690 seconds) 2020-03-31 09:28:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:46 UTC)" (scheduled at 2020-03-31 09:28:36.924497+00:00) 2020-03-31 09:28:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 36, 925949), True, datetime.datetime(2020, 3, 31, 9, 28, 36, 925949), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:28:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 36, 925949), True, datetime.datetime(2020, 3, 31, 9, 28, 36, 925949), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:28:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:28:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:28:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:46 UTC)" executed successfully 2020-03-31 09:28:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:28:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:38,011 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" (scheduled at 2020-03-31 09:28:38.010934+00:00) 2020-03-31 09:28:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:28:38,011 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" (scheduled at 2020-03-31 09:28:38.010934+00:00) 2020-03-31 09:28:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:28:38 UTC)" executed successfully 2020-03-31 09:28:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:46.924497+00:00 (in 8.912831 seconds) storagereplication stdout | 2020-03-31 09:28:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:46.924497+00:00 (in 8.912831 seconds) 2020-03-31 09:28:38,272 [518] [DEBUG] [app] Starting request: urn:request:c33de886-4650-424e-ab44-b2780c4d5462 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:38,272 [518] [DEBUG] [app] Starting request: urn:request:c33de886-4650-424e-ab44-b2780c4d5462 (/health/instance) 2020-03-31 09:28:38,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:38,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:38,280 [556] [DEBUG] [app] Starting request: urn:request:ae50aea1-0a5b-4e52-b70a-15446b6894b7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:38,280 [556] [DEBUG] [app] Starting request: urn:request:ae50aea1-0a5b-4e52-b70a-15446b6894b7 (/v1/_internal_ping) 2020-03-31 09:28:38,281 [556] [DEBUG] [app] Ending request: urn:request:ae50aea1-0a5b-4e52-b70a-15446b6894b7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:38,281 [556] [DEBUG] [app] Ending request: urn:request:ae50aea1-0a5b-4e52-b70a-15446b6894b7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:28:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:38,284 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:28:38,284 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:38,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:28:38,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:38,288 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:38,288 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:38,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:38,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:38,312 [518] [DEBUG] [app] Starting request: urn:request:20269a4b-6b89-4847-91c1-edef6b8e3d77 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:38,312 [518] [DEBUG] [app] Starting request: urn:request:20269a4b-6b89-4847-91c1-edef6b8e3d77 (/_internal_ping) 2020-03-31 09:28:38,313 [518] [DEBUG] [app] Ending request: urn:request:20269a4b-6b89-4847-91c1-edef6b8e3d77 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:38,313 [518] [DEBUG] [app] Ending request: urn:request:20269a4b-6b89-4847-91c1-edef6b8e3d77 (/_internal_ping) 2020-03-31 09:28:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:28:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:28:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:28:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:28:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:38,327 [535] [DEBUG] [app] Starting request: urn:request:b84da403-f42c-4bf9-859d-ea5f811ba4f6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:38,327 [535] [DEBUG] [app] Starting request: urn:request:b84da403-f42c-4bf9-859d-ea5f811ba4f6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:38,328 [535] [DEBUG] [app] Ending request: urn:request:b84da403-f42c-4bf9-859d-ea5f811ba4f6 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:38,328 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:28:38,328 [535] [DEBUG] [app] Ending request: urn:request:b84da403-f42c-4bf9-859d-ea5f811ba4f6 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:38,330 [518] [DEBUG] [app] Ending request: urn:request:c33de886-4650-424e-ab44-b2780c4d5462 (/health/instance) 2020-03-31 09:28:38,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:28:38,328 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:28:38,330 [518] [DEBUG] [app] Ending request: urn:request:c33de886-4650-424e-ab44-b2780c4d5462 (/health/instance) 2020-03-31 09:28:38,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:28:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:28:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:51.440707+00:00 (in 9.999234 seconds) 2020-03-31 09:28:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:51 UTC)" (scheduled at 2020-03-31 09:28:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:28:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:51.440707+00:00 (in 9.999234 seconds) 2020-03-31 09:28:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:51 UTC)" (scheduled at 2020-03-31 09:28:41.440707+00:00) 2020-03-31 09:28:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:28:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:28:41,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:28:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:28:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:51 UTC)" executed successfully 2020-03-31 09:28:43,551 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:43,560 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:33.366791+00:00 (in 49.814842 seconds) exportactionlogsworker stdout | 2020-03-31 09:28:43,551 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:43,560 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:33.366791+00:00 (in 49.814842 seconds) 2020-03-31 09:28:43,561 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:43 UTC)" (scheduled at 2020-03-31 09:28:43.539999+00:00) 2020-03-31 09:28:43,589 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:43,589 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:28:43,561 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:43 UTC)" (scheduled at 2020-03-31 09:28:43.539999+00:00) exportactionlogsworker stdout | 2020-03-31 09:28:43,589 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:43,589 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:43 UTC)" executed successfully 2020-03-31 09:28:44,331 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:44,331 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:49.498966+00:00 (in 5.167473 seconds) 2020-03-31 09:28:44,342 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:54 UTC)" (scheduled at 2020-03-31 09:28:44.323352+00:00) 2020-03-31 09:28:44,342 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:44,349 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 44, 342990), True, datetime.datetime(2020, 3, 31, 9, 28, 44, 342990), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:28:44,331 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:44,331 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:49.498966+00:00 (in 5.167473 seconds) 2020-03-31 09:28:44,342 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:54 UTC)" (scheduled at 2020-03-31 09:28:44.323352+00:00) 2020-03-31 09:28:44,342 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:44,349 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 44, 342990), True, datetime.datetime(2020, 3, 31, 9, 28, 44, 342990), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:28:44,425 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:44,425 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:44,425 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:28:44,425 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:44,425 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:44,425 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:54 UTC)" executed successfully 2020-03-31 09:28:44,509 [520] [DEBUG] [app] Starting request: urn:request:33666a28-a307-4249-bf5b-857587282504 (/health/instance) 2020-03-31 09:28:44,573 [520] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 gunicorn-web stdout | 2020-03-31 09:28:44,509 [520] [DEBUG] [app] Starting request: urn:request:33666a28-a307-4249-bf5b-857587282504 (/health/instance) 2020-03-31 09:28:44,573 [520] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): localhost:8443 2020-03-31 09:28:44,625 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:44,625 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:44,630 [538] [DEBUG] [app] Starting request: urn:request:ac12d2bd-0233-4709-987a-50a0da98c99c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:44,630 [538] [DEBUG] [app] Starting request: urn:request:ac12d2bd-0233-4709-987a-50a0da98c99c (/v1/_internal_ping) 2020-03-31 09:28:44,633 [538] [DEBUG] [app] Ending request: urn:request:ac12d2bd-0233-4709-987a-50a0da98c99c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:44,633 [538] [DEBUG] [app] Ending request: urn:request:ac12d2bd-0233-4709-987a-50a0da98c99c (/v1/_internal_ping) 2020-03-31 09:28:44,635 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-web stdout | 2020-03-31 09:28:44,635 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) 2020-03-31 09:28:44,644 [520] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:44,644 [520] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:44,647 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:28:44,647 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:44,652 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:28:44,652 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:44,655 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:44,655 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:44,658 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:44,658 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:44,666 [520] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:44,698 [520] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:44,706 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:44,708 [518] [DEBUG] [app] Starting request: urn:request:bf066cf3-2ae0-4683-a99b-a498de5a58d4 (/_internal_ping) 2020-03-31 09:28:44,709 [518] [DEBUG] [app] Ending request: urn:request:bf066cf3-2ae0-4683-a99b-a498de5a58d4 (/_internal_ping) 2020-03-31 09:28:44,711 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:28:44,712 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:28:44,666 [520] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:44,698 [520] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:44,706 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:44,708 [518] [DEBUG] [app] Starting request: urn:request:bf066cf3-2ae0-4683-a99b-a498de5a58d4 (/_internal_ping) 2020-03-31 09:28:44,709 [518] [DEBUG] [app] Ending request: urn:request:bf066cf3-2ae0-4683-a99b-a498de5a58d4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:44,711 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:28:44,712 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:44,723 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:44,723 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:44,727 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:28:44,727 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:28:44,752 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:28:44,755 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:44,760 [520] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 28, 44, 757646), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 28, 44, 757708), u'quay', None, datetime.datetime(2020, 3, 30, 9, 28, 44, 757747), datetime.datetime(2020, 3, 24, 9, 28, 44, 757773), None]) gunicorn-web stdout | 2020-03-31 09:28:44,752 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:28:44,755 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:44,760 [520] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2020, 3, 31, 9, 28, 44, 757646), None, u'quay', u'quay', datetime.datetime(2020, 3, 31, 9, 28, 44, 757708), u'quay', None, datetime.datetime(2020, 3, 30, 9, 28, 44, 757747), datetime.datetime(2020, 3, 24, 9, 28, 44, 757773), None]) 2020-03-31 09:28:44,828 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:44,828 [520] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:44,861 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:44,863 [520] [DEBUG] [app] Ending request: urn:request:33666a28-a307-4249-bf5b-857587282504 (/health/instance) 2020-03-31 09:28:44,863 [520] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:44,865 [520] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.019 162 0.019) 10.131.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.362 49 0.362) 2020-03-31 09:28:44,858 [534] [DEBUG] [app] Starting request: urn:request:3cc65fb0-ebf7-4c12-83fc-3320456a6851 (/c1/_internal_ping) 2020-03-31 09:28:44,859 [534] [DEBUG] [app] Ending request: urn:request:3cc65fb0-ebf7-4c12-83fc-3320456a6851 (/c1/_internal_ping) 2020-03-31 09:28:44,860 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:44,861 [520] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:44,863 [520] [DEBUG] [app] Ending request: urn:request:33666a28-a307-4249-bf5b-857587282504 (/health/instance) 2020-03-31 09:28:44,863 [520] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:44,865 [520] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.019 162 0.019) 10.131.0.1 () - - [31/Mar/2020:09:28:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.362 49 0.362) gunicorn-verbs stdout | 2020-03-31 09:28:44,858 [534] [DEBUG] [app] Starting request: urn:request:3cc65fb0-ebf7-4c12-83fc-3320456a6851 (/c1/_internal_ping) 2020-03-31 09:28:44,859 [534] [DEBUG] [app] Ending request: urn:request:3cc65fb0-ebf7-4c12-83fc-3320456a6851 (/c1/_internal_ping) 2020-03-31 09:28:44,860 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:45,072 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:28:45,072 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:45,072 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:15.064514+00:00 (in 29.991956 seconds) gcworker stdout | 2020-03-31 09:28:45,072 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:15.064514+00:00 (in 29.991956 seconds) 2020-03-31 09:28:45,075 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:15 UTC)" (scheduled at 2020-03-31 09:28:45.064514+00:00) gcworker stdout | 2020-03-31 09:28:45,075 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:15 UTC)" (scheduled at 2020-03-31 09:28:45.064514+00:00) 2020-03-31 09:28:45,077 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437325, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:28:45,077 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437325, 1209600, 500, 1, 0]) 2020-03-31 09:28:45,207 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:28:45,207 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:45,207 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:28:45,207 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:28:45,207 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:45,207 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:15 UTC)" executed successfully 2020-03-31 09:28:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:28:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:46 UTC)" (scheduled at 2020-03-31 09:28:46.924497+00:00) 2020-03-31 09:28:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:28:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:46 UTC)" (scheduled at 2020-03-31 09:28:46.924497+00:00) 2020-03-31 09:28:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 46, 926733), True, datetime.datetime(2020, 3, 31, 9, 28, 46, 926733), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:28:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 46, 926733), True, datetime.datetime(2020, 3, 31, 9, 28, 46, 926733), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:28:46,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:56.924497+00:00 (in 9.998409 seconds) storagereplication stdout | 2020-03-31 09:28:46,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:56.924497+00:00 (in 9.998409 seconds) 2020-03-31 09:28:46,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:28:46,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:46,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:28:46,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:28:56 UTC)" executed successfully 2020-03-31 09:28:48,272 [518] [DEBUG] [app] Starting request: urn:request:87f88eb2-a5f8-41c5-8ef1-30d09989664a (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:48,272 [518] [DEBUG] [app] Starting request: urn:request:87f88eb2-a5f8-41c5-8ef1-30d09989664a (/health/instance) 2020-03-31 09:28:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:48,280 [552] [DEBUG] [app] Starting request: urn:request:ebc5ce4e-1270-4379-8e78-6dee3798f7d1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:48,280 [552] [DEBUG] [app] Starting request: urn:request:ebc5ce4e-1270-4379-8e78-6dee3798f7d1 (/v1/_internal_ping) 2020-03-31 09:28:48,282 [552] [DEBUG] [app] Ending request: urn:request:ebc5ce4e-1270-4379-8e78-6dee3798f7d1 (/v1/_internal_ping) 2020-03-31 09:28:48,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:28:48,282 [552] [DEBUG] [app] Ending request: urn:request:ebc5ce4e-1270-4379-8e78-6dee3798f7d1 (/v1/_internal_ping) 2020-03-31 09:28:48,284 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:28:48,282 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:48,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:48,284 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:48,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:28:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:28:48,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:48,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:48,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:48,322 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:48,322 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:48,326 [518] [DEBUG] [app] Starting request: urn:request:bef5095c-ed5e-43c1-9ff3-39879a9a772c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:48,326 [518] [DEBUG] [app] Starting request: urn:request:bef5095c-ed5e-43c1-9ff3-39879a9a772c (/_internal_ping) 2020-03-31 09:28:48,327 [518] [DEBUG] [app] Ending request: urn:request:bef5095c-ed5e-43c1-9ff3-39879a9a772c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:48,327 [518] [DEBUG] [app] Ending request: urn:request:bef5095c-ed5e-43c1-9ff3-39879a9a772c (/_internal_ping) 2020-03-31 09:28:48,328 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:28:48,328 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:28:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:48,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:48,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:48,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:28:48,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:28:48,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:48,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:28:48,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:48,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:48,338 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:48,338 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:48,340 [537] [DEBUG] [app] Starting request: urn:request:9480e421-e521-4d6a-ad1a-664ffe9afb02 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:48,340 [537] [DEBUG] [app] Starting request: urn:request:9480e421-e521-4d6a-ad1a-664ffe9afb02 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:48,341 [537] [DEBUG] [app] Ending request: urn:request:9480e421-e521-4d6a-ad1a-664ffe9afb02 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:48,341 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:48,341 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:28:48,341 [537] [DEBUG] [app] Ending request: urn:request:9480e421-e521-4d6a-ad1a-664ffe9afb02 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:48,341 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:48,343 [518] [DEBUG] [app] Ending request: urn:request:87f88eb2-a5f8-41c5-8ef1-30d09989664a (/health/instance) 2020-03-31 09:28:48,343 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:28:48,341 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:48,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 09:28:48,343 [518] [DEBUG] [app] Ending request: urn:request:87f88eb2-a5f8-41c5-8ef1-30d09989664a (/health/instance) 2020-03-31 09:28:48,343 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:48,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:28:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:28:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:54.323352+00:00 (in 4.823525 seconds) 2020-03-31 09:28:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:49 UTC)" (scheduled at 2020-03-31 09:28:49.498966+00:00) 2020-03-31 09:28:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:28:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:54.323352+00:00 (in 4.823525 seconds) 2020-03-31 09:28:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:49 UTC)" (scheduled at 2020-03-31 09:28:49.498966+00:00) 2020-03-31 09:28:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:49 UTC)" executed successfully 2020-03-31 09:28:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:28:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:59.560326+00:00 (in 8.449017 seconds) 2020-03-31 09:28:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:51 UTC)" (scheduled at 2020-03-31 09:28:51.110610+00:00) 2020-03-31 09:28:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:28:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:28:59.560326+00:00 (in 8.449017 seconds) 2020-03-31 09:28:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:51 UTC)" (scheduled at 2020-03-31 09:28:51.110610+00:00) 2020-03-31 09:28:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 51, 112245), True, datetime.datetime(2020, 3, 31, 9, 28, 51, 112245), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:28:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 51, 112245), True, datetime.datetime(2020, 3, 31, 9, 28, 51, 112245), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:28:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:28:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:28:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:51 UTC)" executed successfully 2020-03-31 09:28:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:28:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:01.440707+00:00 (in 9.997986 seconds) 2020-03-31 09:28:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:01 UTC)" (scheduled at 2020-03-31 09:28:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:28:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:01.440707+00:00 (in 9.997986 seconds) 2020-03-31 09:28:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:01 UTC)" (scheduled at 2020-03-31 09:28:51.440707+00:00) 2020-03-31 09:28:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:28:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:28:51,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:51,458 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:28:51,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:28:51,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:51,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:28:51,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:01 UTC)" executed successfully 2020-03-31 09:28:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:28:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:22.113221+00:00 (in 29.999083 seconds) 2020-03-31 09:28:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:22 UTC)" (scheduled at 2020-03-31 09:28:52.113221+00:00) securityworker stdout | 2020-03-31 09:28:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:22.113221+00:00 (in 29.999083 seconds) 2020-03-31 09:28:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:22 UTC)" (scheduled at 2020-03-31 09:28:52.113221+00:00) 2020-03-31 09:28:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:28:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:28:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:28:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:28:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:28:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:28:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:28:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:22 UTC)" executed successfully 2020-03-31 09:28:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:28:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:04 UTC)" (scheduled at 2020-03-31 09:28:54.323352+00:00) 2020-03-31 09:28:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:28:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:04 UTC)" (scheduled at 2020-03-31 09:28:54.323352+00:00) 2020-03-31 09:28:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:04.323352+00:00 (in 9.998926 seconds) notificationworker stdout | 2020-03-31 09:28:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:04.323352+00:00 (in 9.998926 seconds) 2020-03-31 09:28:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 54, 325320), True, datetime.datetime(2020, 3, 31, 9, 28, 54, 325320), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:28:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 54, 325320), True, datetime.datetime(2020, 3, 31, 9, 28, 54, 325320), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:28:54,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:28:54,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:28:54,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:54,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:04 UTC)" executed successfully 2020-03-31 09:28:54,446 [518] [DEBUG] [app] Starting request: urn:request:ccb148a9-a095-46e4-8919-4cad577d46d9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:54,446 [518] [DEBUG] [app] Starting request: urn:request:ccb148a9-a095-46e4-8919-4cad577d46d9 (/health/instance) 2020-03-31 09:28:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:54,455 [549] [DEBUG] [app] Starting request: urn:request:ed6e4b4a-c9a2-4c30-aef5-fc79f3154b20 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:54,455 [549] [DEBUG] [app] Starting request: urn:request:ed6e4b4a-c9a2-4c30-aef5-fc79f3154b20 (/v1/_internal_ping) 2020-03-31 09:28:54,456 [549] [DEBUG] [app] Ending request: urn:request:ed6e4b4a-c9a2-4c30-aef5-fc79f3154b20 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:54,456 [549] [DEBUG] [app] Ending request: urn:request:ed6e4b4a-c9a2-4c30-aef5-fc79f3154b20 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:54,457 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:28:54,457 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:28:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:28:54,459 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:28:54,459 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:54,461 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:54,461 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:54,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:54,462 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:54,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:54,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:54,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:54,492 [527] [DEBUG] [app] Starting request: urn:request:b0bd9450-ed29-4165-b195-f0b82e7439c0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:54,492 [527] [DEBUG] [app] Starting request: urn:request:b0bd9450-ed29-4165-b195-f0b82e7439c0 (/_internal_ping) 2020-03-31 09:28:54,493 [527] [DEBUG] [app] Ending request: urn:request:b0bd9450-ed29-4165-b195-f0b82e7439c0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:54,493 [527] [DEBUG] [app] Ending request: urn:request:b0bd9450-ed29-4165-b195-f0b82e7439c0 (/_internal_ping) 2020-03-31 09:28:54,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:28:54,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:28:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:28:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:54,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:54,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:28:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:54,507 [534] [DEBUG] [app] Starting request: urn:request:5eb7fd20-fb72-49ea-b390-a9b4324f70b3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:54,507 [534] [DEBUG] [app] Starting request: urn:request:5eb7fd20-fb72-49ea-b390-a9b4324f70b3 (/c1/_internal_ping) 2020-03-31 09:28:54,508 [534] [DEBUG] [app] Ending request: urn:request:5eb7fd20-fb72-49ea-b390-a9b4324f70b3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:54,508 [534] [DEBUG] [app] Ending request: urn:request:5eb7fd20-fb72-49ea-b390-a9b4324f70b3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:28:54,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:54,508 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:54,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:28:54,508 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:54,510 [518] [DEBUG] [app] Ending request: urn:request:ccb148a9-a095-46e4-8919-4cad577d46d9 (/health/instance) 2020-03-31 09:28:54,510 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:54,510 [518] [DEBUG] [app] Ending request: urn:request:ccb148a9-a095-46e4-8919-4cad577d46d9 (/health/instance) 2020-03-31 09:28:54,510 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:28:54,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:28:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:28:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:06.924497+00:00 (in 9.995831 seconds) 2020-03-31 09:28:56,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:06 UTC)" (scheduled at 2020-03-31 09:28:56.924497+00:00) 2020-03-31 09:28:56,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:28:56,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:06.924497+00:00 (in 9.995831 seconds) 2020-03-31 09:28:56,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:06 UTC)" (scheduled at 2020-03-31 09:28:56.924497+00:00) 2020-03-31 09:28:56,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:28:56,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 56, 929395), True, datetime.datetime(2020, 3, 31, 9, 28, 56, 929395), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:28:56,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 28, 56, 929395), True, datetime.datetime(2020, 3, 31, 9, 28, 56, 929395), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:28:56,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:28:56,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:28:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:28:56,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:28:56,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:06 UTC)" executed successfully 2020-03-31 09:28:58,273 [526] [DEBUG] [app] Starting request: urn:request:03f74255-10ca-44ca-8175-b958ea8c558f (/health/instance) gunicorn-web stdout | 2020-03-31 09:28:58,273 [526] [DEBUG] [app] Starting request: urn:request:03f74255-10ca-44ca-8175-b958ea8c558f (/health/instance) 2020-03-31 09:28:58,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:58,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:58,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:58,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:58,281 [538] [DEBUG] [app] Starting request: urn:request:50cbcd6f-15c4-4574-80a7-ccb232e21ecf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:58,281 [538] [DEBUG] [app] Starting request: urn:request:50cbcd6f-15c4-4574-80a7-ccb232e21ecf (/v1/_internal_ping) 2020-03-31 09:28:58,282 [538] [DEBUG] [app] Ending request: urn:request:50cbcd6f-15c4-4574-80a7-ccb232e21ecf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:28:58,282 [538] [DEBUG] [app] Ending request: urn:request:50cbcd6f-15c4-4574-80a7-ccb232e21ecf (/v1/_internal_ping) 2020-03-31 09:28:58,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:58,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:28:58,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:58,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:28:58,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:28:58,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:28:58,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:28:58,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:28:58,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:28:58,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:28:58,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:28:58,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:28:58,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:28:58,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:28:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:28:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:28:58,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:58,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:58,315 [526] [DEBUG] [app] Starting request: urn:request:54d64c84-4e07-4f4e-b6a3-fd67c39b16cd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:58,315 [526] [DEBUG] [app] Starting request: urn:request:54d64c84-4e07-4f4e-b6a3-fd67c39b16cd (/_internal_ping) 2020-03-31 09:28:58,316 [526] [DEBUG] [app] Ending request: urn:request:54d64c84-4e07-4f4e-b6a3-fd67c39b16cd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:28:58,316 [526] [DEBUG] [app] Ending request: urn:request:54d64c84-4e07-4f4e-b6a3-fd67c39b16cd (/_internal_ping) 2020-03-31 09:28:58,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:28:58,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:58,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:28:58,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:28:58,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:58,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:28:58,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:28:58,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:28:58,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:58,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:28:58,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:28:58,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:28:58,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:28:58,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:28:58,330 [534] [DEBUG] [app] Starting request: urn:request:fe55a7d9-0bdf-4906-bf82-b04ab955dfc0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:58,330 [534] [DEBUG] [app] Starting request: urn:request:fe55a7d9-0bdf-4906-bf82-b04ab955dfc0 (/c1/_internal_ping) 2020-03-31 09:28:58,332 [534] [DEBUG] [app] Ending request: urn:request:fe55a7d9-0bdf-4906-bf82-b04ab955dfc0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:28:58,332 [534] [DEBUG] [app] Ending request: urn:request:fe55a7d9-0bdf-4906-bf82-b04ab955dfc0 (/c1/_internal_ping) 2020-03-31 09:28:58,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:28:58,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:28:58,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:28:58,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:28:58,334 [526] [DEBUG] [app] Ending request: urn:request:03f74255-10ca-44ca-8175-b958ea8c558f (/health/instance) 2020-03-31 09:28:58,334 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:28:58,334 [526] [DEBUG] [app] Ending request: urn:request:03f74255-10ca-44ca-8175-b958ea8c558f (/health/instance) 2020-03-31 09:28:58,334 [526] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:28:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:28:58,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:28:58,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:28:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:28:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:28:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:28:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:51.110610+00:00 (in 51.549279 seconds) 2020-03-31 09:28:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:59 UTC)" (scheduled at 2020-03-31 09:28:59.560326+00:00) 2020-03-31 09:28:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:28:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:51.110610+00:00 (in 51.549279 seconds) 2020-03-31 09:28:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:59 UTC)" (scheduled at 2020-03-31 09:28:59.560326+00:00) 2020-03-31 09:28:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:28:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:28:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:29:59 UTC)" executed successfully 2020-03-31 09:29:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:29:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:30.337699+00:00 (in 29.999120 seconds) 2020-03-31 09:29:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:30 UTC)" (scheduled at 2020-03-31 09:29:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:29:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:30.337699+00:00 (in 29.999120 seconds) 2020-03-31 09:29:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:30 UTC)" (scheduled at 2020-03-31 09:29:00.337699+00:00) 2020-03-31 09:29:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 29, 0, 339472), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:29:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 29, 0, 339472), False, 50, 1, 0]) 2020-03-31 09:29:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:29:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:29:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:29:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:30 UTC)" executed successfully 2020-03-31 09:29:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:11.440707+00:00 (in 9.998947 seconds) 2020-03-31 09:29:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:11 UTC)" (scheduled at 2020-03-31 09:29:01.440707+00:00) 2020-03-31 09:29:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:29:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:11.440707+00:00 (in 9.998947 seconds) 2020-03-31 09:29:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:11 UTC)" (scheduled at 2020-03-31 09:29:01.440707+00:00) 2020-03-31 09:29:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:29:01,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:29:01,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:29:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:29:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:11 UTC)" executed successfully 2020-03-31 09:29:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:29:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:23.410371+00:00 (in 20.079495 seconds) 2020-03-31 09:29:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:33 UTC)" (scheduled at 2020-03-31 09:29:03.330112+00:00) 2020-03-31 09:29:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:29:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:23.410371+00:00 (in 20.079495 seconds) 2020-03-31 09:29:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:33 UTC)" (scheduled at 2020-03-31 09:29:03.330112+00:00) 2020-03-31 09:29:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 3, 331827), True, datetime.datetime(2020, 3, 31, 9, 29, 3, 331827), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:29:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 3, 331827), True, datetime.datetime(2020, 3, 31, 9, 29, 3, 331827), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:29:03,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:29:03,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:33 UTC)" executed successfully 2020-03-31 09:29:04,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:29:04,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:04,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:14.323352+00:00 (in 9.996675 seconds) 2020-03-31 09:29:04,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:14 UTC)" (scheduled at 2020-03-31 09:29:04.323352+00:00) 2020-03-31 09:29:04,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:29:04,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:14.323352+00:00 (in 9.996675 seconds) 2020-03-31 09:29:04,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:14 UTC)" (scheduled at 2020-03-31 09:29:04.323352+00:00) 2020-03-31 09:29:04,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:04,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 4, 327302), True, datetime.datetime(2020, 3, 31, 9, 29, 4, 327302), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:29:04,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 4, 327302), True, datetime.datetime(2020, 3, 31, 9, 29, 4, 327302), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:29:04,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:29:04,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:04,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:04,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:29:04,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:04,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:14 UTC)" executed successfully 2020-03-31 09:29:04,456 [549] [DEBUG] [app] Starting request: urn:request:40760c24-a0b2-4fbf-912c-1db7e2150df4 (/v1/_internal_ping) 2020-03-31 09:29:04,447 [518] [DEBUG] [app] Starting request: urn:request:f5f0edce-c5b8-43e7-bd6c-820dc7d77107 (/health/instance) 2020-03-31 09:29:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:04,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-registry stdout | 2020-03-31 09:29:04,456 [549] [DEBUG] [app] Starting request: urn:request:40760c24-a0b2-4fbf-912c-1db7e2150df4 (/v1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:04,447 [518] [DEBUG] [app] Starting request: urn:request:f5f0edce-c5b8-43e7-bd6c-820dc7d77107 (/health/instance) 2020-03-31 09:29:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:04,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:04,458 [549] [DEBUG] [app] Ending request: urn:request:40760c24-a0b2-4fbf-912c-1db7e2150df4 (/v1/_internal_ping) 2020-03-31 09:29:04,461 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:04,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:04,462 [518] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2020-03-31 09:29:04,458 [549] [DEBUG] [app] Ending request: urn:request:40760c24-a0b2-4fbf-912c-1db7e2150df4 (/v1/_internal_ping) 2020-03-31 09:29:04,461 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:04,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:04,462 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) 2020-03-31 09:29:04,466 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:04,466 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:04,468 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:04,468 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:04,469 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:04,469 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:04,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:04,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:04,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:04,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:04,494 [527] [DEBUG] [app] Starting request: urn:request:fdb9ae45-1006-45f7-a9a5-ec07dcbe0591 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:04,494 [527] [DEBUG] [app] Starting request: urn:request:fdb9ae45-1006-45f7-a9a5-ec07dcbe0591 (/_internal_ping) 2020-03-31 09:29:04,495 [527] [DEBUG] [app] Ending request: urn:request:fdb9ae45-1006-45f7-a9a5-ec07dcbe0591 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:04,495 [527] [DEBUG] [app] Ending request: urn:request:fdb9ae45-1006-45f7-a9a5-ec07dcbe0591 (/_internal_ping) 2020-03-31 09:29:04,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:29:04,496 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:29:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:29:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:29:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:29:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:04,509 [537] [DEBUG] [app] Starting request: urn:request:2ed3d28d-829f-4d1c-92b0-6a9c3aee0831 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:04,509 [537] [DEBUG] [app] Starting request: urn:request:2ed3d28d-829f-4d1c-92b0-6a9c3aee0831 (/c1/_internal_ping) 2020-03-31 09:29:04,509 [537] [DEBUG] [app] Ending request: urn:request:2ed3d28d-829f-4d1c-92b0-6a9c3aee0831 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:04,509 [537] [DEBUG] [app] Ending request: urn:request:2ed3d28d-829f-4d1c-92b0-6a9c3aee0831 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:04,510 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:04,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:04,510 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:04,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:04,511 [518] [DEBUG] [app] Ending request: urn:request:f5f0edce-c5b8-43e7-bd6c-820dc7d77107 (/health/instance) 2020-03-31 09:29:04,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:04,511 [518] [DEBUG] [app] Ending request: urn:request:f5f0edce-c5b8-43e7-bd6c-820dc7d77107 (/health/instance) 2020-03-31 09:29:04,512 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:04,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:29:04,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:29:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:29:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:16.924497+00:00 (in 9.998998 seconds) 2020-03-31 09:29:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:16 UTC)" (scheduled at 2020-03-31 09:29:06.924497+00:00) 2020-03-31 09:29:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:29:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:16.924497+00:00 (in 9.998998 seconds) 2020-03-31 09:29:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:16 UTC)" (scheduled at 2020-03-31 09:29:06.924497+00:00) 2020-03-31 09:29:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 6, 926265), True, datetime.datetime(2020, 3, 31, 9, 29, 6, 926265), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:29:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 6, 926265), True, datetime.datetime(2020, 3, 31, 9, 29, 6, 926265), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:29:06,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:29:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:29:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:16 UTC)" executed successfully 2020-03-31 09:29:08,273 [518] [DEBUG] [app] Starting request: urn:request:12cc14e8-5b0e-42af-9041-3ff8b4e6c70e (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:08,273 [518] [DEBUG] [app] Starting request: urn:request:12cc14e8-5b0e-42af-9041-3ff8b4e6c70e (/health/instance) 2020-03-31 09:29:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:08,278 [553] [DEBUG] [app] Starting request: urn:request:1d4e976d-a82e-4eb9-b033-5f531051b0ed (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:08,278 [553] [DEBUG] [app] Starting request: urn:request:1d4e976d-a82e-4eb9-b033-5f531051b0ed (/v1/_internal_ping) 2020-03-31 09:29:08,279 [553] [DEBUG] [app] Ending request: urn:request:1d4e976d-a82e-4eb9-b033-5f531051b0ed (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:08,279 [553] [DEBUG] [app] Ending request: urn:request:1d4e976d-a82e-4eb9-b033-5f531051b0ed (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:08,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:29:08,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:08,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:08,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:08,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:08,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:08,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:08,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:08,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:08,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:08,311 [518] [DEBUG] [app] Starting request: urn:request:cf82ec44-46dc-4e46-9986-10358ce23df9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:08,311 [518] [DEBUG] [app] Starting request: urn:request:cf82ec44-46dc-4e46-9986-10358ce23df9 (/_internal_ping) 2020-03-31 09:29:08,312 [518] [DEBUG] [app] Ending request: urn:request:cf82ec44-46dc-4e46-9986-10358ce23df9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:08,312 [518] [DEBUG] [app] Ending request: urn:request:cf82ec44-46dc-4e46-9986-10358ce23df9 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:29:08,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:29:08,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:08,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:08,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:29:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:29:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:29:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:29:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:08,325 [543] [DEBUG] [app] Starting request: urn:request:b479adf5-5bfb-4b28-b976-3264ba4f4a64 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:08,325 [543] [DEBUG] [app] Starting request: urn:request:b479adf5-5bfb-4b28-b976-3264ba4f4a64 (/c1/_internal_ping) 2020-03-31 09:29:08,326 [543] [DEBUG] [app] Ending request: urn:request:b479adf5-5bfb-4b28-b976-3264ba4f4a64 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:08,326 [543] [DEBUG] [app] Ending request: urn:request:b479adf5-5bfb-4b28-b976-3264ba4f4a64 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:08,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:08,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:08,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:08,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:08,328 [518] [DEBUG] [app] Ending request: urn:request:12cc14e8-5b0e-42af-9041-3ff8b4e6c70e (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:08,328 [518] [DEBUG] [app] Ending request: urn:request:12cc14e8-5b0e-42af-9041-3ff8b4e6c70e (/health/instance) 2020-03-31 09:29:08,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:08,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:08,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:29:08,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:29:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:29:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:21.440707+00:00 (in 9.999092 seconds) 2020-03-31 09:29:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:21 UTC)" (scheduled at 2020-03-31 09:29:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:29:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:21.440707+00:00 (in 9.999092 seconds) 2020-03-31 09:29:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:21 UTC)" (scheduled at 2020-03-31 09:29:11.440707+00:00) 2020-03-31 09:29:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:29:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:29:11,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:29:11,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:29:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:29:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:21 UTC)" executed successfully 2020-03-31 09:29:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:29:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:24.323352+00:00 (in 9.999337 seconds) 2020-03-31 09:29:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:24 UTC)" (scheduled at 2020-03-31 09:29:14.323352+00:00) 2020-03-31 09:29:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:29:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:24.323352+00:00 (in 9.999337 seconds) 2020-03-31 09:29:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:24 UTC)" (scheduled at 2020-03-31 09:29:14.323352+00:00) 2020-03-31 09:29:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 14, 324982), True, datetime.datetime(2020, 3, 31, 9, 29, 14, 324982), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:29:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 14, 324982), True, datetime.datetime(2020, 3, 31, 9, 29, 14, 324982), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:29:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:29:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:29:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:24 UTC)" executed successfully 2020-03-31 09:29:14,446 [527] [DEBUG] [app] Starting request: urn:request:7bff705e-99c5-46b9-ad5e-009affba914f (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:14,446 [527] [DEBUG] [app] Starting request: urn:request:7bff705e-99c5-46b9-ad5e-009affba914f (/health/instance) 2020-03-31 09:29:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:14,454 [553] [DEBUG] [app] Starting request: urn:request:536c11e9-48dd-42b7-be54-203c03f27edf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:14,454 [553] [DEBUG] [app] Starting request: urn:request:536c11e9-48dd-42b7-be54-203c03f27edf (/v1/_internal_ping) 2020-03-31 09:29:14,455 [553] [DEBUG] [app] Ending request: urn:request:536c11e9-48dd-42b7-be54-203c03f27edf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:14,455 [553] [DEBUG] [app] Ending request: urn:request:536c11e9-48dd-42b7-be54-203c03f27edf (/v1/_internal_ping) 2020-03-31 09:29:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:29:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:29:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:29:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:14,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:14,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:14,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:14,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:14,496 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:14,496 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:14,501 [521] [DEBUG] [app] Starting request: urn:request:f024990e-dbbd-47ba-9e8c-09399babb77f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:14,501 [521] [DEBUG] [app] Starting request: urn:request:f024990e-dbbd-47ba-9e8c-09399babb77f (/_internal_ping) 2020-03-31 09:29:14,502 [521] [DEBUG] [app] Ending request: urn:request:f024990e-dbbd-47ba-9e8c-09399babb77f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:14,502 [521] [DEBUG] [app] Ending request: urn:request:f024990e-dbbd-47ba-9e8c-09399babb77f (/_internal_ping) 2020-03-31 09:29:14,503 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:14,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:29:14,503 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:14,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:29:14,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:14,507 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:14,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:14,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:14,511 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:14,511 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:14,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:14,512 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:14,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:14,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:14,517 [537] [DEBUG] [app] Starting request: urn:request:af728702-d411-40bf-8141-76a1080976dd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:14,517 [537] [DEBUG] [app] Starting request: urn:request:af728702-d411-40bf-8141-76a1080976dd (/c1/_internal_ping) 2020-03-31 09:29:14,518 [537] [DEBUG] [app] Ending request: urn:request:af728702-d411-40bf-8141-76a1080976dd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:14,518 [537] [DEBUG] [app] Ending request: urn:request:af728702-d411-40bf-8141-76a1080976dd (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:14,519 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:14,518 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:14,519 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:14,518 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:14,521 [527] [DEBUG] [app] Ending request: urn:request:7bff705e-99c5-46b9-ad5e-009affba914f (/health/instance) 2020-03-31 09:29:14,521 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:14,521 [527] [DEBUG] [app] Ending request: urn:request:7bff705e-99c5-46b9-ad5e-009affba914f (/health/instance) 2020-03-31 09:29:14,521 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) 2020-03-31 09:29:14,522 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:29:14,522 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:29:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:29:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:45.064514+00:00 (in 29.998988 seconds) 2020-03-31 09:29:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:45 UTC)" (scheduled at 2020-03-31 09:29:15.064514+00:00) gcworker stdout | 2020-03-31 09:29:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:45.064514+00:00 (in 29.998988 seconds) 2020-03-31 09:29:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:45 UTC)" (scheduled at 2020-03-31 09:29:15.064514+00:00) 2020-03-31 09:29:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437355, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:29:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437355, 1209600, 500, 1, 0]) 2020-03-31 09:29:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:29:15,087 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:29:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:29:15,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:29:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:45 UTC)" executed successfully 2020-03-31 09:29:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:29:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:26.924497+00:00 (in 9.999253 seconds) 2020-03-31 09:29:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:26 UTC)" (scheduled at 2020-03-31 09:29:16.924497+00:00) 2020-03-31 09:29:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:29:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:26.924497+00:00 (in 9.999253 seconds) 2020-03-31 09:29:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:26 UTC)" (scheduled at 2020-03-31 09:29:16.924497+00:00) 2020-03-31 09:29:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 16, 925969), True, datetime.datetime(2020, 3, 31, 9, 29, 16, 925969), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:29:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 16, 925969), True, datetime.datetime(2020, 3, 31, 9, 29, 16, 925969), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:29:16,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:29:16,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:16,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:29:16,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:26 UTC)" executed successfully 2020-03-31 09:29:18,273 [527] [DEBUG] [app] Starting request: urn:request:0c1a5c14-f287-49a4-aa59-ee88be43c0a1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:18,273 [527] [DEBUG] [app] Starting request: urn:request:0c1a5c14-f287-49a4-aa59-ee88be43c0a1 (/health/instance) 2020-03-31 09:29:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:18,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:18,278 [538] [DEBUG] [app] Starting request: urn:request:9cbb631f-daf0-4d21-87a2-c00a70ef114f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:18,278 [538] [DEBUG] [app] Starting request: urn:request:9cbb631f-daf0-4d21-87a2-c00a70ef114f (/v1/_internal_ping) 2020-03-31 09:29:18,279 [538] [DEBUG] [app] Ending request: urn:request:9cbb631f-daf0-4d21-87a2-c00a70ef114f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:18,279 [538] [DEBUG] [app] Ending request: urn:request:9cbb631f-daf0-4d21-87a2-c00a70ef114f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:18,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:18,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:29:18,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:18,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:18,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:18,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:18,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:18,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:18,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:18,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:18,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:18,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:18,322 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:18,322 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:18,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:18,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:18,328 [518] [DEBUG] [app] Starting request: urn:request:b9d91d4d-4bd5-4a65-9438-a3f665dfd941 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:18,328 [518] [DEBUG] [app] Starting request: urn:request:b9d91d4d-4bd5-4a65-9438-a3f665dfd941 (/_internal_ping) 2020-03-31 09:29:18,328 [518] [DEBUG] [app] Ending request: urn:request:b9d91d4d-4bd5-4a65-9438-a3f665dfd941 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:18,328 [518] [DEBUG] [app] Ending request: urn:request:b9d91d4d-4bd5-4a65-9438-a3f665dfd941 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:29:18,329 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:29:18,329 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:18,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:18,333 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:18,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:18,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:18,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:18,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:18,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:29:18,337 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:18,341 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:18,341 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:18,342 [537] [DEBUG] [app] Starting request: urn:request:8d98c99a-a489-4cdd-8058-6e72d1d80f72 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:18,342 [537] [DEBUG] [app] Starting request: urn:request:8d98c99a-a489-4cdd-8058-6e72d1d80f72 (/c1/_internal_ping) 2020-03-31 09:29:18,343 [537] [DEBUG] [app] Ending request: urn:request:8d98c99a-a489-4cdd-8058-6e72d1d80f72 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:18,343 [537] [DEBUG] [app] Ending request: urn:request:8d98c99a-a489-4cdd-8058-6e72d1d80f72 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:18,344 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:18,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:18,344 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:18,344 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:18,345 [527] [DEBUG] [app] Ending request: urn:request:0c1a5c14-f287-49a4-aa59-ee88be43c0a1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:18,345 [527] [DEBUG] [app] Ending request: urn:request:0c1a5c14-f287-49a4-aa59-ee88be43c0a1 (/health/instance) 2020-03-31 09:29:18,345 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:18,345 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:18,346 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) gunicorn-web stdout | 2020-03-31 09:29:18,346 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) 2020-03-31 09:29:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:29:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:31.440707+00:00 (in 9.998724 seconds) 2020-03-31 09:29:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:31 UTC)" (scheduled at 2020-03-31 09:29:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:29:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:31.440707+00:00 (in 9.998724 seconds) 2020-03-31 09:29:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:31 UTC)" (scheduled at 2020-03-31 09:29:21.440707+00:00) 2020-03-31 09:29:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:29:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:29:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:29:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:29:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:29:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:31 UTC)" executed successfully 2020-03-31 09:29:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:29:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:52.113221+00:00 (in 29.998937 seconds) 2020-03-31 09:29:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:52 UTC)" (scheduled at 2020-03-31 09:29:22.113221+00:00) securityworker stdout | 2020-03-31 09:29:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:52.113221+00:00 (in 29.998937 seconds) 2020-03-31 09:29:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:52 UTC)" (scheduled at 2020-03-31 09:29:22.113221+00:00) 2020-03-31 09:29:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:29:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:29:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:29:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:29:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:29:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:29:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:29:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:29:52 UTC)" executed successfully 2020-03-31 09:29:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:29:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:24.410927+00:00 (in 0.999567 seconds) 2020-03-31 09:29:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:23 UTC)" (scheduled at 2020-03-31 09:29:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:29:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:24.410927+00:00 (in 0.999567 seconds) 2020-03-31 09:29:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:23 UTC)" (scheduled at 2020-03-31 09:29:23.410371+00:00) 2020-03-31 09:29:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:29:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), u'security_notification/%']) 2020-03-31 09:29:23,428 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), True, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:29:23,428 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), True, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), 0, u'security_notification/%']) 2020-03-31 09:29:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), True, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:29:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), True, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 29, 23, 412150), u'security_notification/%']) 2020-03-31 09:29:23,437 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:29:23,437 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:23,439 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:29:23,439 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:23,440 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:29:23,440 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:23,441 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:23 UTC)" executed successfully 2020-03-31 09:29:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:29:23,441 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:23 UTC)" executed successfully 2020-03-31 09:29:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:29:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:29:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:34.323352+00:00 (in 9.998904 seconds) 2020-03-31 09:29:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:34 UTC)" (scheduled at 2020-03-31 09:29:24.323352+00:00) 2020-03-31 09:29:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:29:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:34.323352+00:00 (in 9.998904 seconds) 2020-03-31 09:29:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:34 UTC)" (scheduled at 2020-03-31 09:29:24.323352+00:00) 2020-03-31 09:29:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 24, 325418), True, datetime.datetime(2020, 3, 31, 9, 29, 24, 325418), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:29:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 24, 325418), True, datetime.datetime(2020, 3, 31, 9, 29, 24, 325418), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:29:24,355 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:29:24,355 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:24,356 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:24,356 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:29:24,356 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:24,356 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:34 UTC)" executed successfully 2020-03-31 09:29:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:29:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:33.330112+00:00 (in 8.918317 seconds) 2020-03-31 09:29:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:24 UTC)" (scheduled at 2020-03-31 09:29:24.410927+00:00) 2020-03-31 09:29:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:29:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:33.330112+00:00 (in 8.918317 seconds) 2020-03-31 09:29:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:24 UTC)" (scheduled at 2020-03-31 09:29:24.410927+00:00) 2020-03-31 09:29:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:29:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:24 UTC)" executed successfully 2020-03-31 09:29:24,446 [518] [DEBUG] [app] Starting request: urn:request:1e826253-6812-439f-8aef-01041cfab6d6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:24,446 [518] [DEBUG] [app] Starting request: urn:request:1e826253-6812-439f-8aef-01041cfab6d6 (/health/instance) 2020-03-31 09:29:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:24,454 [556] [DEBUG] [app] Starting request: urn:request:9d71be3a-aa2f-472b-b34b-17bf14bde7a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:24,454 [556] [DEBUG] [app] Starting request: urn:request:9d71be3a-aa2f-472b-b34b-17bf14bde7a0 (/v1/_internal_ping) 2020-03-31 09:29:24,455 [556] [DEBUG] [app] Ending request: urn:request:9d71be3a-aa2f-472b-b34b-17bf14bde7a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:24,455 [556] [DEBUG] [app] Ending request: urn:request:9d71be3a-aa2f-472b-b34b-17bf14bde7a0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:24,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:29:24,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:24,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:24,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:24,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:24,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:24,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:24,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:24,488 [518] [DEBUG] [app] Starting request: urn:request:a62321b9-6f45-499a-84e9-cc0605ef5e34 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:24,488 [518] [DEBUG] [app] Starting request: urn:request:a62321b9-6f45-499a-84e9-cc0605ef5e34 (/_internal_ping) 2020-03-31 09:29:24,489 [518] [DEBUG] [app] Ending request: urn:request:a62321b9-6f45-499a-84e9-cc0605ef5e34 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:24,489 [518] [DEBUG] [app] Ending request: urn:request:a62321b9-6f45-499a-84e9-cc0605ef5e34 (/_internal_ping) 2020-03-31 09:29:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:29:24,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:29:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:29:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:24,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:24,505 [543] [DEBUG] [app] Starting request: urn:request:3d3e9d75-8a5f-46f1-a99e-65c5396725ed (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:24,505 [543] [DEBUG] [app] Starting request: urn:request:3d3e9d75-8a5f-46f1-a99e-65c5396725ed (/c1/_internal_ping) 2020-03-31 09:29:24,506 [543] [DEBUG] [app] Ending request: urn:request:3d3e9d75-8a5f-46f1-a99e-65c5396725ed (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:24,506 [543] [DEBUG] [app] Ending request: urn:request:3d3e9d75-8a5f-46f1-a99e-65c5396725ed (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:24,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:24,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:24,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:24,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:24,508 [518] [DEBUG] [app] Ending request: urn:request:1e826253-6812-439f-8aef-01041cfab6d6 (/health/instance) 2020-03-31 09:29:24,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:24,508 [518] [DEBUG] [app] Ending request: urn:request:1e826253-6812-439f-8aef-01041cfab6d6 (/health/instance) 2020-03-31 09:29:24,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:24,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:29:24,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:29:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:36.924497+00:00 (in 9.997982 seconds) 2020-03-31 09:29:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:36 UTC)" (scheduled at 2020-03-31 09:29:26.924497+00:00) 2020-03-31 09:29:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:29:26,926 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:29:26,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:36.924497+00:00 (in 9.997982 seconds) 2020-03-31 09:29:26,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:36 UTC)" (scheduled at 2020-03-31 09:29:26.924497+00:00) 2020-03-31 09:29:26,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 26, 927446), True, datetime.datetime(2020, 3, 31, 9, 29, 26, 927446), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:29:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 26, 927446), True, datetime.datetime(2020, 3, 31, 9, 29, 26, 927446), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:29:26,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:29:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:29:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:36 UTC)" executed successfully 2020-03-31 09:29:28,272 [518] [DEBUG] [app] Starting request: urn:request:62d06bad-af0c-4cd4-a43f-b15dc6edd441 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:28,272 [518] [DEBUG] [app] Starting request: urn:request:62d06bad-af0c-4cd4-a43f-b15dc6edd441 (/health/instance) 2020-03-31 09:29:28,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:28,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:28,277 [538] [DEBUG] [app] Starting request: urn:request:5489c905-8dbb-4900-82a1-9323c7ffef6d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:28,277 [538] [DEBUG] [app] Starting request: urn:request:5489c905-8dbb-4900-82a1-9323c7ffef6d (/v1/_internal_ping) 2020-03-31 09:29:28,278 [538] [DEBUG] [app] Ending request: urn:request:5489c905-8dbb-4900-82a1-9323c7ffef6d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:28,278 [538] [DEBUG] [app] Ending request: urn:request:5489c905-8dbb-4900-82a1-9323c7ffef6d (/v1/_internal_ping) 2020-03-31 09:29:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:29:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:29:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:28,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:28,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:28,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:28,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:28,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:28,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:28,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:28,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:28,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:28,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:28,310 [526] [DEBUG] [app] Starting request: urn:request:43a06368-ec60-4c48-8cad-ab2e74ff894f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:28,310 [526] [DEBUG] [app] Starting request: urn:request:43a06368-ec60-4c48-8cad-ab2e74ff894f (/_internal_ping) 2020-03-31 09:29:28,311 [526] [DEBUG] [app] Ending request: urn:request:43a06368-ec60-4c48-8cad-ab2e74ff894f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:28,311 [526] [DEBUG] [app] Ending request: urn:request:43a06368-ec60-4c48-8cad-ab2e74ff894f (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:29:28,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:29:28,312 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:29:28,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:29:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:28,325 [543] [DEBUG] [app] Starting request: urn:request:4ed9c665-a22d-45eb-8a2a-5954408b0ae1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:28,325 [543] [DEBUG] [app] Starting request: urn:request:4ed9c665-a22d-45eb-8a2a-5954408b0ae1 (/c1/_internal_ping) 2020-03-31 09:29:28,326 [543] [DEBUG] [app] Ending request: urn:request:4ed9c665-a22d-45eb-8a2a-5954408b0ae1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:28,326 [543] [DEBUG] [app] Ending request: urn:request:4ed9c665-a22d-45eb-8a2a-5954408b0ae1 (/c1/_internal_ping) 2020-03-31 09:29:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:28,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:29:28,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:28,328 [518] [DEBUG] [app] Ending request: urn:request:62d06bad-af0c-4cd4-a43f-b15dc6edd441 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:28,328 [518] [DEBUG] [app] Ending request: urn:request:62d06bad-af0c-4cd4-a43f-b15dc6edd441 (/health/instance) 2020-03-31 09:29:28,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:28,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:29:28,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:28,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:29:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:29:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:00.337699+00:00 (in 29.999133 seconds) 2020-03-31 09:29:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:00 UTC)" (scheduled at 2020-03-31 09:29:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:29:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:00.337699+00:00 (in 29.999133 seconds) 2020-03-31 09:29:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:00 UTC)" (scheduled at 2020-03-31 09:29:30.337699+00:00) 2020-03-31 09:29:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 29, 30, 339546), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:29:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 29, 30, 339546), False, 50, 1, 0]) 2020-03-31 09:29:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:29:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:29:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:29:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:00 UTC)" executed successfully 2020-03-31 09:29:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:29:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:41.440707+00:00 (in 9.999066 seconds) 2020-03-31 09:29:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:41 UTC)" (scheduled at 2020-03-31 09:29:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:29:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:41.440707+00:00 (in 9.999066 seconds) 2020-03-31 09:29:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:41 UTC)" (scheduled at 2020-03-31 09:29:31.440707+00:00) 2020-03-31 09:29:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:29:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:29:31,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:29:31,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:29:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:29:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:41 UTC)" executed successfully 2020-03-31 09:29:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:29:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:03.330112+00:00 (in 29.999019 seconds) 2020-03-31 09:29:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:03 UTC)" (scheduled at 2020-03-31 09:29:33.330112+00:00) 2020-03-31 09:29:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:29:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:03.330112+00:00 (in 29.999019 seconds) 2020-03-31 09:29:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:03 UTC)" (scheduled at 2020-03-31 09:29:33.330112+00:00) 2020-03-31 09:29:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 33, 331817), True, datetime.datetime(2020, 3, 31, 9, 29, 33, 331817), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:29:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 33, 331817), True, datetime.datetime(2020, 3, 31, 9, 29, 33, 331817), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:29:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:29:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:29:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:03 UTC)" executed successfully 2020-03-31 09:29:33,375 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:29:33,375 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:33,376 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:43.539999+00:00 (in 10.163837 seconds) 2020-03-31 09:29:33,376 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:33 UTC)" (scheduled at 2020-03-31 09:29:33.366791+00:00) 2020-03-31 09:29:33,376 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:29:33,376 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:43.539999+00:00 (in 10.163837 seconds) 2020-03-31 09:29:33,376 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:33 UTC)" (scheduled at 2020-03-31 09:29:33.366791+00:00) 2020-03-31 09:29:33,376 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:33,378 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 33, 376952), True, datetime.datetime(2020, 3, 31, 9, 29, 33, 376952), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:29:33,378 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 33, 376952), True, datetime.datetime(2020, 3, 31, 9, 29, 33, 376952), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:29:33,392 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:29:33,392 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:33,392 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:33,393 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:29:33,392 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:33,393 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:33 UTC)" executed successfully 2020-03-31 09:29:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:29:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:44.323352+00:00 (in 9.999136 seconds) 2020-03-31 09:29:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:44 UTC)" (scheduled at 2020-03-31 09:29:34.323352+00:00) 2020-03-31 09:29:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:29:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:44.323352+00:00 (in 9.999136 seconds) 2020-03-31 09:29:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:44 UTC)" (scheduled at 2020-03-31 09:29:34.323352+00:00) 2020-03-31 09:29:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 34, 325206), True, datetime.datetime(2020, 3, 31, 9, 29, 34, 325206), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:29:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 34, 325206), True, datetime.datetime(2020, 3, 31, 9, 29, 34, 325206), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:29:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:29:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:29:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:44 UTC)" executed successfully 2020-03-31 09:29:34,446 [518] [DEBUG] [app] Starting request: urn:request:314d6c89-07a7-4d6e-a865-5188a366df22 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:34,446 [518] [DEBUG] [app] Starting request: urn:request:314d6c89-07a7-4d6e-a865-5188a366df22 (/health/instance) 2020-03-31 09:29:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:34,454 [548] [DEBUG] [app] Starting request: urn:request:621aa276-7f4f-4345-b277-a94485a7b823 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:34,454 [548] [DEBUG] [app] Starting request: urn:request:621aa276-7f4f-4345-b277-a94485a7b823 (/v1/_internal_ping) 2020-03-31 09:29:34,456 [548] [DEBUG] [app] Ending request: urn:request:621aa276-7f4f-4345-b277-a94485a7b823 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:34,456 [548] [DEBUG] [app] Ending request: urn:request:621aa276-7f4f-4345-b277-a94485a7b823 (/v1/_internal_ping) 2020-03-31 09:29:34,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:29:34,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:34,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:34,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:29:34,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:34,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:34,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:34,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:34,461 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:34,461 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:34,462 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:34,462 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:34,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:34,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:34,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:34,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:34,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:34,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:34,487 [523] [DEBUG] [app] Starting request: urn:request:dac1e3df-9a56-49db-b4e6-d236d587f252 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:34,487 [523] [DEBUG] [app] Starting request: urn:request:dac1e3df-9a56-49db-b4e6-d236d587f252 (/_internal_ping) 2020-03-31 09:29:34,488 [523] [DEBUG] [app] Ending request: urn:request:dac1e3df-9a56-49db-b4e6-d236d587f252 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:34,488 [523] [DEBUG] [app] Ending request: urn:request:dac1e3df-9a56-49db-b4e6-d236d587f252 (/_internal_ping) 2020-03-31 09:29:34,489 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:29:34,489 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:29:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:34,502 [536] [DEBUG] [app] Starting request: urn:request:e7dd9c37-9ca4-44c2-96c7-cb126bc52d46 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:34,502 [536] [DEBUG] [app] Starting request: urn:request:e7dd9c37-9ca4-44c2-96c7-cb126bc52d46 (/c1/_internal_ping) 2020-03-31 09:29:34,503 [536] [DEBUG] [app] Ending request: urn:request:e7dd9c37-9ca4-44c2-96c7-cb126bc52d46 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:34,503 [536] [DEBUG] [app] Ending request: urn:request:e7dd9c37-9ca4-44c2-96c7-cb126bc52d46 (/c1/_internal_ping) 2020-03-31 09:29:34,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:34,504 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:34,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:34,505 [518] [DEBUG] [app] Ending request: urn:request:314d6c89-07a7-4d6e-a865-5188a366df22 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:29:34,504 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:34,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:34,505 [518] [DEBUG] [app] Ending request: urn:request:314d6c89-07a7-4d6e-a865-5188a366df22 (/health/instance) 2020-03-31 09:29:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:29:34,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:29:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:38.010934+00:00 (in 1.085451 seconds) 2020-03-31 09:29:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:46 UTC)" (scheduled at 2020-03-31 09:29:36.924497+00:00) 2020-03-31 09:29:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 36, 926348), True, datetime.datetime(2020, 3, 31, 9, 29, 36, 926348), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:29:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:38.010934+00:00 (in 1.085451 seconds) 2020-03-31 09:29:36,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:46 UTC)" (scheduled at 2020-03-31 09:29:36.924497+00:00) 2020-03-31 09:29:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 36, 926348), True, datetime.datetime(2020, 3, 31, 9, 29, 36, 926348), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:29:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:29:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:29:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:36,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:46 UTC)" executed successfully 2020-03-31 09:29:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:46.924497+00:00 (in 8.912601 seconds) 2020-03-31 09:29:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:38 UTC)" (scheduled at 2020-03-31 09:29:38.010934+00:00) 2020-03-31 09:29:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:29:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:46.924497+00:00 (in 8.912601 seconds) 2020-03-31 09:29:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:38 UTC)" (scheduled at 2020-03-31 09:29:38.010934+00:00) 2020-03-31 09:29:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:38 UTC)" executed successfully 2020-03-31 09:29:38,273 [518] [DEBUG] [app] Starting request: urn:request:f62a37e7-bbe5-4fcd-a9a4-a74841d7c942 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:38,273 [518] [DEBUG] [app] Starting request: urn:request:f62a37e7-bbe5-4fcd-a9a4-a74841d7c942 (/health/instance) 2020-03-31 09:29:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:38,279 [538] [DEBUG] [app] Starting request: urn:request:fb0042d2-862b-4b5d-be84-1d6475e70b27 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:38,279 [538] [DEBUG] [app] Starting request: urn:request:fb0042d2-862b-4b5d-be84-1d6475e70b27 (/v1/_internal_ping) 2020-03-31 09:29:38,280 [538] [DEBUG] [app] Ending request: urn:request:fb0042d2-862b-4b5d-be84-1d6475e70b27 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:38,280 [538] [DEBUG] [app] Ending request: urn:request:fb0042d2-862b-4b5d-be84-1d6475e70b27 (/v1/_internal_ping) 2020-03-31 09:29:38,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:29:38,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:29:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:29:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:29:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:38,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:38,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:38,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:38,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:38,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:38,312 [518] [DEBUG] [app] Starting request: urn:request:907c6043-f873-416d-902f-e0a6fd2d6975 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:38,312 [518] [DEBUG] [app] Starting request: urn:request:907c6043-f873-416d-902f-e0a6fd2d6975 (/_internal_ping) 2020-03-31 09:29:38,312 [518] [DEBUG] [app] Ending request: urn:request:907c6043-f873-416d-902f-e0a6fd2d6975 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:38,312 [518] [DEBUG] [app] Ending request: urn:request:907c6043-f873-416d-902f-e0a6fd2d6975 (/_internal_ping) 2020-03-31 09:29:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:29:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:29:38,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:38,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:29:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:38,326 [543] [DEBUG] [app] Starting request: urn:request:aa4d27d2-bb3c-474a-87a0-5a6d232b6ba2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:38,326 [543] [DEBUG] [app] Starting request: urn:request:aa4d27d2-bb3c-474a-87a0-5a6d232b6ba2 (/c1/_internal_ping) 2020-03-31 09:29:38,326 [543] [DEBUG] [app] Ending request: urn:request:aa4d27d2-bb3c-474a-87a0-5a6d232b6ba2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:38,326 [543] [DEBUG] [app] Ending request: urn:request:aa4d27d2-bb3c-474a-87a0-5a6d232b6ba2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:38,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:38,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:38,329 [518] [DEBUG] [app] Ending request: urn:request:f62a37e7-bbe5-4fcd-a9a4-a74841d7c942 (/health/instance) 2020-03-31 09:29:38,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:38,329 [518] [DEBUG] [app] Ending request: urn:request:f62a37e7-bbe5-4fcd-a9a4-a74841d7c942 (/health/instance) 2020-03-31 09:29:38,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:38,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:29:38,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:29:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:29:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:51.440707+00:00 (in 9.998740 seconds) 2020-03-31 09:29:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:51 UTC)" (scheduled at 2020-03-31 09:29:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:29:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:51.440707+00:00 (in 9.998740 seconds) 2020-03-31 09:29:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:51 UTC)" (scheduled at 2020-03-31 09:29:41.440707+00:00) 2020-03-31 09:29:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:29:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:29:41,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:29:41,465 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:29:41,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:29:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:41,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:29:41,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:51 UTC)" executed successfully 2020-03-31 09:29:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:29:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:14.539408+00:00 (in 30.998629 seconds) 2020-03-31 09:29:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:43 UTC)" (scheduled at 2020-03-31 09:29:43.539999+00:00) 2020-03-31 09:29:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:29:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:14.539408+00:00 (in 30.998629 seconds) 2020-03-31 09:29:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:43 UTC)" (scheduled at 2020-03-31 09:29:43.539999+00:00) 2020-03-31 09:29:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:29:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:43 UTC)" executed successfully 2020-03-31 09:29:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:29:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:49.498966+00:00 (in 5.174542 seconds) 2020-03-31 09:29:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:54 UTC)" (scheduled at 2020-03-31 09:29:44.323352+00:00) 2020-03-31 09:29:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:29:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:49.498966+00:00 (in 5.174542 seconds) 2020-03-31 09:29:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:54 UTC)" (scheduled at 2020-03-31 09:29:44.323352+00:00) 2020-03-31 09:29:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 44, 325305), True, datetime.datetime(2020, 3, 31, 9, 29, 44, 325305), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:29:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 44, 325305), True, datetime.datetime(2020, 3, 31, 9, 29, 44, 325305), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:29:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:29:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:29:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:54 UTC)" executed successfully 2020-03-31 09:29:44,447 [518] [DEBUG] [app] Starting request: urn:request:08ce88a2-b684-4697-9182-987dcabca44f (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:44,447 [518] [DEBUG] [app] Starting request: urn:request:08ce88a2-b684-4697-9182-987dcabca44f (/health/instance) 2020-03-31 09:29:44,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:44,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:44,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:44,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:44,455 [553] [DEBUG] [app] Starting request: urn:request:af09b4a4-b82f-4c63-a7af-46bf5fe0641a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:44,455 [553] [DEBUG] [app] Starting request: urn:request:af09b4a4-b82f-4c63-a7af-46bf5fe0641a (/v1/_internal_ping) 2020-03-31 09:29:44,456 [553] [DEBUG] [app] Ending request: urn:request:af09b4a4-b82f-4c63-a7af-46bf5fe0641a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:44,456 [553] [DEBUG] [app] Ending request: urn:request:af09b4a4-b82f-4c63-a7af-46bf5fe0641a (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:29:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:44,459 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:29:44,459 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:44,461 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:44,461 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:44,463 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:44,463 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:44,463 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:44,463 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:44,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:44,490 [518] [DEBUG] [app] Starting request: urn:request:fc613c78-203b-4401-bda8-a99f8c04cfc6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:44,490 [518] [DEBUG] [app] Starting request: urn:request:fc613c78-203b-4401-bda8-a99f8c04cfc6 (/_internal_ping) 2020-03-31 09:29:44,491 [518] [DEBUG] [app] Ending request: urn:request:fc613c78-203b-4401-bda8-a99f8c04cfc6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:44,491 [518] [DEBUG] [app] Ending request: urn:request:fc613c78-203b-4401-bda8-a99f8c04cfc6 (/_internal_ping) 2020-03-31 09:29:44,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:29:44,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:29:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:29:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:29:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:29:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:44,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:44,506 [539] [DEBUG] [app] Starting request: urn:request:ec5c7e71-aaa7-4b40-ac4f-73af89753146 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:44,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:29:44,506 [539] [DEBUG] [app] Starting request: urn:request:ec5c7e71-aaa7-4b40-ac4f-73af89753146 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:44,507 [539] [DEBUG] [app] Ending request: urn:request:ec5c7e71-aaa7-4b40-ac4f-73af89753146 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:44,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:44,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:29:44,507 [539] [DEBUG] [app] Ending request: urn:request:ec5c7e71-aaa7-4b40-ac4f-73af89753146 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:44,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:44,509 [518] [DEBUG] [app] Ending request: urn:request:08ce88a2-b684-4697-9182-987dcabca44f (/health/instance) 2020-03-31 09:29:44,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:29:44,507 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:44,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:29:44,509 [518] [DEBUG] [app] Ending request: urn:request:08ce88a2-b684-4697-9182-987dcabca44f (/health/instance) 2020-03-31 09:29:44,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:44,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:29:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:29:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:15.064514+00:00 (in 29.998975 seconds) 2020-03-31 09:29:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:15 UTC)" (scheduled at 2020-03-31 09:29:45.064514+00:00) gcworker stdout | 2020-03-31 09:29:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:15.064514+00:00 (in 29.998975 seconds) 2020-03-31 09:29:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:15 UTC)" (scheduled at 2020-03-31 09:29:45.064514+00:00) 2020-03-31 09:29:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437385, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:29:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437385, 1209600, 500, 1, 0]) 2020-03-31 09:29:45,081 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:29:45,081 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:29:45,081 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:45,081 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:29:45,081 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:45,081 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:15 UTC)" executed successfully 2020-03-31 09:29:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:56.924497+00:00 (in 9.999125 seconds) 2020-03-31 09:29:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:56 UTC)" (scheduled at 2020-03-31 09:29:46.924497+00:00) 2020-03-31 09:29:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:29:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:29:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:56.924497+00:00 (in 9.999125 seconds) 2020-03-31 09:29:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:56 UTC)" (scheduled at 2020-03-31 09:29:46.924497+00:00) 2020-03-31 09:29:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 46, 926181), True, datetime.datetime(2020, 3, 31, 9, 29, 46, 926181), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:29:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 46, 926181), True, datetime.datetime(2020, 3, 31, 9, 29, 46, 926181), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:29:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:29:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:29:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:29:56 UTC)" executed successfully 2020-03-31 09:29:48,274 [518] [DEBUG] [app] Starting request: urn:request:1cf1c818-99cf-491d-8354-8dd1b7a645fd (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:48,274 [518] [DEBUG] [app] Starting request: urn:request:1cf1c818-99cf-491d-8354-8dd1b7a645fd (/health/instance) 2020-03-31 09:29:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:48,279 [556] [DEBUG] [app] Starting request: urn:request:125ed9fa-9cef-4c68-8a57-c8fda504ef37 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:48,279 [556] [DEBUG] [app] Starting request: urn:request:125ed9fa-9cef-4c68-8a57-c8fda504ef37 (/v1/_internal_ping) 2020-03-31 09:29:48,281 [556] [DEBUG] [app] Ending request: urn:request:125ed9fa-9cef-4c68-8a57-c8fda504ef37 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:48,281 [556] [DEBUG] [app] Ending request: urn:request:125ed9fa-9cef-4c68-8a57-c8fda504ef37 (/v1/_internal_ping) 2020-03-31 09:29:48,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:29:48,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:48,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:48,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:48,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:29:48,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:48,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:29:48,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:48,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:48,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:48,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:48,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:48,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:48,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:48,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:48,313 [521] [DEBUG] [app] Starting request: urn:request:319407ae-0aed-4d3f-81b7-8380967f1610 (/_internal_ping) 2020-03-31 09:29:48,314 [521] [DEBUG] [app] Ending request: urn:request:319407ae-0aed-4d3f-81b7-8380967f1610 (/_internal_ping) 2020-03-31 09:29:48,314 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:48,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:29:48,313 [521] [DEBUG] [app] Starting request: urn:request:319407ae-0aed-4d3f-81b7-8380967f1610 (/_internal_ping) 2020-03-31 09:29:48,314 [521] [DEBUG] [app] Ending request: urn:request:319407ae-0aed-4d3f-81b7-8380967f1610 (/_internal_ping) 2020-03-31 09:29:48,314 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:48,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:29:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:48,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:48,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:48,325 [543] [DEBUG] [app] Starting request: urn:request:9df7dc05-4e31-4912-b74b-93c8108fd242 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:48,325 [543] [DEBUG] [app] Starting request: urn:request:9df7dc05-4e31-4912-b74b-93c8108fd242 (/c1/_internal_ping) 2020-03-31 09:29:48,326 [543] [DEBUG] [app] Ending request: urn:request:9df7dc05-4e31-4912-b74b-93c8108fd242 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:48,326 [543] [DEBUG] [app] Ending request: urn:request:9df7dc05-4e31-4912-b74b-93c8108fd242 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:48,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:48,328 [518] [DEBUG] [app] Ending request: urn:request:1cf1c818-99cf-491d-8354-8dd1b7a645fd (/health/instance) 2020-03-31 09:29:48,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:48,328 [518] [DEBUG] [app] Ending request: urn:request:1cf1c818-99cf-491d-8354-8dd1b7a645fd (/health/instance) 2020-03-31 09:29:48,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:29:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:29:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:29:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:54.323352+00:00 (in 4.823594 seconds) 2020-03-31 09:29:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:49 UTC)" (scheduled at 2020-03-31 09:29:49.498966+00:00) 2020-03-31 09:29:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:29:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:54.323352+00:00 (in 4.823594 seconds) 2020-03-31 09:29:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:49 UTC)" (scheduled at 2020-03-31 09:29:49.498966+00:00) 2020-03-31 09:29:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:49 UTC)" executed successfully 2020-03-31 09:29:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:29:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:51 UTC)" (scheduled at 2020-03-31 09:29:51.110610+00:00) 2020-03-31 09:29:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:29:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:51 UTC)" (scheduled at 2020-03-31 09:29:51.110610+00:00) 2020-03-31 09:29:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:53.556282+00:00 (in 2.444257 seconds) namespacegcworker stdout | 2020-03-31 09:29:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:53.556282+00:00 (in 2.444257 seconds) 2020-03-31 09:29:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 51, 113013), True, datetime.datetime(2020, 3, 31, 9, 29, 51, 113013), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:29:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 51, 113013), True, datetime.datetime(2020, 3, 31, 9, 29, 51, 113013), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:29:51,135 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:29:51,135 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:51,135 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:51,135 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:29:51,135 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:51,135 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:51 UTC)" executed successfully 2020-03-31 09:29:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:29:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:01.440707+00:00 (in 9.998496 seconds) 2020-03-31 09:29:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:01 UTC)" (scheduled at 2020-03-31 09:29:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:29:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:01.440707+00:00 (in 9.998496 seconds) 2020-03-31 09:29:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:01 UTC)" (scheduled at 2020-03-31 09:29:51.440707+00:00) 2020-03-31 09:29:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:29:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:29:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:29:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:29:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:29:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:01 UTC)" executed successfully 2020-03-31 09:29:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:29:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:22.113221+00:00 (in 29.999285 seconds) securityworker stdout | 2020-03-31 09:29:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:22.113221+00:00 (in 29.999285 seconds) 2020-03-31 09:29:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:22 UTC)" (scheduled at 2020-03-31 09:29:52.113221+00:00) securityworker stdout | 2020-03-31 09:29:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:22 UTC)" (scheduled at 2020-03-31 09:29:52.113221+00:00) 2020-03-31 09:29:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:29:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:29:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:29:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:29:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:29:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:29:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:29:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:22 UTC)" executed successfully 2020-03-31 09:29:53,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:29:53,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:53,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:59.560326+00:00 (in 5.999838 seconds) 2020-03-31 09:29:53,561 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:53 UTC)" (scheduled at 2020-03-31 09:29:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:29:53,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:29:59.560326+00:00 (in 5.999838 seconds) 2020-03-31 09:29:53,561 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:53 UTC)" (scheduled at 2020-03-31 09:29:53.556282+00:00) 2020-03-31 09:29:53,563 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:29:53,563 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), u'namespacegc/%']) 2020-03-31 09:29:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), True, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:29:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), True, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), 0, u'namespacegc/%']) 2020-03-31 09:29:53,583 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), True, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:29:53,583 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), True, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 29, 53, 561722), u'namespacegc/%']) 2020-03-31 09:29:53,587 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:29:53,587 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:53,587 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:29:53,587 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:53 UTC)" executed successfully 2020-03-31 09:29:53,590 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:29:53,590 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:53,591 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:29:53,591 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:53,592 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:29:53,592 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:29:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:29:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:04.323352+00:00 (in 9.999237 seconds) 2020-03-31 09:29:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:04 UTC)" (scheduled at 2020-03-31 09:29:54.323352+00:00) 2020-03-31 09:29:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:29:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:04.323352+00:00 (in 9.999237 seconds) 2020-03-31 09:29:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:04 UTC)" (scheduled at 2020-03-31 09:29:54.323352+00:00) 2020-03-31 09:29:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 54, 325013), True, datetime.datetime(2020, 3, 31, 9, 29, 54, 325013), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:29:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 54, 325013), True, datetime.datetime(2020, 3, 31, 9, 29, 54, 325013), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:29:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:29:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:29:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:04 UTC)" executed successfully 2020-03-31 09:29:54,446 [527] [DEBUG] [app] Starting request: urn:request:b5bee049-9cbe-4b18-9294-d74f91b16ee5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:54,446 [527] [DEBUG] [app] Starting request: urn:request:b5bee049-9cbe-4b18-9294-d74f91b16ee5 (/health/instance) 2020-03-31 09:29:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:54,454 [538] [DEBUG] [app] Starting request: urn:request:be6ee408-c61f-4160-812b-e933698cce9e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:54,454 [538] [DEBUG] [app] Starting request: urn:request:be6ee408-c61f-4160-812b-e933698cce9e (/v1/_internal_ping) 2020-03-31 09:29:54,455 [538] [DEBUG] [app] Ending request: urn:request:be6ee408-c61f-4160-812b-e933698cce9e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:54,455 [538] [DEBUG] [app] Ending request: urn:request:be6ee408-c61f-4160-812b-e933698cce9e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:29:54,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:29:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:29:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:29:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:54,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:54,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:54,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:54,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:54,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:54,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:54,489 [527] [DEBUG] [app] Starting request: urn:request:8b6ea181-8e96-42c8-8b31-580a9aa33197 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:54,489 [527] [DEBUG] [app] Starting request: urn:request:8b6ea181-8e96-42c8-8b31-580a9aa33197 (/_internal_ping) 2020-03-31 09:29:54,490 [527] [DEBUG] [app] Ending request: urn:request:8b6ea181-8e96-42c8-8b31-580a9aa33197 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:54,490 [527] [DEBUG] [app] Ending request: urn:request:8b6ea181-8e96-42c8-8b31-580a9aa33197 (/_internal_ping) 2020-03-31 09:29:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:29:54,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:54,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:29:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:54,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:54,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:54,505 [543] [DEBUG] [app] Starting request: urn:request:b74f8f1a-d2a9-4778-a5f2-f598a42df781 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:54,505 [543] [DEBUG] [app] Starting request: urn:request:b74f8f1a-d2a9-4778-a5f2-f598a42df781 (/c1/_internal_ping) 2020-03-31 09:29:54,506 [543] [DEBUG] [app] Ending request: urn:request:b74f8f1a-d2a9-4778-a5f2-f598a42df781 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:54,506 [543] [DEBUG] [app] Ending request: urn:request:b74f8f1a-d2a9-4778-a5f2-f598a42df781 (/c1/_internal_ping) 2020-03-31 09:29:54,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:54,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:54,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:29:54,508 [527] [DEBUG] [app] Ending request: urn:request:b5bee049-9cbe-4b18-9294-d74f91b16ee5 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:29:54,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:54,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:29:54,508 [527] [DEBUG] [app] Ending request: urn:request:b5bee049-9cbe-4b18-9294-d74f91b16ee5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:54,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:29:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:29:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:06.924497+00:00 (in 9.999213 seconds) 2020-03-31 09:29:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:06 UTC)" (scheduled at 2020-03-31 09:29:56.924497+00:00) 2020-03-31 09:29:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:29:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:06.924497+00:00 (in 9.999213 seconds) 2020-03-31 09:29:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:06 UTC)" (scheduled at 2020-03-31 09:29:56.924497+00:00) 2020-03-31 09:29:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:29:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 56, 925977), True, datetime.datetime(2020, 3, 31, 9, 29, 56, 925977), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:29:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 29, 56, 925977), True, datetime.datetime(2020, 3, 31, 9, 29, 56, 925977), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:29:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:29:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:29:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:29:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:29:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:06 UTC)" executed successfully 2020-03-31 09:29:58,273 [527] [DEBUG] [app] Starting request: urn:request:c1836ed2-a20c-4955-834b-e25c7090d7fb (/health/instance) gunicorn-web stdout | 2020-03-31 09:29:58,273 [527] [DEBUG] [app] Starting request: urn:request:c1836ed2-a20c-4955-834b-e25c7090d7fb (/health/instance) 2020-03-31 09:29:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:58,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:58,278 [538] [DEBUG] [app] Starting request: urn:request:cf96f816-9e08-4a56-a0be-0447b2f201c5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:58,278 [538] [DEBUG] [app] Starting request: urn:request:cf96f816-9e08-4a56-a0be-0447b2f201c5 (/v1/_internal_ping) 2020-03-31 09:29:58,279 [538] [DEBUG] [app] Ending request: urn:request:cf96f816-9e08-4a56-a0be-0447b2f201c5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:29:58,279 [538] [DEBUG] [app] Ending request: urn:request:cf96f816-9e08-4a56-a0be-0447b2f201c5 (/v1/_internal_ping) 2020-03-31 09:29:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:29:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:29:58,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:29:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:29:58,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:29:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:29:58,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:29:58,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:29:58,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:29:58,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:29:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:29:58,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:29:58,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:29:58,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:29:58,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:58,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:58,312 [525] [DEBUG] [app] Starting request: urn:request:a4730e94-bc9e-422a-87ec-b1566a9d441b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:58,312 [525] [DEBUG] [app] Starting request: urn:request:a4730e94-bc9e-422a-87ec-b1566a9d441b (/_internal_ping) 2020-03-31 09:29:58,313 [525] [DEBUG] [app] Ending request: urn:request:a4730e94-bc9e-422a-87ec-b1566a9d441b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:29:58,313 [525] [DEBUG] [app] Ending request: urn:request:a4730e94-bc9e-422a-87ec-b1566a9d441b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:29:58,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:29:58,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:58,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:58,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:58,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:29:58,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:29:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:29:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:29:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:29:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:29:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:29:58,326 [539] [DEBUG] [app] Starting request: urn:request:f42f388b-4195-4bd0-b5c3-0622a49742ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:58,326 [539] [DEBUG] [app] Starting request: urn:request:f42f388b-4195-4bd0-b5c3-0622a49742ec (/c1/_internal_ping) 2020-03-31 09:29:58,327 [539] [DEBUG] [app] Ending request: urn:request:f42f388b-4195-4bd0-b5c3-0622a49742ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:29:58,327 [539] [DEBUG] [app] Ending request: urn:request:f42f388b-4195-4bd0-b5c3-0622a49742ec (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:29:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:29:58,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:29:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:29:58,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:29:58,330 [527] [DEBUG] [app] Ending request: urn:request:c1836ed2-a20c-4955-834b-e25c7090d7fb (/health/instance) 2020-03-31 09:29:58,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:29:58,330 [527] [DEBUG] [app] Ending request: urn:request:c1836ed2-a20c-4955-834b-e25c7090d7fb (/health/instance) 2020-03-31 09:29:58,330 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:29:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:29:58,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:29:58,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:29:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:29:59,563 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:29:59,563 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:29:59,564 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:51.110610+00:00 (in 51.546907 seconds) 2020-03-31 09:29:59,564 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:59 UTC)" (scheduled at 2020-03-31 09:29:59.560326+00:00) 2020-03-31 09:29:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:29:59,564 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:51.110610+00:00 (in 51.546907 seconds) 2020-03-31 09:29:59,564 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:59 UTC)" (scheduled at 2020-03-31 09:29:59.560326+00:00) 2020-03-31 09:29:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:29:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:59 UTC)" executed successfully 2020-03-31 09:30:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:30:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:30.337699+00:00 (in 29.998624 seconds) 2020-03-31 09:30:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:30 UTC)" (scheduled at 2020-03-31 09:30:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:30:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:30.337699+00:00 (in 29.998624 seconds) 2020-03-31 09:30:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:30 UTC)" (scheduled at 2020-03-31 09:30:00.337699+00:00) 2020-03-31 09:30:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 30, 0, 339982), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:30:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 30, 0, 339982), False, 50, 1, 0]) 2020-03-31 09:30:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:30:00,356 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:30:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:30:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:30:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:30 UTC)" executed successfully 2020-03-31 09:30:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:30:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:11.440707+00:00 (in 9.999002 seconds) 2020-03-31 09:30:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:11 UTC)" (scheduled at 2020-03-31 09:30:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:30:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:11.440707+00:00 (in 9.999002 seconds) 2020-03-31 09:30:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:11 UTC)" (scheduled at 2020-03-31 09:30:01.440707+00:00) 2020-03-31 09:30:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:30:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:30:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:30:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:30:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:11 UTC)" executed successfully 2020-03-31 09:30:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:30:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:24.410927+00:00 (in 21.080080 seconds) 2020-03-31 09:30:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:33 UTC)" (scheduled at 2020-03-31 09:30:03.330112+00:00) 2020-03-31 09:30:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:30:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:24.410927+00:00 (in 21.080080 seconds) 2020-03-31 09:30:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:33 UTC)" (scheduled at 2020-03-31 09:30:03.330112+00:00) 2020-03-31 09:30:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 3, 331817), True, datetime.datetime(2020, 3, 31, 9, 30, 3, 331817), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:30:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 3, 331817), True, datetime.datetime(2020, 3, 31, 9, 30, 3, 331817), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:30:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:30:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:30:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:33 UTC)" executed successfully 2020-03-31 09:30:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:30:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:14.323352+00:00 (in 9.998929 seconds) 2020-03-31 09:30:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:14 UTC)" (scheduled at 2020-03-31 09:30:04.323352+00:00) 2020-03-31 09:30:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:30:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:14.323352+00:00 (in 9.998929 seconds) 2020-03-31 09:30:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:14 UTC)" (scheduled at 2020-03-31 09:30:04.323352+00:00) 2020-03-31 09:30:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 4, 325149), True, datetime.datetime(2020, 3, 31, 9, 30, 4, 325149), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:30:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 4, 325149), True, datetime.datetime(2020, 3, 31, 9, 30, 4, 325149), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:30:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:30:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:30:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:14 UTC)" executed successfully 2020-03-31 09:30:04,447 [518] [DEBUG] [app] Starting request: urn:request:e4d08035-4339-4fe0-b4e7-6a63b5da7afe (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:04,447 [518] [DEBUG] [app] Starting request: urn:request:e4d08035-4339-4fe0-b4e7-6a63b5da7afe (/health/instance) 2020-03-31 09:30:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:04,454 [556] [DEBUG] [app] Starting request: urn:request:662fb88e-c9fb-44f2-9a73-aae134ca3a32 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:04,454 [556] [DEBUG] [app] Starting request: urn:request:662fb88e-c9fb-44f2-9a73-aae134ca3a32 (/v1/_internal_ping) 2020-03-31 09:30:04,455 [556] [DEBUG] [app] Ending request: urn:request:662fb88e-c9fb-44f2-9a73-aae134ca3a32 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:04,455 [556] [DEBUG] [app] Ending request: urn:request:662fb88e-c9fb-44f2-9a73-aae134ca3a32 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:04,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:30:04,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:30:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:30:04,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:30:04,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:04,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:04,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:04,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:04,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:04,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:04,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:04,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:04,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:04,490 [527] [DEBUG] [app] Starting request: urn:request:267392ad-92c0-40b9-89c2-1587a9887e4e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:04,490 [527] [DEBUG] [app] Starting request: urn:request:267392ad-92c0-40b9-89c2-1587a9887e4e (/_internal_ping) 2020-03-31 09:30:04,492 [527] [DEBUG] [app] Ending request: urn:request:267392ad-92c0-40b9-89c2-1587a9887e4e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:04,492 [527] [DEBUG] [app] Ending request: urn:request:267392ad-92c0-40b9-89c2-1587a9887e4e (/_internal_ping) 2020-03-31 09:30:04,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:30:04,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:30:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:30:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:30:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:04,505 [543] [DEBUG] [app] Starting request: urn:request:648b5258-9ad6-425d-ba26-828a88c9d6b5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:04,505 [543] [DEBUG] [app] Starting request: urn:request:648b5258-9ad6-425d-ba26-828a88c9d6b5 (/c1/_internal_ping) 2020-03-31 09:30:04,506 [543] [DEBUG] [app] Ending request: urn:request:648b5258-9ad6-425d-ba26-828a88c9d6b5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:04,506 [543] [DEBUG] [app] Ending request: urn:request:648b5258-9ad6-425d-ba26-828a88c9d6b5 (/c1/_internal_ping) 2020-03-31 09:30:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:30:04,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-verbs stdout | 2020-03-31 09:30:04,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:04,509 [518] [DEBUG] [app] Ending request: urn:request:e4d08035-4339-4fe0-b4e7-6a63b5da7afe (/health/instance) 2020-03-31 09:30:04,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:04,509 [518] [DEBUG] [app] Ending request: urn:request:e4d08035-4339-4fe0-b4e7-6a63b5da7afe (/health/instance) 2020-03-31 09:30:04,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:04,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:30:04,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:30:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:30:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:16.924497+00:00 (in 9.999277 seconds) 2020-03-31 09:30:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:16 UTC)" (scheduled at 2020-03-31 09:30:06.924497+00:00) 2020-03-31 09:30:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:30:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:16.924497+00:00 (in 9.999277 seconds) 2020-03-31 09:30:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:16 UTC)" (scheduled at 2020-03-31 09:30:06.924497+00:00) 2020-03-31 09:30:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 6, 926130), True, datetime.datetime(2020, 3, 31, 9, 30, 6, 926130), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:30:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 6, 926130), True, datetime.datetime(2020, 3, 31, 9, 30, 6, 926130), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:30:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:06,943 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:30:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:30:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:16 UTC)" executed successfully 2020-03-31 09:30:08,272 [527] [DEBUG] [app] Starting request: urn:request:1df66468-241f-4911-b942-71cdc397c737 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:08,272 [527] [DEBUG] [app] Starting request: urn:request:1df66468-241f-4911-b942-71cdc397c737 (/health/instance) 2020-03-31 09:30:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:08,281 [552] [DEBUG] [app] Starting request: urn:request:4f0d6ec9-1392-4bfb-9ac7-e55f1ddb680e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:08,281 [552] [DEBUG] [app] Starting request: urn:request:4f0d6ec9-1392-4bfb-9ac7-e55f1ddb680e (/v1/_internal_ping) 2020-03-31 09:30:08,282 [552] [DEBUG] [app] Ending request: urn:request:4f0d6ec9-1392-4bfb-9ac7-e55f1ddb680e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:08,282 [552] [DEBUG] [app] Ending request: urn:request:4f0d6ec9-1392-4bfb-9ac7-e55f1ddb680e (/v1/_internal_ping) 2020-03-31 09:30:08,283 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:30:08,283 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:30:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:30:08,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:30:08,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:08,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:08,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:08,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:08,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:08,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:08,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:08,315 [518] [DEBUG] [app] Starting request: urn:request:66336760-116b-497b-8e2f-b50f5990f80e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:08,315 [518] [DEBUG] [app] Starting request: urn:request:66336760-116b-497b-8e2f-b50f5990f80e (/_internal_ping) 2020-03-31 09:30:08,316 [518] [DEBUG] [app] Ending request: urn:request:66336760-116b-497b-8e2f-b50f5990f80e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:08,316 [518] [DEBUG] [app] Ending request: urn:request:66336760-116b-497b-8e2f-b50f5990f80e (/_internal_ping) 2020-03-31 09:30:08,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:08,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:08,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:08,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:30:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:08,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:08,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:08,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:08,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:08,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:30:08,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:30:08,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:08,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:08,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:08,330 [537] [DEBUG] [app] Starting request: urn:request:9ef61ba4-0046-421b-9f90-29b8dbbbfc39 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:08,330 [537] [DEBUG] [app] Starting request: urn:request:9ef61ba4-0046-421b-9f90-29b8dbbbfc39 (/c1/_internal_ping) 2020-03-31 09:30:08,331 [537] [DEBUG] [app] Ending request: urn:request:9ef61ba4-0046-421b-9f90-29b8dbbbfc39 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:08,331 [537] [DEBUG] [app] Ending request: urn:request:9ef61ba4-0046-421b-9f90-29b8dbbbfc39 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:08,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:08,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:30:08,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:08,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:08,334 [527] [DEBUG] [app] Ending request: urn:request:1df66468-241f-4911-b942-71cdc397c737 (/health/instance) 2020-03-31 09:30:08,334 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:08,334 [527] [DEBUG] [app] Ending request: urn:request:1df66468-241f-4911-b942-71cdc397c737 (/health/instance) 2020-03-31 09:30:08,334 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:08,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:30:08,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:30:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:30:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:21.440707+00:00 (in 9.999031 seconds) 2020-03-31 09:30:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:21 UTC)" (scheduled at 2020-03-31 09:30:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:30:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:21.440707+00:00 (in 9.999031 seconds) 2020-03-31 09:30:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:21 UTC)" (scheduled at 2020-03-31 09:30:11.440707+00:00) 2020-03-31 09:30:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:30:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:30:11,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:30:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:30:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:21 UTC)" executed successfully 2020-03-31 09:30:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:30:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:24.323352+00:00 (in 9.999058 seconds) 2020-03-31 09:30:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:24 UTC)" (scheduled at 2020-03-31 09:30:14.323352+00:00) 2020-03-31 09:30:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:30:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:24.323352+00:00 (in 9.999058 seconds) 2020-03-31 09:30:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:24 UTC)" (scheduled at 2020-03-31 09:30:14.323352+00:00) 2020-03-31 09:30:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 14, 325284), True, datetime.datetime(2020, 3, 31, 9, 30, 14, 325284), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:30:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 14, 325284), True, datetime.datetime(2020, 3, 31, 9, 30, 14, 325284), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:30:14,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:30:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:30:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:24 UTC)" executed successfully 2020-03-31 09:30:14,447 [518] [DEBUG] [app] Starting request: urn:request:821299a3-96fa-46f1-882d-70d1cfd79940 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:14,447 [518] [DEBUG] [app] Starting request: urn:request:821299a3-96fa-46f1-882d-70d1cfd79940 (/health/instance) 2020-03-31 09:30:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:14,455 [553] [DEBUG] [app] Starting request: urn:request:c16bf362-fe78-4a53-a234-62706e7c9432 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:14,455 [553] [DEBUG] [app] Starting request: urn:request:c16bf362-fe78-4a53-a234-62706e7c9432 (/v1/_internal_ping) 2020-03-31 09:30:14,456 [553] [DEBUG] [app] Ending request: urn:request:c16bf362-fe78-4a53-a234-62706e7c9432 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:14,456 [553] [DEBUG] [app] Ending request: urn:request:c16bf362-fe78-4a53-a234-62706e7c9432 (/v1/_internal_ping) 2020-03-31 09:30:14,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:30:14,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:14,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:30:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:30:14,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:14,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:14,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:14,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:14,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:14,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:14,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:14,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:14,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:14,488 [518] [DEBUG] [app] Starting request: urn:request:91cbdef5-f260-43d3-b97e-42bd927fa204 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:14,488 [518] [DEBUG] [app] Starting request: urn:request:91cbdef5-f260-43d3-b97e-42bd927fa204 (/_internal_ping) 2020-03-31 09:30:14,489 [518] [DEBUG] [app] Ending request: urn:request:91cbdef5-f260-43d3-b97e-42bd927fa204 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:14,489 [518] [DEBUG] [app] Ending request: urn:request:91cbdef5-f260-43d3-b97e-42bd927fa204 (/_internal_ping) 2020-03-31 09:30:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:30:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:30:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:14,502 [543] [DEBUG] [app] Starting request: urn:request:1446e875-1117-4114-b935-d2ee8b400187 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:14,502 [543] [DEBUG] [app] Starting request: urn:request:1446e875-1117-4114-b935-d2ee8b400187 (/c1/_internal_ping) 2020-03-31 09:30:14,503 [543] [DEBUG] [app] Ending request: urn:request:1446e875-1117-4114-b935-d2ee8b400187 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:14,503 [543] [DEBUG] [app] Ending request: urn:request:1446e875-1117-4114-b935-d2ee8b400187 (/c1/_internal_ping) 2020-03-31 09:30:14,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:14,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:14,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:14,505 [518] [DEBUG] [app] Ending request: urn:request:821299a3-96fa-46f1-882d-70d1cfd79940 (/health/instance) 2020-03-31 09:30:14,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:30:14,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:30:14,505 [518] [DEBUG] [app] Ending request: urn:request:821299a3-96fa-46f1-882d-70d1cfd79940 (/health/instance) 2020-03-31 09:30:14,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:14,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:30:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:30:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:33.366791+00:00 (in 18.826294 seconds) exportactionlogsworker stdout | 2020-03-31 09:30:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:33.366791+00:00 (in 18.826294 seconds) 2020-03-31 09:30:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:35:14 UTC)" (scheduled at 2020-03-31 09:30:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:30:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:35:14 UTC)" (scheduled at 2020-03-31 09:30:14.539408+00:00) 2020-03-31 09:30:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:30:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), u'exportactionlogs/%']) 2020-03-31 09:30:14,558 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), True, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:30:14,558 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), True, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), 0, u'exportactionlogs/%']) 2020-03-31 09:30:14,563 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), True, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:30:14,563 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), True, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 30, 14, 541374), u'exportactionlogs/%']) 2020-03-31 09:30:14,567 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:30:14,567 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:14,569 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:35:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:30:14,569 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:35:14 UTC)" executed successfully 2020-03-31 09:30:14,570 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:30:14,570 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:14,571 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:30:14,571 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:14,572 [166] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:30:14,572 [166] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:14,574 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:30:14,574 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:14,575 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:30:14,575 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:14,576 [166] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:30:14,576 [166] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:30:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:30:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:45.064514+00:00 (in 29.999127 seconds) 2020-03-31 09:30:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:45 UTC)" (scheduled at 2020-03-31 09:30:15.064514+00:00) gcworker stdout | 2020-03-31 09:30:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:45.064514+00:00 (in 29.999127 seconds) 2020-03-31 09:30:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:45 UTC)" (scheduled at 2020-03-31 09:30:15.064514+00:00) 2020-03-31 09:30:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437415, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:30:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437415, 1209600, 500, 1, 0]) 2020-03-31 09:30:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:30:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:30:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:30:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:45 UTC)" executed successfully 2020-03-31 09:30:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:30:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:26.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:30:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:26 UTC)" (scheduled at 2020-03-31 09:30:16.924497+00:00) 2020-03-31 09:30:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:30:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:26.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:30:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:26 UTC)" (scheduled at 2020-03-31 09:30:16.924497+00:00) 2020-03-31 09:30:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 16, 926025), True, datetime.datetime(2020, 3, 31, 9, 30, 16, 926025), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:30:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 16, 926025), True, datetime.datetime(2020, 3, 31, 9, 30, 16, 926025), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:30:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:30:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:30:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:26 UTC)" executed successfully 2020-03-31 09:30:18,273 [518] [DEBUG] [app] Starting request: urn:request:1f7ca31b-f42b-490d-a762-7ae0aef49873 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:18,273 [518] [DEBUG] [app] Starting request: urn:request:1f7ca31b-f42b-490d-a762-7ae0aef49873 (/health/instance) 2020-03-31 09:30:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:18,278 [556] [DEBUG] [app] Starting request: urn:request:5262e3f6-78d3-4176-a2fe-f61388529661 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:18,278 [556] [DEBUG] [app] Starting request: urn:request:5262e3f6-78d3-4176-a2fe-f61388529661 (/v1/_internal_ping) 2020-03-31 09:30:18,279 [556] [DEBUG] [app] Ending request: urn:request:5262e3f6-78d3-4176-a2fe-f61388529661 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:18,279 [556] [DEBUG] [app] Ending request: urn:request:5262e3f6-78d3-4176-a2fe-f61388529661 (/v1/_internal_ping) 2020-03-31 09:30:18,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:30:18,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:18,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:30:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:30:18,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:18,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:18,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:18,283 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:18,284 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:18,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:18,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:18,319 [526] [DEBUG] [app] Starting request: urn:request:1004b68e-66bc-44d6-8a82-459260b4f85d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:18,319 [526] [DEBUG] [app] Starting request: urn:request:1004b68e-66bc-44d6-8a82-459260b4f85d (/_internal_ping) 2020-03-31 09:30:18,320 [526] [DEBUG] [app] Ending request: urn:request:1004b68e-66bc-44d6-8a82-459260b4f85d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:18,320 [526] [DEBUG] [app] Ending request: urn:request:1004b68e-66bc-44d6-8a82-459260b4f85d (/_internal_ping) 2020-03-31 09:30:18,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:18,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:30:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:18,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:30:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:30:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:18,333 [537] [DEBUG] [app] Starting request: urn:request:b3a50e32-56ee-43a3-a476-6827ef8d7f32 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:18,333 [537] [DEBUG] [app] Starting request: urn:request:b3a50e32-56ee-43a3-a476-6827ef8d7f32 (/c1/_internal_ping) 2020-03-31 09:30:18,334 [537] [DEBUG] [app] Ending request: urn:request:b3a50e32-56ee-43a3-a476-6827ef8d7f32 (/c1/_internal_ping) 2020-03-31 09:30:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:18,334 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:18,335 [518] [DEBUG] [app] Ending request: urn:request:1f7ca31b-f42b-490d-a762-7ae0aef49873 (/health/instance) 2020-03-31 09:30:18,336 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-verbs stdout | 2020-03-31 09:30:18,334 [537] [DEBUG] [app] Ending request: urn:request:b3a50e32-56ee-43a3-a476-6827ef8d7f32 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:30:18,334 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:18,335 [518] [DEBUG] [app] Ending request: urn:request:1f7ca31b-f42b-490d-a762-7ae0aef49873 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:18,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:30:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:30:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:31.440707+00:00 (in 9.998940 seconds) 2020-03-31 09:30:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:31 UTC)" (scheduled at 2020-03-31 09:30:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:30:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:31.440707+00:00 (in 9.998940 seconds) 2020-03-31 09:30:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:31 UTC)" (scheduled at 2020-03-31 09:30:21.440707+00:00) 2020-03-31 09:30:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:30:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:30:21,512 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:30:21,512 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:21,512 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:21,512 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:30:21,512 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:21,512 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:31 UTC)" executed successfully 2020-03-31 09:30:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:30:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:52.113221+00:00 (in 29.998546 seconds) 2020-03-31 09:30:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:52 UTC)" (scheduled at 2020-03-31 09:30:22.113221+00:00) securityworker stdout | 2020-03-31 09:30:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:52.113221+00:00 (in 29.998546 seconds) 2020-03-31 09:30:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:52 UTC)" (scheduled at 2020-03-31 09:30:22.113221+00:00) 2020-03-31 09:30:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:30:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:30:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:30:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:30:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:30:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:30:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:30:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:30:52 UTC)" executed successfully 2020-03-31 09:30:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:30:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:34.323352+00:00 (in 9.999130 seconds) 2020-03-31 09:30:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:34 UTC)" (scheduled at 2020-03-31 09:30:24.323352+00:00) 2020-03-31 09:30:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:30:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:34.323352+00:00 (in 9.999130 seconds) 2020-03-31 09:30:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:34 UTC)" (scheduled at 2020-03-31 09:30:24.323352+00:00) 2020-03-31 09:30:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 24, 325009), True, datetime.datetime(2020, 3, 31, 9, 30, 24, 325009), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:30:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 24, 325009), True, datetime.datetime(2020, 3, 31, 9, 30, 24, 325009), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:30:24,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:30:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:30:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:34 UTC)" executed successfully 2020-03-31 09:30:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:30:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:33.330112+00:00 (in 8.918229 seconds) 2020-03-31 09:30:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:24 UTC)" (scheduled at 2020-03-31 09:30:24.410927+00:00) 2020-03-31 09:30:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:30:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:33.330112+00:00 (in 8.918229 seconds) 2020-03-31 09:30:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:24 UTC)" (scheduled at 2020-03-31 09:30:24.410927+00:00) 2020-03-31 09:30:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:24 UTC)" executed successfully 2020-03-31 09:30:24,446 [518] [DEBUG] [app] Starting request: urn:request:c187ab46-acb2-44de-aefd-9b7618535a5e (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:24,446 [518] [DEBUG] [app] Starting request: urn:request:c187ab46-acb2-44de-aefd-9b7618535a5e (/health/instance) 2020-03-31 09:30:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:24,454 [547] [DEBUG] [app] Starting request: urn:request:2e819e59-de07-48f0-b990-4f2d706e106a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:24,454 [547] [DEBUG] [app] Starting request: urn:request:2e819e59-de07-48f0-b990-4f2d706e106a (/v1/_internal_ping) 2020-03-31 09:30:24,455 [547] [DEBUG] [app] Ending request: urn:request:2e819e59-de07-48f0-b990-4f2d706e106a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:24,455 [547] [DEBUG] [app] Ending request: urn:request:2e819e59-de07-48f0-b990-4f2d706e106a (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:24,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:30:24,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:24,457 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:30:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:30:24,457 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:24,459 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:24,459 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:24,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:24,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:24,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:24,490 [527] [DEBUG] [app] Starting request: urn:request:c891b22d-c2c7-4e81-b201-3b6a7a0b4ad2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:24,490 [527] [DEBUG] [app] Starting request: urn:request:c891b22d-c2c7-4e81-b201-3b6a7a0b4ad2 (/_internal_ping) 2020-03-31 09:30:24,491 [527] [DEBUG] [app] Ending request: urn:request:c891b22d-c2c7-4e81-b201-3b6a7a0b4ad2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:24,491 [527] [DEBUG] [app] Ending request: urn:request:c891b22d-c2c7-4e81-b201-3b6a7a0b4ad2 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:30:24,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:30:24,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:30:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:30:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:24,512 [543] [DEBUG] [app] Starting request: urn:request:3f016101-4d6a-4751-9e66-b848278f59a2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:24,512 [543] [DEBUG] [app] Starting request: urn:request:3f016101-4d6a-4751-9e66-b848278f59a2 (/c1/_internal_ping) 2020-03-31 09:30:24,513 [543] [DEBUG] [app] Ending request: urn:request:3f016101-4d6a-4751-9e66-b848278f59a2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:24,513 [543] [DEBUG] [app] Ending request: urn:request:3f016101-4d6a-4751-9e66-b848278f59a2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:24,514 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:24,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:24,516 [518] [DEBUG] [app] Ending request: urn:request:c187ab46-acb2-44de-aefd-9b7618535a5e (/health/instance) 2020-03-31 09:30:24,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:24,517 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-verbs stdout | 2020-03-31 09:30:24,514 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:24,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:30:24,516 [518] [DEBUG] [app] Ending request: urn:request:c187ab46-acb2-44de-aefd-9b7618535a5e (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:24,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:24,517 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:30:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:30:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:36.924497+00:00 (in 9.999233 seconds) 2020-03-31 09:30:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:36 UTC)" (scheduled at 2020-03-31 09:30:26.924497+00:00) 2020-03-31 09:30:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:30:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:36.924497+00:00 (in 9.999233 seconds) 2020-03-31 09:30:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:36 UTC)" (scheduled at 2020-03-31 09:30:26.924497+00:00) 2020-03-31 09:30:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 26, 926130), True, datetime.datetime(2020, 3, 31, 9, 30, 26, 926130), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:30:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 26, 926130), True, datetime.datetime(2020, 3, 31, 9, 30, 26, 926130), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:30:26,952 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:26,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:26,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:30:26,952 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:26,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:26,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:36 UTC)" executed successfully 2020-03-31 09:30:28,273 [527] [DEBUG] [app] Starting request: urn:request:db9e5445-fe33-4de2-84e3-7ff860cb7763 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:28,273 [527] [DEBUG] [app] Starting request: urn:request:db9e5445-fe33-4de2-84e3-7ff860cb7763 (/health/instance) 2020-03-31 09:30:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:28,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:28,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:28,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,282 [538] [DEBUG] [app] Starting request: urn:request:b7bc5ac6-2993-4831-9eee-812a82ee8889 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:28,282 [538] [DEBUG] [app] Starting request: urn:request:b7bc5ac6-2993-4831-9eee-812a82ee8889 (/v1/_internal_ping) 2020-03-31 09:30:28,284 [538] [DEBUG] [app] Ending request: urn:request:b7bc5ac6-2993-4831-9eee-812a82ee8889 (/v1/_internal_ping) 2020-03-31 09:30:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:30:28,284 [538] [DEBUG] [app] Ending request: urn:request:b7bc5ac6-2993-4831-9eee-812a82ee8889 (/v1/_internal_ping) 2020-03-31 09:30:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:30:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:30:28,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:30:28,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:30:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:28,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:30:28,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:28,290 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:28,290 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:28,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:28,297 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:28,312 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:28,312 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:28,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,318 [518] [DEBUG] [app] Starting request: urn:request:91137bd9-eb1c-4ec9-b4f1-c49a01ac5c75 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:28,318 [518] [DEBUG] [app] Starting request: urn:request:91137bd9-eb1c-4ec9-b4f1-c49a01ac5c75 (/_internal_ping) 2020-03-31 09:30:28,320 [518] [DEBUG] [app] Ending request: urn:request:91137bd9-eb1c-4ec9-b4f1-c49a01ac5c75 (/_internal_ping) 2020-03-31 09:30:28,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:28,320 [518] [DEBUG] [app] Ending request: urn:request:91137bd9-eb1c-4ec9-b4f1-c49a01ac5c75 (/_internal_ping) 2020-03-31 09:30:28,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:28,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:30:28,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:30:28,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:30:28,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:28,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:28,341 [527] [DEBUG] [app] Ending request: urn:request:db9e5445-fe33-4de2-84e3-7ff860cb7763 (/health/instance) 2020-03-31 09:30:28,341 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:28,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:30:28,337 [534] [DEBUG] [app] Starting request: urn:request:218ff72f-999e-453e-aacb-851a65005cd0 (/c1/_internal_ping) 2020-03-31 09:30:28,338 [534] [DEBUG] [app] Ending request: urn:request:218ff72f-999e-453e-aacb-851a65005cd0 (/c1/_internal_ping) 2020-03-31 09:30:28,342 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:28,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:30:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:28,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:28,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:28,341 [527] [DEBUG] [app] Ending request: urn:request:db9e5445-fe33-4de2-84e3-7ff860cb7763 (/health/instance) 2020-03-31 09:30:28,341 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:28,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 127.0.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:30:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-verbs stdout | 2020-03-31 09:30:28,337 [534] [DEBUG] [app] Starting request: urn:request:218ff72f-999e-453e-aacb-851a65005cd0 (/c1/_internal_ping) 2020-03-31 09:30:28,338 [534] [DEBUG] [app] Ending request: urn:request:218ff72f-999e-453e-aacb-851a65005cd0 (/c1/_internal_ping) 2020-03-31 09:30:28,342 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:30:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:00.337699+00:00 (in 29.996810 seconds) 2020-03-31 09:30:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:00 UTC)" (scheduled at 2020-03-31 09:30:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:30:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:00.337699+00:00 (in 29.996810 seconds) 2020-03-31 09:30:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:00 UTC)" (scheduled at 2020-03-31 09:30:30.337699+00:00) 2020-03-31 09:30:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 30, 30, 341692), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:30:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 30, 30, 341692), False, 50, 1, 0]) 2020-03-31 09:30:30,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:30:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:30:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:30:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:00 UTC)" executed successfully 2020-03-31 09:30:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:41.440707+00:00 (in 9.998675 seconds) 2020-03-31 09:30:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:41 UTC)" (scheduled at 2020-03-31 09:30:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:30:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:41.440707+00:00 (in 9.998675 seconds) 2020-03-31 09:30:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:41 UTC)" (scheduled at 2020-03-31 09:30:31.440707+00:00) 2020-03-31 09:30:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:30:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:30:31,483 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:31,483 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:31,483 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:30:31,483 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:31,483 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:31,483 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:41 UTC)" executed successfully 2020-03-31 09:30:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:30:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:03.330112+00:00 (in 29.999090 seconds) 2020-03-31 09:30:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:03 UTC)" (scheduled at 2020-03-31 09:30:33.330112+00:00) 2020-03-31 09:30:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:30:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:03.330112+00:00 (in 29.999090 seconds) 2020-03-31 09:30:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:03 UTC)" (scheduled at 2020-03-31 09:30:33.330112+00:00) 2020-03-31 09:30:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 33, 331757), True, datetime.datetime(2020, 3, 31, 9, 30, 33, 331757), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:30:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 33, 331757), True, datetime.datetime(2020, 3, 31, 9, 30, 33, 331757), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:30:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:30:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:30:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:03 UTC)" executed successfully 2020-03-31 09:30:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:30:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:43.539999+00:00 (in 10.172094 seconds) 2020-03-31 09:30:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:33 UTC)" (scheduled at 2020-03-31 09:30:33.366791+00:00) 2020-03-31 09:30:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:30:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:43.539999+00:00 (in 10.172094 seconds) 2020-03-31 09:30:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:33 UTC)" (scheduled at 2020-03-31 09:30:33.366791+00:00) 2020-03-31 09:30:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 33, 368938), True, datetime.datetime(2020, 3, 31, 9, 30, 33, 368938), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:30:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 33, 368938), True, datetime.datetime(2020, 3, 31, 9, 30, 33, 368938), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:30:33,387 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:30:33,387 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:30:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:33 UTC)" executed successfully 2020-03-31 09:30:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:30:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:44.323352+00:00 (in 9.999069 seconds) 2020-03-31 09:30:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:44 UTC)" (scheduled at 2020-03-31 09:30:34.323352+00:00) 2020-03-31 09:30:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:30:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:44.323352+00:00 (in 9.999069 seconds) 2020-03-31 09:30:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:44 UTC)" (scheduled at 2020-03-31 09:30:34.323352+00:00) 2020-03-31 09:30:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 34, 325147), True, datetime.datetime(2020, 3, 31, 9, 30, 34, 325147), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:30:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 34, 325147), True, datetime.datetime(2020, 3, 31, 9, 30, 34, 325147), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:30:34,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:30:34,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:34,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:30:34,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:44 UTC)" executed successfully 2020-03-31 09:30:34,446 [526] [DEBUG] [app] Starting request: urn:request:7977b5d4-7c2d-4e84-9521-de3ed434824a (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:34,446 [526] [DEBUG] [app] Starting request: urn:request:7977b5d4-7c2d-4e84-9521-de3ed434824a (/health/instance) 2020-03-31 09:30:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:34,454 [538] [DEBUG] [app] Starting request: urn:request:fe1d25ea-c785-47f9-a41c-e7dd8e823d4e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:34,454 [538] [DEBUG] [app] Starting request: urn:request:fe1d25ea-c785-47f9-a41c-e7dd8e823d4e (/v1/_internal_ping) 2020-03-31 09:30:34,455 [538] [DEBUG] [app] Ending request: urn:request:fe1d25ea-c785-47f9-a41c-e7dd8e823d4e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:34,455 [538] [DEBUG] [app] Ending request: urn:request:fe1d25ea-c785-47f9-a41c-e7dd8e823d4e (/v1/_internal_ping) 2020-03-31 09:30:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:30:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:30:34,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:30:34,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:30:34,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:34,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:34,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:34,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:34,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:34,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:34,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:34,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:34,485 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:34,487 [523] [DEBUG] [app] Starting request: urn:request:2d8cfdfb-9237-4e86-aa6d-52111a07f8e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:34,487 [523] [DEBUG] [app] Starting request: urn:request:2d8cfdfb-9237-4e86-aa6d-52111a07f8e3 (/_internal_ping) 2020-03-31 09:30:34,488 [523] [DEBUG] [app] Ending request: urn:request:2d8cfdfb-9237-4e86-aa6d-52111a07f8e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:34,488 [523] [DEBUG] [app] Ending request: urn:request:2d8cfdfb-9237-4e86-aa6d-52111a07f8e3 (/_internal_ping) 2020-03-31 09:30:34,489 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:34,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:30:34,489 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:34,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:30:34,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:34,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:34,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:34,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:34,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:30:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:30:34,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:34,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:34,502 [536] [DEBUG] [app] Starting request: urn:request:5b922d66-ae46-4574-85f4-289825bcb300 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:34,502 [536] [DEBUG] [app] Starting request: urn:request:5b922d66-ae46-4574-85f4-289825bcb300 (/c1/_internal_ping) 2020-03-31 09:30:34,502 [536] [DEBUG] [app] Ending request: urn:request:5b922d66-ae46-4574-85f4-289825bcb300 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:34,502 [536] [DEBUG] [app] Ending request: urn:request:5b922d66-ae46-4574-85f4-289825bcb300 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:34,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:34,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:30:34,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:34,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:34,505 [526] [DEBUG] [app] Ending request: urn:request:7977b5d4-7c2d-4e84-9521-de3ed434824a (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:34,505 [526] [DEBUG] [app] Ending request: urn:request:7977b5d4-7c2d-4e84-9521-de3ed434824a (/health/instance) 2020-03-31 09:30:34,505 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:34,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:30:34,505 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:34,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:30:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:38.010934+00:00 (in 1.085517 seconds) 2020-03-31 09:30:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:46 UTC)" (scheduled at 2020-03-31 09:30:36.924497+00:00) 2020-03-31 09:30:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:30:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 36, 926142), True, datetime.datetime(2020, 3, 31, 9, 30, 36, 926142), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:30:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:38.010934+00:00 (in 1.085517 seconds) 2020-03-31 09:30:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:46 UTC)" (scheduled at 2020-03-31 09:30:36.924497+00:00) 2020-03-31 09:30:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:30:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 36, 926142), True, datetime.datetime(2020, 3, 31, 9, 30, 36, 926142), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:30:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:30:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:30:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:46 UTC)" executed successfully 2020-03-31 09:30:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:30:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:46.924497+00:00 (in 8.912715 seconds) 2020-03-31 09:30:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:38 UTC)" (scheduled at 2020-03-31 09:30:38.010934+00:00) 2020-03-31 09:30:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:30:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:46.924497+00:00 (in 8.912715 seconds) 2020-03-31 09:30:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:38 UTC)" (scheduled at 2020-03-31 09:30:38.010934+00:00) 2020-03-31 09:30:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:38 UTC)" executed successfully 2020-03-31 09:30:38,273 [518] [DEBUG] [app] Starting request: urn:request:285aac22-29e5-46c3-8d22-ba6cb5dd7cc1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:38,273 [518] [DEBUG] [app] Starting request: urn:request:285aac22-29e5-46c3-8d22-ba6cb5dd7cc1 (/health/instance) 2020-03-31 09:30:38,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:38,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:38,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:38,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:38,282 [538] [DEBUG] [app] Starting request: urn:request:6870d1e5-1bd9-4276-a141-085d3f4d782e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:38,282 [538] [DEBUG] [app] Starting request: urn:request:6870d1e5-1bd9-4276-a141-085d3f4d782e (/v1/_internal_ping) 2020-03-31 09:30:38,284 [538] [DEBUG] [app] Ending request: urn:request:6870d1e5-1bd9-4276-a141-085d3f4d782e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:38,284 [538] [DEBUG] [app] Ending request: urn:request:6870d1e5-1bd9-4276-a141-085d3f4d782e (/v1/_internal_ping) 2020-03-31 09:30:38,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:38,285 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:30:38,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:38,286 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:38,285 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:38,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:30:38,286 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:30:38,287 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:38,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:38,289 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:38,290 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:38,290 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:38,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:38,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:38,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:38,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:38,319 [518] [DEBUG] [app] Starting request: urn:request:b0156dbd-a198-4520-bdd3-42ea36e93a54 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:38,319 [518] [DEBUG] [app] Starting request: urn:request:b0156dbd-a198-4520-bdd3-42ea36e93a54 (/_internal_ping) 2020-03-31 09:30:38,320 [518] [DEBUG] [app] Ending request: urn:request:b0156dbd-a198-4520-bdd3-42ea36e93a54 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:38,320 [518] [DEBUG] [app] Ending request: urn:request:b0156dbd-a198-4520-bdd3-42ea36e93a54 (/_internal_ping) 2020-03-31 09:30:38,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:38,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:30:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:30:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:38,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:30:38,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:38,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:30:38,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:38,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:38,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:38,337 [536] [DEBUG] [app] Starting request: urn:request:98bf6721-2a01-461d-80d8-ca1b2d84f4d4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:38,337 [536] [DEBUG] [app] Starting request: urn:request:98bf6721-2a01-461d-80d8-ca1b2d84f4d4 (/c1/_internal_ping) 2020-03-31 09:30:38,338 [536] [DEBUG] [app] Ending request: urn:request:98bf6721-2a01-461d-80d8-ca1b2d84f4d4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:38,338 [536] [DEBUG] [app] Ending request: urn:request:98bf6721-2a01-461d-80d8-ca1b2d84f4d4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:38,339 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:38,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:30:38,339 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:38,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:38,342 [518] [DEBUG] [app] Ending request: urn:request:285aac22-29e5-46c3-8d22-ba6cb5dd7cc1 (/health/instance) 2020-03-31 09:30:38,342 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:38,342 [518] [DEBUG] [app] Ending request: urn:request:285aac22-29e5-46c3-8d22-ba6cb5dd7cc1 (/health/instance) 2020-03-31 09:30:38,342 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:38,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:30:38,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:30:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:51.440707+00:00 (in 9.999165 seconds) 2020-03-31 09:30:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:51 UTC)" (scheduled at 2020-03-31 09:30:41.440707+00:00) 2020-03-31 09:30:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:30:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:51.440707+00:00 (in 9.999165 seconds) 2020-03-31 09:30:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:51 UTC)" (scheduled at 2020-03-31 09:30:41.440707+00:00) 2020-03-31 09:30:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:30:41,529 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:41,529 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:41,530 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:30:41,529 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:41,529 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:41,530 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:51 UTC)" executed successfully 2020-03-31 09:30:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:30:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:33.366791+00:00 (in 49.825940 seconds) 2020-03-31 09:30:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:43 UTC)" (scheduled at 2020-03-31 09:30:43.539999+00:00) 2020-03-31 09:30:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:30:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:33.366791+00:00 (in 49.825940 seconds) 2020-03-31 09:30:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:43 UTC)" (scheduled at 2020-03-31 09:30:43.539999+00:00) 2020-03-31 09:30:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:43 UTC)" executed successfully 2020-03-31 09:30:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:30:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:44 UTC)" (scheduled at 2020-03-31 09:30:44.323352+00:00) 2020-03-31 09:30:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:30:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:44 UTC)" (scheduled at 2020-03-31 09:30:44.323352+00:00) 2020-03-31 09:30:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:49.498966+00:00 (in 5.174786 seconds) notificationworker stdout | 2020-03-31 09:30:44,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:49.498966+00:00 (in 5.174786 seconds) 2020-03-31 09:30:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 44, 325096), True, datetime.datetime(2020, 3, 31, 9, 30, 44, 325096), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:30:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 44, 325096), True, datetime.datetime(2020, 3, 31, 9, 30, 44, 325096), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:30:44,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:44,344 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:30:44,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:30:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:54 UTC)" executed successfully 2020-03-31 09:30:44,446 [518] [DEBUG] [app] Starting request: urn:request:7cc171d7-918b-4f06-843f-6f0fdfcfc602 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:44,446 [518] [DEBUG] [app] Starting request: urn:request:7cc171d7-918b-4f06-843f-6f0fdfcfc602 (/health/instance) 2020-03-31 09:30:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:44,453 [548] [DEBUG] [app] Starting request: urn:request:6fed0f30-7158-4d01-8778-a8bd0ba62b06 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:44,453 [548] [DEBUG] [app] Starting request: urn:request:6fed0f30-7158-4d01-8778-a8bd0ba62b06 (/v1/_internal_ping) 2020-03-31 09:30:44,454 [548] [DEBUG] [app] Ending request: urn:request:6fed0f30-7158-4d01-8778-a8bd0ba62b06 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:44,454 [548] [DEBUG] [app] Ending request: urn:request:6fed0f30-7158-4d01-8778-a8bd0ba62b06 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:44,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:30:44,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:30:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:30:44,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:30:44,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:44,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:44,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:44,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:44,459 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:44,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:44,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:44,485 [518] [DEBUG] [app] Starting request: urn:request:6cb4bde3-ad67-455f-b1db-b6d6490c92e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:44,485 [518] [DEBUG] [app] Starting request: urn:request:6cb4bde3-ad67-455f-b1db-b6d6490c92e3 (/_internal_ping) 2020-03-31 09:30:44,486 [518] [DEBUG] [app] Ending request: urn:request:6cb4bde3-ad67-455f-b1db-b6d6490c92e3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:44,486 [518] [DEBUG] [app] Ending request: urn:request:6cb4bde3-ad67-455f-b1db-b6d6490c92e3 (/_internal_ping) 2020-03-31 09:30:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:30:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:30:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:44,500 [536] [DEBUG] [app] Starting request: urn:request:e501f51c-2cfe-471d-b4bc-c86eab83a3b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:44,500 [536] [DEBUG] [app] Starting request: urn:request:e501f51c-2cfe-471d-b4bc-c86eab83a3b1 (/c1/_internal_ping) 2020-03-31 09:30:44,501 [536] [DEBUG] [app] Ending request: urn:request:e501f51c-2cfe-471d-b4bc-c86eab83a3b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:44,501 [536] [DEBUG] [app] Ending request: urn:request:e501f51c-2cfe-471d-b4bc-c86eab83a3b1 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:44,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:30:44,502 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:44,503 [518] [DEBUG] [app] Ending request: urn:request:7cc171d7-918b-4f06-843f-6f0fdfcfc602 (/health/instance) 2020-03-31 09:30:44,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:44,503 [518] [DEBUG] [app] Ending request: urn:request:7cc171d7-918b-4f06-843f-6f0fdfcfc602 (/health/instance) 2020-03-31 09:30:44,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:30:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:30:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:30:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:15.064514+00:00 (in 29.999095 seconds) 2020-03-31 09:30:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:15 UTC)" (scheduled at 2020-03-31 09:30:45.064514+00:00) gcworker stdout | 2020-03-31 09:30:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:15.064514+00:00 (in 29.999095 seconds) 2020-03-31 09:30:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:15 UTC)" (scheduled at 2020-03-31 09:30:45.064514+00:00) 2020-03-31 09:30:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437445, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:30:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437445, 1209600, 500, 1, 0]) 2020-03-31 09:30:45,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:30:45,086 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:30:45,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:30:45,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:45,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:30:45,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:15 UTC)" executed successfully 2020-03-31 09:30:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:30:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:46 UTC)" (scheduled at 2020-03-31 09:30:46.924497+00:00) 2020-03-31 09:30:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:30:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:46 UTC)" (scheduled at 2020-03-31 09:30:46.924497+00:00) 2020-03-31 09:30:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:56.924497+00:00 (in 9.999028 seconds) storagereplication stdout | 2020-03-31 09:30:46,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:56.924497+00:00 (in 9.999028 seconds) 2020-03-31 09:30:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 46, 926315), True, datetime.datetime(2020, 3, 31, 9, 30, 46, 926315), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:30:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 46, 926315), True, datetime.datetime(2020, 3, 31, 9, 30, 46, 926315), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:30:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:30:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:30:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:30:56 UTC)" executed successfully 2020-03-31 09:30:48,273 [527] [DEBUG] [app] Starting request: urn:request:03e78f0c-d10a-441e-afe4-d23b86078b37 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:48,273 [527] [DEBUG] [app] Starting request: urn:request:03e78f0c-d10a-441e-afe4-d23b86078b37 (/health/instance) 2020-03-31 09:30:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:48,280 [551] [DEBUG] [app] Starting request: urn:request:005acc50-11d1-4cbd-902d-2ba37db3d931 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:48,280 [551] [DEBUG] [app] Starting request: urn:request:005acc50-11d1-4cbd-902d-2ba37db3d931 (/v1/_internal_ping) 2020-03-31 09:30:48,282 [551] [DEBUG] [app] Ending request: urn:request:005acc50-11d1-4cbd-902d-2ba37db3d931 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:48,282 [551] [DEBUG] [app] Ending request: urn:request:005acc50-11d1-4cbd-902d-2ba37db3d931 (/v1/_internal_ping) 2020-03-31 09:30:48,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:30:48,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:48,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:30:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:30:48,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:48,287 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:48,287 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:48,288 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:48,288 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:48,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:48,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:48,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:48,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:48,313 [526] [DEBUG] [app] Starting request: urn:request:be5c4734-be4b-42de-896d-c8fd10eb445c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:48,313 [526] [DEBUG] [app] Starting request: urn:request:be5c4734-be4b-42de-896d-c8fd10eb445c (/_internal_ping) 2020-03-31 09:30:48,314 [526] [DEBUG] [app] Ending request: urn:request:be5c4734-be4b-42de-896d-c8fd10eb445c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:48,314 [526] [DEBUG] [app] Ending request: urn:request:be5c4734-be4b-42de-896d-c8fd10eb445c (/_internal_ping) 2020-03-31 09:30:48,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:30:48,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:30:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:30:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:30:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:30:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:30:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:48,329 [543] [DEBUG] [app] Starting request: urn:request:8879cdfe-7f88-4d78-aeaf-7436be7b0b45 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:48,329 [543] [DEBUG] [app] Starting request: urn:request:8879cdfe-7f88-4d78-aeaf-7436be7b0b45 (/c1/_internal_ping) 2020-03-31 09:30:48,330 [543] [DEBUG] [app] Ending request: urn:request:8879cdfe-7f88-4d78-aeaf-7436be7b0b45 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:48,330 [543] [DEBUG] [app] Ending request: urn:request:8879cdfe-7f88-4d78-aeaf-7436be7b0b45 (/c1/_internal_ping) 2020-03-31 09:30:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:48,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:30:48,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:48,332 [527] [DEBUG] [app] Ending request: urn:request:03e78f0c-d10a-441e-afe4-d23b86078b37 (/health/instance) 2020-03-31 09:30:48,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:48,332 [527] [DEBUG] [app] Ending request: urn:request:03e78f0c-d10a-441e-afe4-d23b86078b37 (/health/instance) 2020-03-31 09:30:48,333 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:48,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:30:48,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:30:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:30:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:54.323352+00:00 (in 4.821861 seconds) 2020-03-31 09:30:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:49 UTC)" (scheduled at 2020-03-31 09:30:49.498966+00:00) 2020-03-31 09:30:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:30:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:54.323352+00:00 (in 4.821861 seconds) 2020-03-31 09:30:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:49 UTC)" (scheduled at 2020-03-31 09:30:49.498966+00:00) 2020-03-31 09:30:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:49 UTC)" executed successfully 2020-03-31 09:30:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:30:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:51 UTC)" (scheduled at 2020-03-31 09:30:51.110610+00:00) 2020-03-31 09:30:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:30:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:51 UTC)" (scheduled at 2020-03-31 09:30:51.110610+00:00) 2020-03-31 09:30:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:59.560326+00:00 (in 8.448420 seconds) namespacegcworker stdout | 2020-03-31 09:30:51,113 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:30:59.560326+00:00 (in 8.448420 seconds) 2020-03-31 09:30:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 51, 112732), True, datetime.datetime(2020, 3, 31, 9, 30, 51, 112732), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:30:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 51, 112732), True, datetime.datetime(2020, 3, 31, 9, 30, 51, 112732), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:30:51,146 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:30:51,146 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:51,146 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:51,146 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:30:51,146 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:51,146 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:51 UTC)" executed successfully 2020-03-31 09:30:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:30:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:01.440707+00:00 (in 9.999357 seconds) 2020-03-31 09:30:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:01 UTC)" (scheduled at 2020-03-31 09:30:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:30:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:01.440707+00:00 (in 9.999357 seconds) 2020-03-31 09:30:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:01 UTC)" (scheduled at 2020-03-31 09:30:51.440707+00:00) 2020-03-31 09:30:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:30:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:30:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:30:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:30:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:30:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:01 UTC)" executed successfully 2020-03-31 09:30:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:30:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:22.113221+00:00 (in 29.999212 seconds) 2020-03-31 09:30:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:22 UTC)" (scheduled at 2020-03-31 09:30:52.113221+00:00) securityworker stdout | 2020-03-31 09:30:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:22.113221+00:00 (in 29.999212 seconds) 2020-03-31 09:30:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:22 UTC)" (scheduled at 2020-03-31 09:30:52.113221+00:00) 2020-03-31 09:30:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:30:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:30:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:30:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:30:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:30:52,130 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:30:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:30:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:30:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:22 UTC)" executed successfully 2020-03-31 09:30:53,713 [170] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2020-03-31 09:30:53,713 [170] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:53,714 [170] [INFO] [apscheduler.executors.default] Running job "_cleanup_queue (trigger: interval[1 day, 0:00:00], next run at: 2020-03-31 09:30:53 UTC)" (scheduled at 2020-03-31 09:30:53.713440+00:00) queuecleanupworker stdout | 2020-03-31 09:30:53,714 [170] [INFO] [apscheduler.executors.default] Running job "_cleanup_queue (trigger: interval[1 day, 0:00:00], next run at: 2020-03-31 09:30:53 UTC)" (scheduled at 2020-03-31 09:30:53.713440+00:00) 2020-03-31 09:30:53,716 [170] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."processing_expires" <= %s) LIMIT %s', [datetime.datetime(2020, 3, 30, 9, 30, 53, 714862), 500]) queuecleanupworker stdout | 2020-03-31 09:30:53,716 [170] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."processing_expires" <= %s) LIMIT %s', [datetime.datetime(2020, 3, 30, 9, 30, 53, 714862), 500]) 2020-03-31 09:30:53,716 [170] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-04-01 09:30:53.713440+00:00 (in 86399.999503 seconds) queuecleanupworker stdout | 2020-03-31 09:30:53,716 [170] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-04-01 09:30:53.713440+00:00 (in 86399.999503 seconds) 2020-03-31 09:30:53,730 [170] [DEBUG] [data.database] Disconnecting from database. queuecleanupworker stdout | 2020-03-31 09:30:53,730 [170] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:53,730 [170] [INFO] [apscheduler.executors.default] Job "_cleanup_queue (trigger: interval[1 day, 0:00:00], next run at: 2020-04-01 09:30:53 UTC)" executed successfully queuecleanupworker stdout | 2020-03-31 09:30:53,730 [170] [INFO] [apscheduler.executors.default] Job "_cleanup_queue (trigger: interval[1 day, 0:00:00], next run at: 2020-04-01 09:30:53 UTC)" executed successfully 2020-03-31 09:30:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:30:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:04.323352+00:00 (in 9.999042 seconds) 2020-03-31 09:30:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:04 UTC)" (scheduled at 2020-03-31 09:30:54.323352+00:00) 2020-03-31 09:30:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:30:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:04.323352+00:00 (in 9.999042 seconds) 2020-03-31 09:30:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:04 UTC)" (scheduled at 2020-03-31 09:30:54.323352+00:00) 2020-03-31 09:30:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 54, 325287), True, datetime.datetime(2020, 3, 31, 9, 30, 54, 325287), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:30:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 54, 325287), True, datetime.datetime(2020, 3, 31, 9, 30, 54, 325287), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:30:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:30:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:30:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:30:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:04 UTC)" executed successfully 2020-03-31 09:30:54,446 [518] [DEBUG] [app] Starting request: urn:request:7f35a01c-b925-4e97-863f-ef77df726524 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:54,446 [518] [DEBUG] [app] Starting request: urn:request:7f35a01c-b925-4e97-863f-ef77df726524 (/health/instance) 2020-03-31 09:30:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:54,453 [552] [DEBUG] [app] Starting request: urn:request:f559cb97-7254-4880-bb3c-06d4d96689b8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:54,453 [552] [DEBUG] [app] Starting request: urn:request:f559cb97-7254-4880-bb3c-06d4d96689b8 (/v1/_internal_ping) 2020-03-31 09:30:54,455 [552] [DEBUG] [app] Ending request: urn:request:f559cb97-7254-4880-bb3c-06d4d96689b8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:54,455 [552] [DEBUG] [app] Ending request: urn:request:f559cb97-7254-4880-bb3c-06d4d96689b8 (/v1/_internal_ping) 2020-03-31 09:30:54,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:30:54,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:54,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:30:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:30:54,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:54,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:54,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:54,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:54,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:54,485 [518] [DEBUG] [app] Starting request: urn:request:f7f21f5c-4e7f-4edf-a8f9-cd318d02694e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:54,485 [518] [DEBUG] [app] Starting request: urn:request:f7f21f5c-4e7f-4edf-a8f9-cd318d02694e (/_internal_ping) 2020-03-31 09:30:54,486 [518] [DEBUG] [app] Ending request: urn:request:f7f21f5c-4e7f-4edf-a8f9-cd318d02694e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:54,486 [518] [DEBUG] [app] Ending request: urn:request:f7f21f5c-4e7f-4edf-a8f9-cd318d02694e (/_internal_ping) 2020-03-31 09:30:54,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:30:54,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:30:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:30:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:30:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:30:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:30:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:54,498 [534] [DEBUG] [app] Starting request: urn:request:28da059f-a577-43d0-9daf-be9a79b39054 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:54,498 [534] [DEBUG] [app] Starting request: urn:request:28da059f-a577-43d0-9daf-be9a79b39054 (/c1/_internal_ping) 2020-03-31 09:30:54,499 [534] [DEBUG] [app] Ending request: urn:request:28da059f-a577-43d0-9daf-be9a79b39054 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:54,499 [534] [DEBUG] [app] Ending request: urn:request:28da059f-a577-43d0-9daf-be9a79b39054 (/c1/_internal_ping) 2020-03-31 09:30:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:54,499 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:30:54,501 [518] [DEBUG] [app] Ending request: urn:request:7f35a01c-b925-4e97-863f-ef77df726524 (/health/instance) 2020-03-31 09:30:54,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:30:54,499 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:30:54,501 [518] [DEBUG] [app] Ending request: urn:request:7f35a01c-b925-4e97-863f-ef77df726524 (/health/instance) 2020-03-31 09:30:54,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:30:56,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:06.924497+00:00 (in 9.996735 seconds) 2020-03-31 09:30:56,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:06 UTC)" (scheduled at 2020-03-31 09:30:56.924497+00:00) 2020-03-31 09:30:56,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:30:56,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 56, 928499), True, datetime.datetime(2020, 3, 31, 9, 30, 56, 928499), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:30:56,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:30:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:06.924497+00:00 (in 9.996735 seconds) 2020-03-31 09:30:56,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:06 UTC)" (scheduled at 2020-03-31 09:30:56.924497+00:00) 2020-03-31 09:30:56,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:30:56,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 30, 56, 928499), True, datetime.datetime(2020, 3, 31, 9, 30, 56, 928499), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:30:56,946 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 50, 56, 928499), 1, u'd3e7eb63-7220-4ec9-b81f-d6a777c7cab0', 3, u'1485479f-4b38-48af-b179-ecc0a45d59a5']) storagereplication stdout | 2020-03-31 09:30:56,946 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 50, 56, 928499), 1, u'd3e7eb63-7220-4ec9-b81f-d6a777c7cab0', 3, u'1485479f-4b38-48af-b179-ecc0a45d59a5']) 2020-03-31 09:30:56,953 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} storagereplication stdout | 2020-03-31 09:30:56,953 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:30:56,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:56,954 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 storagereplication stdout | 2020-03-31 09:30:56,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:56,954 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 2020-03-31 09:30:56,955 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:30:56,955 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:30:56,973 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:30:56,973 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:30:56,978 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) storagereplication stdout | 2020-03-31 09:30:56,978 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) 2020-03-31 09:30:56,982 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:30:56,982 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:30:56,983 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:53361052-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:30:56 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:30:56,983 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=53361052-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '53361052-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:o1MfhNKSJWpfVwARkppFwd/XHa/sFntCBuiZtkw6L8s=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:56 GMT'}. storagereplication stdout | 2020-03-31 09:30:56,983 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:53361052-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:30:56 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:30:56,983 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=53361052-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '53361052-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:o1MfhNKSJWpfVwARkppFwd/XHa/sFntCBuiZtkw6L8s=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:56 GMT'}. 2020-03-31 09:30:56,985 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:30:56,985 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:30:57,104 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:30:57,104 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:30:57,105 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=53361052-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:56 GMT, Server-Request-ID=7772f851-401e-009e-6f3f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '53361052-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:o1MfhNKSJWpfVwARkppFwd/XHa/sFntCBuiZtkw6L8s=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:56 GMT'}. storagereplication stdout | 2020-03-31 09:30:57,105 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=53361052-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:56 GMT, Server-Request-ID=7772f851-401e-009e-6f3f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '53361052-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:o1MfhNKSJWpfVwARkppFwd/XHa/sFntCBuiZtkw6L8s=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:56 GMT'}. 2020-03-31 09:30:57,106 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:30:57,106 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:30:57,106 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:30:57,106 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:57,106 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:30:57,106 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:30:57,107 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:5348f3fc-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=EnNXzTEdqmaRqRsWnzj/ZzkoXfikARB2mkhlZ5cGrik%3D&sv=2017-07-29&se=2020-03-31T09%3A31%3A57Z x-ms-date:Tue, 31 Mar 2020 09:30:57 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:30:57,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5348f3fc-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=EnNXzTEdqmaRqRsWnzj/ZzkoXfikARB2mkhlZ5cGrik%3D&sv=2017-07-29&se=2020-03-31T09%3A31%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:RPmhOYIT50Jbbqu6+7+zgXmXZ0oiCnzih72uDBK9+Ok=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '5348f3fc-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. storagereplication stdout | 2020-03-31 09:30:57,107 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:5348f3fc-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=EnNXzTEdqmaRqRsWnzj/ZzkoXfikARB2mkhlZ5cGrik%3D&sv=2017-07-29&se=2020-03-31T09%3A31%3A57Z x-ms-date:Tue, 31 Mar 2020 09:30:57 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:30:57,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5348f3fc-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=EnNXzTEdqmaRqRsWnzj/ZzkoXfikARB2mkhlZ5cGrik%3D&sv=2017-07-29&se=2020-03-31T09%3A31%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:RPmhOYIT50Jbbqu6+7+zgXmXZ0oiCnzih72uDBK9+Ok=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '5348f3fc-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. 2020-03-31 09:30:57,209 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:30:57,209 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 2020-03-31 09:30:57,210 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5348f3fc-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:56 GMT, Server-Request-ID=7772f854-401e-009e-703f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=EnNXzTEdqmaRqRsWnzj/ZzkoXfikARB2mkhlZ5cGrik%3D&sv=2017-07-29&se=2020-03-31T09%3A31%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:RPmhOYIT50Jbbqu6+7+zgXmXZ0oiCnzih72uDBK9+Ok=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '5348f3fc-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. storagereplication stdout | 2020-03-31 09:30:57,210 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5348f3fc-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:56 GMT, Server-Request-ID=7772f854-401e-009e-703f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=EnNXzTEdqmaRqRsWnzj/ZzkoXfikARB2mkhlZ5cGrik%3D&sv=2017-07-29&se=2020-03-31T09%3A31%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:RPmhOYIT50Jbbqu6+7+zgXmXZ0oiCnzih72uDBK9+Ok=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '5348f3fc-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. 2020-03-31 09:30:57,211 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:30:57,211 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:30:57,211 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:5358dd44-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:30:57 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:30:57,211 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5358dd44-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:/z+rTGIfOMVNSpmxeM73AAeSzaVTMfGT58OXLAfhMWg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. storagereplication stdout | 2020-03-31 09:30:57,211 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:5358dd44-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:30:57 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:30:57,211 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5358dd44-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:/z+rTGIfOMVNSpmxeM73AAeSzaVTMfGT58OXLAfhMWg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. 2020-03-31 09:30:57,213 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:30:57,213 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:30:57,678 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:30:57,678 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:30:57,679 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:57 GMT, Server-Request-ID=7b7e4e11-601e-0049-4a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5358dd44-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:/z+rTGIfOMVNSpmxeM73AAeSzaVTMfGT58OXLAfhMWg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. 2020-03-31 09:30:57,679 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:30:57 GMT, Server-Request-ID=7b7e4e11-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:30:57,679 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:30:57 GMT, Server-Request-ID=7b7e4e11-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:30:57,679 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:57 GMT, Server-Request-ID=7b7e4e11-601e-0049-4a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5358dd44-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:/z+rTGIfOMVNSpmxeM73AAeSzaVTMfGT58OXLAfhMWg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:57 GMT'}. 2020-03-31 09:30:57,679 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:30:57 GMT, Server-Request-ID=7b7e4e11-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:30:57,679 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=5358dd44-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:30:57 GMT, Server-Request-ID=7b7e4e11-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:30:57,679 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:30:57,679 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:30:58,273 [518] [DEBUG] [app] Starting request: urn:request:aa8d7184-5f59-43e7-af83-796fe1d54ef0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:30:58,273 [518] [DEBUG] [app] Starting request: urn:request:aa8d7184-5f59-43e7-af83-796fe1d54ef0 (/health/instance) 2020-03-31 09:30:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:58,277 [550] [DEBUG] [app] Starting request: urn:request:3e463006-e4b0-4916-9aa2-d43b4674d69f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:58,277 [550] [DEBUG] [app] Starting request: urn:request:3e463006-e4b0-4916-9aa2-d43b4674d69f (/v1/_internal_ping) 2020-03-31 09:30:58,278 [550] [DEBUG] [app] Ending request: urn:request:3e463006-e4b0-4916-9aa2-d43b4674d69f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:30:58,278 [550] [DEBUG] [app] Ending request: urn:request:3e463006-e4b0-4916-9aa2-d43b4674d69f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:58,279 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:30:58,279 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:30:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:58,281 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:30:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:30:58,281 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:30:58,282 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:30:58,282 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:30:58,283 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:30:58,283 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:30:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:30:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:30:58,321 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:30:58,321 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:30:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:58,327 [518] [DEBUG] [app] Starting request: urn:request:9e61a412-ef2b-4649-9ea9-bea1492b74c4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:58,327 [518] [DEBUG] [app] Starting request: urn:request:9e61a412-ef2b-4649-9ea9-bea1492b74c4 (/_internal_ping) 2020-03-31 09:30:58,328 [518] [DEBUG] [app] Ending request: urn:request:9e61a412-ef2b-4649-9ea9-bea1492b74c4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:30:58,328 [518] [DEBUG] [app] Ending request: urn:request:9e61a412-ef2b-4649-9ea9-bea1492b74c4 (/_internal_ping) 2020-03-31 09:30:58,328 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:58,328 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:30:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:30:58,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:58,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:58,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:30:58,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:30:58,337 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:58,337 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:58,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:30:58,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:30:58,342 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:30:58,342 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:30:58,343 [543] [DEBUG] [app] Starting request: urn:request:fe4b64c6-6431-41f9-96b8-c267960b26ff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:58,343 [543] [DEBUG] [app] Starting request: urn:request:fe4b64c6-6431-41f9-96b8-c267960b26ff (/c1/_internal_ping) 2020-03-31 09:30:58,344 [543] [DEBUG] [app] Ending request: urn:request:fe4b64c6-6431-41f9-96b8-c267960b26ff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:30:58,344 [543] [DEBUG] [app] Ending request: urn:request:fe4b64c6-6431-41f9-96b8-c267960b26ff (/c1/_internal_ping) 2020-03-31 09:30:58,345 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:30:58,345 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:30:58,345 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:30:58,345 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:30:58,347 [518] [DEBUG] [app] Ending request: urn:request:aa8d7184-5f59-43e7-af83-796fe1d54ef0 (/health/instance) 2020-03-31 09:30:58,347 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:30:58,347 [518] [DEBUG] [app] Ending request: urn:request:aa8d7184-5f59-43e7-af83-796fe1d54ef0 (/health/instance) 2020-03-31 09:30:58,347 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:30:58,348 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) gunicorn-web stdout | 2020-03-31 09:30:58,348 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:30:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:30:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) 2020-03-31 09:30:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:30:59,562 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:30:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:59 UTC)" (scheduled at 2020-03-31 09:30:59.560326+00:00) 2020-03-31 09:30:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:51.110610+00:00 (in 51.548151 seconds) 2020-03-31 09:30:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:30:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:30:59 UTC)" (scheduled at 2020-03-31 09:30:59.560326+00:00) 2020-03-31 09:30:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:51.110610+00:00 (in 51.548151 seconds) 2020-03-31 09:30:59,563 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:30:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:30:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:31:59 UTC)" executed successfully 2020-03-31 09:30:59,681 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:54d1c9ce-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:30:59 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:30:59,681 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:54d1c9ce-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:30:59 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:30:59,682 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54d1c9ce-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jyKmSs0jCAoEoXwQqvL/DZ9zkV7/2xr3/epEyFLD2y8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:59 GMT'}. storagereplication stdout | 2020-03-31 09:30:59,682 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54d1c9ce-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jyKmSs0jCAoEoXwQqvL/DZ9zkV7/2xr3/epEyFLD2y8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:59 GMT'}. 2020-03-31 09:30:59,769 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:30:59,769 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:30:59,770 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:59 GMT, Server-Request-ID=7b7e5364-601e-0049-0f3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54d1c9ce-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jyKmSs0jCAoEoXwQqvL/DZ9zkV7/2xr3/epEyFLD2y8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:59 GMT'}. 2020-03-31 09:30:59,770 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:30:59 GMT, Server-Request-ID=7b7e5364-601e-0049-0f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:30:59,770 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:30:59 GMT, Server-Request-ID=7b7e5364-601e-0049-0f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:30:59,770 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:30:59 GMT, Server-Request-ID=7b7e5364-601e-0049-0f3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54d1c9ce-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:jyKmSs0jCAoEoXwQqvL/DZ9zkV7/2xr3/epEyFLD2y8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:30:59 GMT'}. 2020-03-31 09:30:59,770 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:30:59 GMT, Server-Request-ID=7b7e5364-601e-0049-0f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:30:59,770 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=54d1c9ce-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:30:59 GMT, Server-Request-ID=7b7e5364-601e-0049-0f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:30:59,771 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:30:59,771 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:31:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:31:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:30.337699+00:00 (in 29.999004 seconds) 2020-03-31 09:31:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:30 UTC)" (scheduled at 2020-03-31 09:31:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:31:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:30.337699+00:00 (in 29.999004 seconds) 2020-03-31 09:31:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:30 UTC)" (scheduled at 2020-03-31 09:31:00.337699+00:00) 2020-03-31 09:31:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 31, 0, 339658), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:31:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 31, 0, 339658), False, 50, 1, 0]) 2020-03-31 09:31:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:31:00,356 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:31:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:31:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:31:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:30 UTC)" executed successfully 2020-03-31 09:31:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:31:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:11.440707+00:00 (in 9.999018 seconds) 2020-03-31 09:31:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:11 UTC)" (scheduled at 2020-03-31 09:31:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:31:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:11.440707+00:00 (in 9.999018 seconds) 2020-03-31 09:31:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:11 UTC)" (scheduled at 2020-03-31 09:31:01.440707+00:00) 2020-03-31 09:31:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:31:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:31:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:31:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:31:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:31:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:11 UTC)" executed successfully 2020-03-31 09:31:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:31:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:24.410927+00:00 (in 21.079994 seconds) 2020-03-31 09:31:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:33 UTC)" (scheduled at 2020-03-31 09:31:03.330112+00:00) 2020-03-31 09:31:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:31:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:24.410927+00:00 (in 21.079994 seconds) 2020-03-31 09:31:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:33 UTC)" (scheduled at 2020-03-31 09:31:03.330112+00:00) 2020-03-31 09:31:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 3, 331963), True, datetime.datetime(2020, 3, 31, 9, 31, 3, 331963), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:31:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 3, 331963), True, datetime.datetime(2020, 3, 31, 9, 31, 3, 331963), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:31:03,354 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:31:03,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:31:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:33 UTC)" executed successfully 2020-03-31 09:31:03,774 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:57424f6c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:31:03,774 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '57424f6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:UAQ0F40LejLDA0t1D/OeGrsph+Eo3Ql94ccxynZ76n4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:03 GMT'}. storagereplication stdout | 2020-03-31 09:31:03,774 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:57424f6c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:31:03,774 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '57424f6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:UAQ0F40LejLDA0t1D/OeGrsph+Eo3Ql94ccxynZ76n4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:03 GMT'}. 2020-03-31 09:31:03,862 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:31:03,862 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:31:03,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:03 GMT, Server-Request-ID=7b7e5de8-601e-0049-5b3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '57424f6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:UAQ0F40LejLDA0t1D/OeGrsph+Eo3Ql94ccxynZ76n4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:03 GMT'}. 2020-03-31 09:31:03,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:03 GMT, Server-Request-ID=7b7e5de8-601e-0049-5b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:03,863 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:03 GMT, Server-Request-ID=7b7e5de8-601e-0049-5b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:03,863 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:31:03,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:03 GMT, Server-Request-ID=7b7e5de8-601e-0049-5b3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '57424f6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:UAQ0F40LejLDA0t1D/OeGrsph+Eo3Ql94ccxynZ76n4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:03 GMT'}. 2020-03-31 09:31:03,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:03 GMT, Server-Request-ID=7b7e5de8-601e-0049-5b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:03,863 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=57424f6c-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:03 GMT, Server-Request-ID=7b7e5de8-601e-0049-5b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:03,863 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:31:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:31:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:14 UTC)" (scheduled at 2020-03-31 09:31:04.323352+00:00) 2020-03-31 09:31:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:14.323352+00:00 (in 9.998721 seconds) notificationworker stdout | 2020-03-31 09:31:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:14 UTC)" (scheduled at 2020-03-31 09:31:04.323352+00:00) 2020-03-31 09:31:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:14.323352+00:00 (in 9.998721 seconds) 2020-03-31 09:31:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 4, 325680), True, datetime.datetime(2020, 3, 31, 9, 31, 4, 325680), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:31:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 4, 325680), True, datetime.datetime(2020, 3, 31, 9, 31, 4, 325680), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:31:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:31:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:31:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:04,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:14 UTC)" executed successfully 2020-03-31 09:31:04,446 [518] [DEBUG] [app] Starting request: urn:request:0d8fc75f-562d-428e-a5a6-eab46341c9de (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:04,446 [518] [DEBUG] [app] Starting request: urn:request:0d8fc75f-562d-428e-a5a6-eab46341c9de (/health/instance) 2020-03-31 09:31:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:04,453 [547] [DEBUG] [app] Starting request: urn:request:88cc79cd-0d4f-4caf-9ab4-52d4ad8c3094 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:04,453 [547] [DEBUG] [app] Starting request: urn:request:88cc79cd-0d4f-4caf-9ab4-52d4ad8c3094 (/v1/_internal_ping) 2020-03-31 09:31:04,454 [547] [DEBUG] [app] Ending request: urn:request:88cc79cd-0d4f-4caf-9ab4-52d4ad8c3094 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:04,454 [547] [DEBUG] [app] Ending request: urn:request:88cc79cd-0d4f-4caf-9ab4-52d4ad8c3094 (/v1/_internal_ping) 2020-03-31 09:31:04,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:31:04,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:04,457 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:31:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:31:04,457 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:04,458 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:04,458 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:04,459 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:04,459 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:04,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:04,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:04,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:04,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:04,488 [527] [DEBUG] [app] Starting request: urn:request:17b05d3b-6817-4cc6-8e2d-3f4b27d06fc1 (/_internal_ping) 2020-03-31 09:31:04,489 [527] [DEBUG] [app] Ending request: urn:request:17b05d3b-6817-4cc6-8e2d-3f4b27d06fc1 (/_internal_ping) 2020-03-31 09:31:04,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:04,488 [527] [DEBUG] [app] Starting request: urn:request:17b05d3b-6817-4cc6-8e2d-3f4b27d06fc1 (/_internal_ping) 2020-03-31 09:31:04,489 [527] [DEBUG] [app] Ending request: urn:request:17b05d3b-6817-4cc6-8e2d-3f4b27d06fc1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:04,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:31:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:31:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:31:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:31:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:31:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:31:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:04,505 [543] [DEBUG] [app] Starting request: urn:request:9f19e331-0919-4286-aa13-09013d77fc61 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:04,505 [543] [DEBUG] [app] Starting request: urn:request:9f19e331-0919-4286-aa13-09013d77fc61 (/c1/_internal_ping) 2020-03-31 09:31:04,506 [543] [DEBUG] [app] Ending request: urn:request:9f19e331-0919-4286-aa13-09013d77fc61 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:04,506 [543] [DEBUG] [app] Ending request: urn:request:9f19e331-0919-4286-aa13-09013d77fc61 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:04,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:31:04,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:04,508 [518] [DEBUG] [app] Ending request: urn:request:0d8fc75f-562d-428e-a5a6-eab46341c9de (/health/instance) 2020-03-31 09:31:04,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:04,508 [518] [DEBUG] [app] Ending request: urn:request:0d8fc75f-562d-428e-a5a6-eab46341c9de (/health/instance) 2020-03-31 09:31:04,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:31:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:31:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:31:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:16.924497+00:00 (in 9.999241 seconds) storagereplication stdout | 2020-03-31 09:31:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:16.924497+00:00 (in 9.999241 seconds) 2020-03-31 09:31:08,272 [527] [DEBUG] [app] Starting request: urn:request:8d2c55c2-9ee2-45e2-895f-19ddf9934bb2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:08,272 [527] [DEBUG] [app] Starting request: urn:request:8d2c55c2-9ee2-45e2-895f-19ddf9934bb2 (/health/instance) 2020-03-31 09:31:08,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:08,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:08,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:08,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:08,280 [545] [DEBUG] [app] Starting request: urn:request:ee0c526b-f77b-4e34-8873-28be5e6397b3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:08,280 [545] [DEBUG] [app] Starting request: urn:request:ee0c526b-f77b-4e34-8873-28be5e6397b3 (/v1/_internal_ping) 2020-03-31 09:31:08,282 [545] [DEBUG] [app] Ending request: urn:request:ee0c526b-f77b-4e34-8873-28be5e6397b3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:08,282 [545] [DEBUG] [app] Ending request: urn:request:ee0c526b-f77b-4e34-8873-28be5e6397b3 (/v1/_internal_ping) 2020-03-31 09:31:08,282 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:31:08,282 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:08,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:08,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:31:08,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:08,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:08,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:08,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:08,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:08,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:08,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:08,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:08,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:08,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:08,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:08,313 [527] [DEBUG] [app] Starting request: urn:request:bfc86def-1d66-4a09-8284-77ea261910af (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:08,313 [527] [DEBUG] [app] Starting request: urn:request:bfc86def-1d66-4a09-8284-77ea261910af (/_internal_ping) 2020-03-31 09:31:08,314 [527] [DEBUG] [app] Ending request: urn:request:bfc86def-1d66-4a09-8284-77ea261910af (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:08,314 [527] [DEBUG] [app] Ending request: urn:request:bfc86def-1d66-4a09-8284-77ea261910af (/_internal_ping) 2020-03-31 09:31:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:08,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:08,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:31:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:31:08,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:08,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:08,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:31:08,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:31:08,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:08,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:08,327 [534] [DEBUG] [app] Starting request: urn:request:0050125b-ffc6-4828-a42c-ecf745490dc7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:08,327 [534] [DEBUG] [app] Starting request: urn:request:0050125b-ffc6-4828-a42c-ecf745490dc7 (/c1/_internal_ping) 2020-03-31 09:31:08,328 [534] [DEBUG] [app] Ending request: urn:request:0050125b-ffc6-4828-a42c-ecf745490dc7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:08,328 [534] [DEBUG] [app] Ending request: urn:request:0050125b-ffc6-4828-a42c-ecf745490dc7 (/c1/_internal_ping) 2020-03-31 09:31:08,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:08,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:08,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:08,330 [527] [DEBUG] [app] Ending request: urn:request:8d2c55c2-9ee2-45e2-895f-19ddf9934bb2 (/health/instance) 2020-03-31 09:31:08,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:31:08,329 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:08,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:31:08,330 [527] [DEBUG] [app] Ending request: urn:request:8d2c55c2-9ee2-45e2-895f-19ddf9934bb2 (/health/instance) 2020-03-31 09:31:08,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:08,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:31:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:31:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:21.440707+00:00 (in 9.999254 seconds) 2020-03-31 09:31:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:21 UTC)" (scheduled at 2020-03-31 09:31:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:31:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:21.440707+00:00 (in 9.999254 seconds) 2020-03-31 09:31:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:21 UTC)" (scheduled at 2020-03-31 09:31:11.440707+00:00) 2020-03-31 09:31:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:31:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:31:11,469 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:31:11,469 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:31:11,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:11,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:31:11,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:11,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:21 UTC)" executed successfully 2020-03-31 09:31:11,873 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:5c161fa0-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:11 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:31:11,874 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5c161fa0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1dRRSGYjpmN5GFTzzBptY/y+vw5rXlqlNqpeuMlSlrw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:11 GMT'}. storagereplication stdout | 2020-03-31 09:31:11,873 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:5c161fa0-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:11 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:31:11,874 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5c161fa0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1dRRSGYjpmN5GFTzzBptY/y+vw5rXlqlNqpeuMlSlrw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:11 GMT'}. 2020-03-31 09:31:11,963 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:31:11,963 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:31:11,964 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:11 GMT, Server-Request-ID=7b7e7340-601e-0049-243f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5c161fa0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1dRRSGYjpmN5GFTzzBptY/y+vw5rXlqlNqpeuMlSlrw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:11 GMT'}. 2020-03-31 09:31:11,964 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:11 GMT, Server-Request-ID=7b7e7340-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:31:11,964 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:11 GMT, Server-Request-ID=7b7e7340-601e-0049-243f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5c161fa0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:1dRRSGYjpmN5GFTzzBptY/y+vw5rXlqlNqpeuMlSlrw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:11 GMT'}. 2020-03-31 09:31:11,964 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:11 GMT, Server-Request-ID=7b7e7340-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:11,964 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:11 GMT, Server-Request-ID=7b7e7340-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:11,964 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:31:11,964 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=5c161fa0-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:11 GMT, Server-Request-ID=7b7e7340-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:11,964 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:31:14,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:31:14,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:24.323352+00:00 (in 9.995101 seconds) 2020-03-31 09:31:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:24 UTC)" (scheduled at 2020-03-31 09:31:14.323352+00:00) notificationworker stdout | 2020-03-31 09:31:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:24.323352+00:00 (in 9.995101 seconds) 2020-03-31 09:31:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:24 UTC)" (scheduled at 2020-03-31 09:31:14.323352+00:00) 2020-03-31 09:31:14,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:31:14,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 14, 329241), True, datetime.datetime(2020, 3, 31, 9, 31, 14, 329241), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:31:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 14, 329241), True, datetime.datetime(2020, 3, 31, 9, 31, 14, 329241), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:31:14,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:31:14,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:14,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:14,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:31:14,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:14,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:24 UTC)" executed successfully 2020-03-31 09:31:14,446 [518] [DEBUG] [app] Starting request: urn:request:268eb4b2-c627-4dff-b3b9-00f9ce16bb70 (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:14,446 [518] [DEBUG] [app] Starting request: urn:request:268eb4b2-c627-4dff-b3b9-00f9ce16bb70 (/health/instance) 2020-03-31 09:31:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:14,456 [538] [DEBUG] [app] Starting request: urn:request:2fc869b4-4fd6-476a-9a52-851521800c88 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:14,456 [538] [DEBUG] [app] Starting request: urn:request:2fc869b4-4fd6-476a-9a52-851521800c88 (/v1/_internal_ping) 2020-03-31 09:31:14,459 [538] [DEBUG] [app] Ending request: urn:request:2fc869b4-4fd6-476a-9a52-851521800c88 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:14,459 [538] [DEBUG] [app] Ending request: urn:request:2fc869b4-4fd6-476a-9a52-851521800c88 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) 2020-03-31 09:31:14,460 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:31:14,460 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:14,461 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:31:14,461 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:14,462 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:14,462 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:14,463 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:14,463 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:14,464 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:14,464 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:14,465 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:14,465 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:14,484 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:14,484 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:14,504 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:14,504 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:14,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:14,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:14,509 [518] [DEBUG] [app] Starting request: urn:request:2663b559-b1e4-4f8d-8a49-1ce8378d7749 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:14,509 [518] [DEBUG] [app] Starting request: urn:request:2663b559-b1e4-4f8d-8a49-1ce8378d7749 (/_internal_ping) 2020-03-31 09:31:14,511 [518] [DEBUG] [app] Ending request: urn:request:2663b559-b1e4-4f8d-8a49-1ce8378d7749 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:31:14,511 [518] [DEBUG] [app] Ending request: urn:request:2663b559-b1e4-4f8d-8a49-1ce8378d7749 (/_internal_ping) 2020-03-31 09:31:14,512 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:14,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:31:14,512 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:14,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:14,518 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:14,518 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:14,518 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:31:14,518 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:14,521 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:14,521 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:31:14,522 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:14,522 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:14,525 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:14,525 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:14,527 [543] [DEBUG] [app] Starting request: urn:request:4540c8d1-7f5b-440b-ab10-0654e09327a3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:14,527 [543] [DEBUG] [app] Starting request: urn:request:4540c8d1-7f5b-440b-ab10-0654e09327a3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:14,528 [543] [DEBUG] [app] Ending request: urn:request:4540c8d1-7f5b-440b-ab10-0654e09327a3 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:14,529 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:14,529 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:31:14,528 [543] [DEBUG] [app] Ending request: urn:request:4540c8d1-7f5b-440b-ab10-0654e09327a3 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:14,529 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:14,530 [518] [DEBUG] [app] Ending request: urn:request:268eb4b2-c627-4dff-b3b9-00f9ce16bb70 (/health/instance) 2020-03-31 09:31:14,531 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:31:14,529 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:14,531 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.086 49 0.086) gunicorn-web stdout | 2020-03-31 09:31:14,530 [518] [DEBUG] [app] Ending request: urn:request:268eb4b2-c627-4dff-b3b9-00f9ce16bb70 (/health/instance) 2020-03-31 09:31:14,531 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:14,531 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.086 49 0.086) 2020-03-31 09:31:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:31:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:45 UTC)" (scheduled at 2020-03-31 09:31:15.064514+00:00) gcworker stdout | 2020-03-31 09:31:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:45 UTC)" (scheduled at 2020-03-31 09:31:15.064514+00:00) 2020-03-31 09:31:15,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:45.064514+00:00 (in 29.998415 seconds) gcworker stdout | 2020-03-31 09:31:15,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:45.064514+00:00 (in 29.998415 seconds) 2020-03-31 09:31:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437475, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:31:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437475, 1209600, 500, 1, 0]) 2020-03-31 09:31:15,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:31:15,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:31:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:31:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:45 UTC)" executed successfully 2020-03-31 09:31:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:31:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:26.924497+00:00 (in 9.999131 seconds) storagereplication stdout | 2020-03-31 09:31:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:26.924497+00:00 (in 9.999131 seconds) 2020-03-31 09:31:18,273 [518] [DEBUG] [app] Starting request: urn:request:75d123e6-1a29-4b60-9e1e-861301bdb684 (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:18,273 [518] [DEBUG] [app] Starting request: urn:request:75d123e6-1a29-4b60-9e1e-861301bdb684 (/health/instance) 2020-03-31 09:31:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:18,278 [538] [DEBUG] [app] Starting request: urn:request:16ab1bc0-fc62-4109-aa47-5e7508653e66 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:18,278 [538] [DEBUG] [app] Starting request: urn:request:16ab1bc0-fc62-4109-aa47-5e7508653e66 (/v1/_internal_ping) 2020-03-31 09:31:18,279 [538] [DEBUG] [app] Ending request: urn:request:16ab1bc0-fc62-4109-aa47-5e7508653e66 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:18,279 [538] [DEBUG] [app] Ending request: urn:request:16ab1bc0-fc62-4109-aa47-5e7508653e66 (/v1/_internal_ping) 2020-03-31 09:31:18,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:31:18,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:18,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:31:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:18,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:18,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:18,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:18,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:18,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:18,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:18,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:18,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:18,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:18,312 [518] [DEBUG] [app] Starting request: urn:request:1c9486c7-2d60-49fc-8783-e849313c4d94 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:18,312 [518] [DEBUG] [app] Starting request: urn:request:1c9486c7-2d60-49fc-8783-e849313c4d94 (/_internal_ping) 2020-03-31 09:31:18,313 [518] [DEBUG] [app] Ending request: urn:request:1c9486c7-2d60-49fc-8783-e849313c4d94 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:18,313 [518] [DEBUG] [app] Ending request: urn:request:1c9486c7-2d60-49fc-8783-e849313c4d94 (/_internal_ping) 2020-03-31 09:31:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:31:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:31:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:18,325 [534] [DEBUG] [app] Starting request: urn:request:88540943-0dc7-4333-b6bb-2346ba0b1ec2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:18,325 [534] [DEBUG] [app] Starting request: urn:request:88540943-0dc7-4333-b6bb-2346ba0b1ec2 (/c1/_internal_ping) 2020-03-31 09:31:18,326 [534] [DEBUG] [app] Ending request: urn:request:88540943-0dc7-4333-b6bb-2346ba0b1ec2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:18,326 [534] [DEBUG] [app] Ending request: urn:request:88540943-0dc7-4333-b6bb-2346ba0b1ec2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:18,327 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:31:18,327 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:18,328 [518] [DEBUG] [app] Ending request: urn:request:75d123e6-1a29-4b60-9e1e-861301bdb684 (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:18,328 [518] [DEBUG] [app] Ending request: urn:request:75d123e6-1a29-4b60-9e1e-861301bdb684 (/health/instance) 2020-03-31 09:31:18,328 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:31:18,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:18,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:31:18,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:31:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:31:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:31 UTC)" (scheduled at 2020-03-31 09:31:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:31:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:31 UTC)" (scheduled at 2020-03-31 09:31:21.440707+00:00) 2020-03-31 09:31:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:31.440707+00:00 (in 9.999025 seconds) repositoryactioncounter stdout | 2020-03-31 09:31:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:31.440707+00:00 (in 9.999025 seconds) 2020-03-31 09:31:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:31:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:31:21,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:31:21,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:31:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:31:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:31 UTC)" executed successfully 2020-03-31 09:31:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:31:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:52.113221+00:00 (in 29.998736 seconds) 2020-03-31 09:31:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:52 UTC)" (scheduled at 2020-03-31 09:31:22.113221+00:00) securityworker stdout | 2020-03-31 09:31:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:52.113221+00:00 (in 29.998736 seconds) 2020-03-31 09:31:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:52 UTC)" (scheduled at 2020-03-31 09:31:22.113221+00:00) 2020-03-31 09:31:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:31:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:31:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:31:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:31:22,139 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:31:22,139 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:31:22,139 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:22,139 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:31:22,139 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:22,139 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:31:52 UTC)" executed successfully 2020-03-31 09:31:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:31:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:34.323352+00:00 (in 9.999103 seconds) 2020-03-31 09:31:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:34 UTC)" (scheduled at 2020-03-31 09:31:24.323352+00:00) 2020-03-31 09:31:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:31:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:34.323352+00:00 (in 9.999103 seconds) 2020-03-31 09:31:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:34 UTC)" (scheduled at 2020-03-31 09:31:24.323352+00:00) 2020-03-31 09:31:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 24, 325117), True, datetime.datetime(2020, 3, 31, 9, 31, 24, 325117), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:31:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 24, 325117), True, datetime.datetime(2020, 3, 31, 9, 31, 24, 325117), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:31:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:31:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:31:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:34 UTC)" executed successfully 2020-03-31 09:31:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:31:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:33.330112+00:00 (in 8.918105 seconds) 2020-03-31 09:31:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:24 UTC)" (scheduled at 2020-03-31 09:31:24.410927+00:00) 2020-03-31 09:31:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:31:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:33.330112+00:00 (in 8.918105 seconds) 2020-03-31 09:31:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:24 UTC)" (scheduled at 2020-03-31 09:31:24.410927+00:00) 2020-03-31 09:31:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:24 UTC)" executed successfully 2020-03-31 09:31:24,446 [526] [DEBUG] [app] Starting request: urn:request:0b397294-7518-489a-9dbf-bed994a23fac (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:24,446 [526] [DEBUG] [app] Starting request: urn:request:0b397294-7518-489a-9dbf-bed994a23fac (/health/instance) 2020-03-31 09:31:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:24,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:24,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:24,454 [538] [DEBUG] [app] Starting request: urn:request:bda7f0a3-f8ff-4f7a-8aef-a0921112218d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:24,454 [538] [DEBUG] [app] Starting request: urn:request:bda7f0a3-f8ff-4f7a-8aef-a0921112218d (/v1/_internal_ping) 2020-03-31 09:31:24,455 [538] [DEBUG] [app] Ending request: urn:request:bda7f0a3-f8ff-4f7a-8aef-a0921112218d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:24,455 [538] [DEBUG] [app] Ending request: urn:request:bda7f0a3-f8ff-4f7a-8aef-a0921112218d (/v1/_internal_ping) 2020-03-31 09:31:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:24,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:31:24,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:24,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:24,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:31:24,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:24,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:24,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:24,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:24,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:24,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:24,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:24,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:24,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:24,484 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:24,484 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:24,488 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:24,488 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:24,489 [526] [DEBUG] [app] Starting request: urn:request:af32c0b8-cb96-46cb-8c00-21ad540b0d9d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:24,489 [526] [DEBUG] [app] Starting request: urn:request:af32c0b8-cb96-46cb-8c00-21ad540b0d9d (/_internal_ping) 2020-03-31 09:31:24,490 [526] [DEBUG] [app] Ending request: urn:request:af32c0b8-cb96-46cb-8c00-21ad540b0d9d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:24,490 [526] [DEBUG] [app] Ending request: urn:request:af32c0b8-cb96-46cb-8c00-21ad540b0d9d (/_internal_ping) 2020-03-31 09:31:24,491 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:31:24,491 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:24,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:31:24,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:24,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:24,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:24,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:31:24,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:24,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:24,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:31:24,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:24,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:24,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:24,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:24,502 [534] [DEBUG] [app] Starting request: urn:request:2c277d41-0606-4f92-b648-40a2fffac5d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:24,502 [534] [DEBUG] [app] Starting request: urn:request:2c277d41-0606-4f92-b648-40a2fffac5d2 (/c1/_internal_ping) 2020-03-31 09:31:24,503 [534] [DEBUG] [app] Ending request: urn:request:2c277d41-0606-4f92-b648-40a2fffac5d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:24,503 [534] [DEBUG] [app] Ending request: urn:request:2c277d41-0606-4f92-b648-40a2fffac5d2 (/c1/_internal_ping) 2020-03-31 09:31:24,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:24,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:24,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:24,505 [526] [DEBUG] [app] Ending request: urn:request:0b397294-7518-489a-9dbf-bed994a23fac (/health/instance) 2020-03-31 09:31:24,506 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:31:24,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:24,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:31:24,505 [526] [DEBUG] [app] Ending request: urn:request:0b397294-7518-489a-9dbf-bed994a23fac (/health/instance) 2020-03-31 09:31:24,506 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:24,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:31:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:31:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:36.924497+00:00 (in 9.999221 seconds) storagereplication stdout | 2020-03-31 09:31:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:36.924497+00:00 (in 9.999221 seconds) 2020-03-31 09:31:27,980 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:31:27,980 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy 2020-03-31 09:31:27,981 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:31:27,981 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:31:27,981 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:31:27,981 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:31:27,990 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) storagereplication stdout | 2020-03-31 09:31:27,990 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) 2020-03-31 09:31:27,994 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 36, 27, 989329), True, u'146d5d09-d37f-4627-8baf-e607709b29af', 3]) storagereplication stdout | 2020-03-31 09:31:27,994 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 36, 27, 989329), True, u'146d5d09-d37f-4627-8baf-e607709b29af', 3]) 2020-03-31 09:31:28,000 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 27, 999482), True, datetime.datetime(2020, 3, 31, 9, 31, 27, 999482), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:31:28,000 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 27, 999482), True, datetime.datetime(2020, 3, 31, 9, 31, 27, 999482), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:31:28,005 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 51, 27, 999482), 1, u'af8014ed-2fa8-457e-a28e-0a870e976e20', 4, u'3ebbd6bd-e213-4a1a-b73f-af66d33ccc61']) storagereplication stdout | 2020-03-31 09:31:28,005 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 51, 27, 999482), 1, u'af8014ed-2fa8-457e-a28e-0a870e976e20', 4, u'3ebbd6bd-e213-4a1a-b73f-af66d33ccc61']) 2020-03-31 09:31:28,010 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} storagereplication stdout | 2020-03-31 09:31:28,010 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:31:28,011 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:28,011 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 storagereplication stdout | 2020-03-31 09:31:28,011 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:28,011 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 2020-03-31 09:31:28,013 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:31:28,013 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:31:28,024 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:31:28,024 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:31:28,028 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) storagereplication stdout | 2020-03-31 09:31:28,028 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) 2020-03-31 09:31:28,032 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:31:28,032 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:31:28,033 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:65b7dd3c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:31:28,033 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b7dd3c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65b7dd3c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZW7A1sFkrQOrHuCUnUoMvjHsIIEmd/B3/WB1+xCiIdo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. storagereplication stdout | 2020-03-31 09:31:28,033 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:65b7dd3c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:31:28,033 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b7dd3c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65b7dd3c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZW7A1sFkrQOrHuCUnUoMvjHsIIEmd/B3/WB1+xCiIdo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. 2020-03-31 09:31:28,042 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:31:28,042 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:31:28,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b7dd3c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7772ff39-401e-009e-2c3f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65b7dd3c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZW7A1sFkrQOrHuCUnUoMvjHsIIEmd/B3/WB1+xCiIdo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. storagereplication stdout | 2020-03-31 09:31:28,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b7dd3c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7772ff39-401e-009e-2c3f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65b7dd3c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ZW7A1sFkrQOrHuCUnUoMvjHsIIEmd/B3/WB1+xCiIdo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. 2020-03-31 09:31:28,045 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us storagereplication stdout | 2020-03-31 09:31:28,045 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us 2020-03-31 09:31:28,045 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:28,046 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:31:28,045 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:28,046 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:31:28,046 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:65b9f3ba-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=dQTfMitzAK%2BjCxyMOUTwN/YUeS/aBqEhpp1HoPxbD1k%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A28Z x-ms-date:Tue, 31 Mar 2020 09:31:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:31:28,046 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b9f3ba-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=dQTfMitzAK%2BjCxyMOUTwN/YUeS/aBqEhpp1HoPxbD1k%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:33WbFSHQ6fQ3KiB/Ztq8bjpWIhgNJ4MwIcytTDQwKV0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '65b9f3ba-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. storagereplication stdout | 2020-03-31 09:31:28,046 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:65b9f3ba-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=dQTfMitzAK%2BjCxyMOUTwN/YUeS/aBqEhpp1HoPxbD1k%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A28Z x-ms-date:Tue, 31 Mar 2020 09:31:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:31:28,046 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b9f3ba-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=dQTfMitzAK%2BjCxyMOUTwN/YUeS/aBqEhpp1HoPxbD1k%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:33WbFSHQ6fQ3KiB/Ztq8bjpWIhgNJ4MwIcytTDQwKV0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '65b9f3ba-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. 2020-03-31 09:31:28,071 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:31:28,071 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 2020-03-31 09:31:28,072 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b9f3ba-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7772ff3d-401e-009e-2e3f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=dQTfMitzAK%2BjCxyMOUTwN/YUeS/aBqEhpp1HoPxbD1k%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:33WbFSHQ6fQ3KiB/Ztq8bjpWIhgNJ4MwIcytTDQwKV0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '65b9f3ba-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. storagereplication stdout | 2020-03-31 09:31:28,072 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65b9f3ba-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7772ff3d-401e-009e-2e3f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=dQTfMitzAK%2BjCxyMOUTwN/YUeS/aBqEhpp1HoPxbD1k%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:33WbFSHQ6fQ3KiB/Ztq8bjpWIhgNJ4MwIcytTDQwKV0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '65b9f3ba-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. 2020-03-31 09:31:28,073 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:31:28,073 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:65be1148-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:28 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:31:28,073 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65be1148-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tI49JO/8l/rA/h/L5O/C8CS9kgaiSkEkHWLGdD2wc1M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. storagereplication stdout | 2020-03-31 09:31:28,073 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:31:28,073 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:65be1148-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:28 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:31:28,073 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65be1148-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tI49JO/8l/rA/h/L5O/C8CS9kgaiSkEkHWLGdD2wc1M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. 2020-03-31 09:31:28,167 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:31:28,168 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7b7e9d49-601e-0049-4a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65be1148-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tI49JO/8l/rA/h/L5O/C8CS9kgaiSkEkHWLGdD2wc1M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. 2020-03-31 09:31:28,169 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7b7e9d49-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:28,169 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7b7e9d49-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:28,169 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:31:28,167 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:31:28,168 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7b7e9d49-601e-0049-4a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '65be1148-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tI49JO/8l/rA/h/L5O/C8CS9kgaiSkEkHWLGdD2wc1M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:28 GMT'}. 2020-03-31 09:31:28,169 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7b7e9d49-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:28,169 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=65be1148-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:27 GMT, Server-Request-ID=7b7e9d49-601e-0049-4a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:31:28,169 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:31:28,273 [526] [DEBUG] [app] Starting request: urn:request:978be893-70f8-40c0-bafa-6e8a2213a842 (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:28,273 [526] [DEBUG] [app] Starting request: urn:request:978be893-70f8-40c0-bafa-6e8a2213a842 (/health/instance) 2020-03-31 09:31:28,277 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:28,277 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:28,278 [538] [DEBUG] [app] Starting request: urn:request:650710ac-eaf8-4eb1-83c5-6b4e3d13398c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:28,278 [538] [DEBUG] [app] Starting request: urn:request:650710ac-eaf8-4eb1-83c5-6b4e3d13398c (/v1/_internal_ping) 2020-03-31 09:31:28,279 [538] [DEBUG] [app] Ending request: urn:request:650710ac-eaf8-4eb1-83c5-6b4e3d13398c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:28,279 [538] [DEBUG] [app] Ending request: urn:request:650710ac-eaf8-4eb1-83c5-6b4e3d13398c (/v1/_internal_ping) 2020-03-31 09:31:28,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:31:28,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:28,280 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:28,281 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:28,280 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:28,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:31:28,281 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:31:28,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:28,291 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:28,291 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:28,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:28,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:28,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:28,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:28,313 [526] [DEBUG] [app] Starting request: urn:request:f93cca02-ae22-46aa-8424-91e2778390ab (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:28,313 [526] [DEBUG] [app] Starting request: urn:request:f93cca02-ae22-46aa-8424-91e2778390ab (/_internal_ping) 2020-03-31 09:31:28,314 [526] [DEBUG] [app] Ending request: urn:request:f93cca02-ae22-46aa-8424-91e2778390ab (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:28,314 [526] [DEBUG] [app] Ending request: urn:request:f93cca02-ae22-46aa-8424-91e2778390ab (/_internal_ping) 2020-03-31 09:31:28,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:28,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:28,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:28,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:28,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:28,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:28,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:31:28,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:31:28,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:28,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:31:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:28,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:28,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:28,331 [537] [DEBUG] [app] Starting request: urn:request:9f9e9827-9e30-4817-94db-99260004ad01 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:28,331 [537] [DEBUG] [app] Starting request: urn:request:9f9e9827-9e30-4817-94db-99260004ad01 (/c1/_internal_ping) 2020-03-31 09:31:28,332 [537] [DEBUG] [app] Ending request: urn:request:9f9e9827-9e30-4817-94db-99260004ad01 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:28,332 [537] [DEBUG] [app] Ending request: urn:request:9f9e9827-9e30-4817-94db-99260004ad01 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:28,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:28,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:28,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:31:28,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:28,334 [526] [DEBUG] [app] Ending request: urn:request:978be893-70f8-40c0-bafa-6e8a2213a842 (/health/instance) 2020-03-31 09:31:28,335 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:28,334 [526] [DEBUG] [app] Ending request: urn:request:978be893-70f8-40c0-bafa-6e8a2213a842 (/health/instance) 2020-03-31 09:31:28,335 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:28,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:31:28,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:31:30,172 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:66fe36e6-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:30 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:31:30,172 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:66fe36e6-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:30 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:31:30,172 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '66fe36e6-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qPL2iFS0wRn9K4QV2OizQOqI1Acw57rLaKytqD8HrSs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:30 GMT'}. storagereplication stdout | 2020-03-31 09:31:30,172 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '66fe36e6-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qPL2iFS0wRn9K4QV2OizQOqI1Acw57rLaKytqD8HrSs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:30 GMT'}. 2020-03-31 09:31:30,261 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:31:30,261 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:31:30,261 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:30 GMT, Server-Request-ID=7b7ea1d5-601e-0049-2a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '66fe36e6-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qPL2iFS0wRn9K4QV2OizQOqI1Acw57rLaKytqD8HrSs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:30 GMT'}. 2020-03-31 09:31:30,262 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:30 GMT, Server-Request-ID=7b7ea1d5-601e-0049-2a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:31:30,261 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:30 GMT, Server-Request-ID=7b7ea1d5-601e-0049-2a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '66fe36e6-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qPL2iFS0wRn9K4QV2OizQOqI1Acw57rLaKytqD8HrSs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:30 GMT'}. 2020-03-31 09:31:30,262 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:30 GMT, Server-Request-ID=7b7ea1d5-601e-0049-2a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:30,262 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:30 GMT, Server-Request-ID=7b7ea1d5-601e-0049-2a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:30,262 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:31:30,262 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=66fe36e6-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:30 GMT, Server-Request-ID=7b7ea1d5-601e-0049-2a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:30,262 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:31:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:31:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:00.337699+00:00 (in 29.998885 seconds) 2020-03-31 09:31:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:00 UTC)" (scheduled at 2020-03-31 09:31:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:31:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:00.337699+00:00 (in 29.998885 seconds) 2020-03-31 09:31:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:00 UTC)" (scheduled at 2020-03-31 09:31:30.337699+00:00) 2020-03-31 09:31:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 31, 30, 339410), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:31:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 31, 30, 339410), False, 50, 1, 0]) 2020-03-31 09:31:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:31:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:31:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:31:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:00 UTC)" executed successfully 2020-03-31 09:31:31,450 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:31,450 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:41.440707+00:00 (in 9.990141 seconds) 2020-03-31 09:31:31,452 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:41 UTC)" (scheduled at 2020-03-31 09:31:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:31:31,450 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:31:31,450 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:41.440707+00:00 (in 9.990141 seconds) repositoryactioncounter stdout | 2020-03-31 09:31:31,452 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:41 UTC)" (scheduled at 2020-03-31 09:31:31.440707+00:00) 2020-03-31 09:31:31,454 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:31:31,454 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:31:31,468 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:31:31,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:31,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:31:31,468 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:31:31,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:31,469 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:41 UTC)" executed successfully 2020-03-31 09:31:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:31:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:03.330112+00:00 (in 29.999016 seconds) 2020-03-31 09:31:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:03 UTC)" (scheduled at 2020-03-31 09:31:33.330112+00:00) 2020-03-31 09:31:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:31:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:03.330112+00:00 (in 29.999016 seconds) 2020-03-31 09:31:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:03 UTC)" (scheduled at 2020-03-31 09:31:33.330112+00:00) 2020-03-31 09:31:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 33, 332171), True, datetime.datetime(2020, 3, 31, 9, 31, 33, 332171), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:31:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 33, 332171), True, datetime.datetime(2020, 3, 31, 9, 31, 33, 332171), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:31:33,357 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:31:33,357 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:33,357 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:33,357 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:31:33,357 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:33,357 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:03 UTC)" executed successfully 2020-03-31 09:31:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:31:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:43.539999+00:00 (in 10.172278 seconds) 2020-03-31 09:31:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:33 UTC)" (scheduled at 2020-03-31 09:31:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:31:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:43.539999+00:00 (in 10.172278 seconds) 2020-03-31 09:31:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:33 UTC)" (scheduled at 2020-03-31 09:31:33.366791+00:00) 2020-03-31 09:31:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:31:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 33, 368665), True, datetime.datetime(2020, 3, 31, 9, 31, 33, 368665), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:31:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 33, 368665), True, datetime.datetime(2020, 3, 31, 9, 31, 33, 368665), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:31:33,387 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:31:33,387 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:31:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:33 UTC)" executed successfully 2020-03-31 09:31:34,264 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:696ebf9a-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:34 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:31:34,264 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:696ebf9a-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:34 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:31:34,264 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '696ebf9a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:omWLb51SKXS/qai+Hyn85tnafiEdfe0krlDdUw3q3z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:34 GMT'}. storagereplication stdout | 2020-03-31 09:31:34,264 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '696ebf9a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:omWLb51SKXS/qai+Hyn85tnafiEdfe0krlDdUw3q3z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:34 GMT'}. 2020-03-31 09:31:34,330 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:34,330 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:44.323352+00:00 (in 9.992872 seconds) 2020-03-31 09:31:34,331 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:44 UTC)" (scheduled at 2020-03-31 09:31:34.323352+00:00) 2020-03-31 09:31:34,331 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:34,333 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 34, 331301), True, datetime.datetime(2020, 3, 31, 9, 31, 34, 331301), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:31:34,330 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:34,330 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:44.323352+00:00 (in 9.992872 seconds) 2020-03-31 09:31:34,331 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:44 UTC)" (scheduled at 2020-03-31 09:31:34.323352+00:00) 2020-03-31 09:31:34,331 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:34,333 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 34, 331301), True, datetime.datetime(2020, 3, 31, 9, 31, 34, 331301), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:31:34,348 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:31:34,348 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:34,348 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:34,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:31:34,348 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:34,348 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:44 UTC)" executed successfully 2020-03-31 09:31:34,354 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:31:34,354 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:31:34,354 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:34 GMT, Server-Request-ID=7b7eac08-601e-0049-803f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '696ebf9a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:omWLb51SKXS/qai+Hyn85tnafiEdfe0krlDdUw3q3z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:34 GMT'}. 2020-03-31 09:31:34,355 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:34 GMT, Server-Request-ID=7b7eac08-601e-0049-803f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:34,355 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:34 GMT, Server-Request-ID=7b7eac08-601e-0049-803f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:34,355 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:31:34,354 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:34 GMT, Server-Request-ID=7b7eac08-601e-0049-803f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '696ebf9a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:omWLb51SKXS/qai+Hyn85tnafiEdfe0krlDdUw3q3z4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:34 GMT'}. 2020-03-31 09:31:34,355 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:34 GMT, Server-Request-ID=7b7eac08-601e-0049-803f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:34,355 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=696ebf9a-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:34 GMT, Server-Request-ID=7b7eac08-601e-0049-803f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:34,355 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:31:34,446 [518] [DEBUG] [app] Starting request: urn:request:8be0ba9e-f742-4f01-8196-330f98b00dab (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:34,446 [518] [DEBUG] [app] Starting request: urn:request:8be0ba9e-f742-4f01-8196-330f98b00dab (/health/instance) 2020-03-31 09:31:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:34,454 [555] [DEBUG] [app] Starting request: urn:request:064e9c71-9d62-4d74-9d55-41cfbdbf4cdf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:34,454 [555] [DEBUG] [app] Starting request: urn:request:064e9c71-9d62-4d74-9d55-41cfbdbf4cdf (/v1/_internal_ping) 2020-03-31 09:31:34,455 [555] [DEBUG] [app] Ending request: urn:request:064e9c71-9d62-4d74-9d55-41cfbdbf4cdf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:34,455 [555] [DEBUG] [app] Ending request: urn:request:064e9c71-9d62-4d74-9d55-41cfbdbf4cdf (/v1/_internal_ping) 2020-03-31 09:31:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:31:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:34,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:34,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:31:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:34,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:34,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:34,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:34,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:34,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:34,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:34,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:34,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:34,488 [518] [DEBUG] [app] Starting request: urn:request:bd513893-baf4-4ca8-85dd-08e020056c38 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:34,488 [518] [DEBUG] [app] Starting request: urn:request:bd513893-baf4-4ca8-85dd-08e020056c38 (/_internal_ping) 2020-03-31 09:31:34,489 [518] [DEBUG] [app] Ending request: urn:request:bd513893-baf4-4ca8-85dd-08e020056c38 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:34,489 [518] [DEBUG] [app] Ending request: urn:request:bd513893-baf4-4ca8-85dd-08e020056c38 (/_internal_ping) 2020-03-31 09:31:34,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:34,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:31:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:31:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:31:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:31:34,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:34,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:34,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:34,503 [537] [DEBUG] [app] Starting request: urn:request:99a5a517-a9b8-427d-ab53-10000876c9a2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:34,503 [537] [DEBUG] [app] Starting request: urn:request:99a5a517-a9b8-427d-ab53-10000876c9a2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:34,504 [537] [DEBUG] [app] Ending request: urn:request:99a5a517-a9b8-427d-ab53-10000876c9a2 (/c1/_internal_ping) 2020-03-31 09:31:34,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:34,507 [518] [DEBUG] [app] Ending request: urn:request:8be0ba9e-f742-4f01-8196-330f98b00dab (/health/instance) 2020-03-31 09:31:34,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-verbs stdout | 2020-03-31 09:31:34,504 [537] [DEBUG] [app] Ending request: urn:request:99a5a517-a9b8-427d-ab53-10000876c9a2 (/c1/_internal_ping) 2020-03-31 09:31:34,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:34,507 [518] [DEBUG] [app] Ending request: urn:request:8be0ba9e-f742-4f01-8196-330f98b00dab (/health/instance) 2020-03-31 09:31:34,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:31:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:31:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:38.010934+00:00 (in 1.085662 seconds) storagereplication stdout | 2020-03-31 09:31:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:38.010934+00:00 (in 1.085662 seconds) 2020-03-31 09:31:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:31:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:46.924497+00:00 (in 8.912738 seconds) 2020-03-31 09:31:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:38 UTC)" (scheduled at 2020-03-31 09:31:38.010934+00:00) 2020-03-31 09:31:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:31:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:46.924497+00:00 (in 8.912738 seconds) 2020-03-31 09:31:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:38 UTC)" (scheduled at 2020-03-31 09:31:38.010934+00:00) 2020-03-31 09:31:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:38 UTC)" executed successfully 2020-03-31 09:31:38,272 [526] [DEBUG] [app] Starting request: urn:request:11bcaa87-2681-4f49-b399-7f053bed45da (/health/instance) 2020-03-31 09:31:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:38,272 [526] [DEBUG] [app] Starting request: urn:request:11bcaa87-2681-4f49-b399-7f053bed45da (/health/instance) 2020-03-31 09:31:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:38,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:38,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:38,282 [556] [DEBUG] [app] Starting request: urn:request:6793b8df-5b46-47ec-b58e-b841eba0202c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:38,282 [556] [DEBUG] [app] Starting request: urn:request:6793b8df-5b46-47ec-b58e-b841eba0202c (/v1/_internal_ping) 2020-03-31 09:31:38,284 [556] [DEBUG] [app] Ending request: urn:request:6793b8df-5b46-47ec-b58e-b841eba0202c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:38,284 [556] [DEBUG] [app] Ending request: urn:request:6793b8df-5b46-47ec-b58e-b841eba0202c (/v1/_internal_ping) 2020-03-31 09:31:38,284 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:38,285 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:31:38,284 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:38,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:38,285 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:31:38,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:38,288 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:38,288 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:38,290 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:38,290 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:38,291 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:38,291 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:38,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:38,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:38,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:38,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:38,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:38,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:38,315 [518] [DEBUG] [app] Starting request: urn:request:540b5a9a-ba10-4bbd-9d9b-8dd54675417d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:38,315 [518] [DEBUG] [app] Starting request: urn:request:540b5a9a-ba10-4bbd-9d9b-8dd54675417d (/_internal_ping) 2020-03-31 09:31:38,316 [518] [DEBUG] [app] Ending request: urn:request:540b5a9a-ba10-4bbd-9d9b-8dd54675417d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:38,316 [518] [DEBUG] [app] Ending request: urn:request:540b5a9a-ba10-4bbd-9d9b-8dd54675417d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:31:38,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:31:38,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:38,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:38,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:31:38,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:38,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:38,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:31:38,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:38,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:38,329 [543] [DEBUG] [app] Starting request: urn:request:2b8ce41f-769c-4b99-aa22-255a166031de (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:38,329 [543] [DEBUG] [app] Starting request: urn:request:2b8ce41f-769c-4b99-aa22-255a166031de (/c1/_internal_ping) 2020-03-31 09:31:38,330 [543] [DEBUG] [app] Ending request: urn:request:2b8ce41f-769c-4b99-aa22-255a166031de (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:38,330 [543] [DEBUG] [app] Ending request: urn:request:2b8ce41f-769c-4b99-aa22-255a166031de (/c1/_internal_ping) 2020-03-31 09:31:38,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:38,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:38,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:31:38,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:38,332 [526] [DEBUG] [app] Ending request: urn:request:11bcaa87-2681-4f49-b399-7f053bed45da (/health/instance) 2020-03-31 09:31:38,333 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:38,332 [526] [DEBUG] [app] Ending request: urn:request:11bcaa87-2681-4f49-b399-7f053bed45da (/health/instance) 2020-03-31 09:31:38,333 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:38,333 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:31:38,333 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:31:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:31:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:51.440707+00:00 (in 9.999267 seconds) 2020-03-31 09:31:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:51 UTC)" (scheduled at 2020-03-31 09:31:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:31:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:51.440707+00:00 (in 9.999267 seconds) 2020-03-31 09:31:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:51 UTC)" (scheduled at 2020-03-31 09:31:41.440707+00:00) 2020-03-31 09:31:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:31:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:31:41,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:31:41,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:31:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:31:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:51 UTC)" executed successfully 2020-03-31 09:31:42,363 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:6e4287fe-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:42 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:31:42,363 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:6e4287fe-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:42 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:31:42,363 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '6e4287fe-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:vq4b4EKfBIoBmQsArhnfNmRWkzosAtXBlUzJYPoZ/wQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:42 GMT'}. storagereplication stdout | 2020-03-31 09:31:42,363 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '6e4287fe-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:vq4b4EKfBIoBmQsArhnfNmRWkzosAtXBlUzJYPoZ/wQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:42 GMT'}. 2020-03-31 09:31:42,452 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:31:42,452 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:31:42,453 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:42 GMT, Server-Request-ID=7b7ec291-601e-0049-463f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '6e4287fe-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:vq4b4EKfBIoBmQsArhnfNmRWkzosAtXBlUzJYPoZ/wQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:42 GMT'}. 2020-03-31 09:31:42,454 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:42 GMT, Server-Request-ID=7b7ec291-601e-0049-463f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:42,454 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:42 GMT, Server-Request-ID=7b7ec291-601e-0049-463f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:31:42,453 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:42 GMT, Server-Request-ID=7b7ec291-601e-0049-463f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '6e4287fe-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:vq4b4EKfBIoBmQsArhnfNmRWkzosAtXBlUzJYPoZ/wQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:42 GMT'}. 2020-03-31 09:31:42,454 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:42 GMT, Server-Request-ID=7b7ec291-601e-0049-463f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:42,454 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=6e4287fe-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:42 GMT, Server-Request-ID=7b7ec291-601e-0049-463f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:42,454 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:31:42,454 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:31:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:31:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:33.366791+00:00 (in 49.826067 seconds) 2020-03-31 09:31:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:43 UTC)" (scheduled at 2020-03-31 09:31:43.539999+00:00) 2020-03-31 09:31:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:31:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:33.366791+00:00 (in 49.826067 seconds) 2020-03-31 09:31:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:43 UTC)" (scheduled at 2020-03-31 09:31:43.539999+00:00) 2020-03-31 09:31:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:31:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:43 UTC)" executed successfully 2020-03-31 09:31:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:31:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:49.498966+00:00 (in 5.174862 seconds) 2020-03-31 09:31:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:54 UTC)" (scheduled at 2020-03-31 09:31:44.323352+00:00) 2020-03-31 09:31:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:31:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:49.498966+00:00 (in 5.174862 seconds) 2020-03-31 09:31:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:54 UTC)" (scheduled at 2020-03-31 09:31:44.323352+00:00) 2020-03-31 09:31:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 44, 325088), True, datetime.datetime(2020, 3, 31, 9, 31, 44, 325088), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:31:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 44, 325088), True, datetime.datetime(2020, 3, 31, 9, 31, 44, 325088), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:31:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:31:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:31:44,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:54 UTC)" executed successfully 2020-03-31 09:31:44,447 [527] [DEBUG] [app] Starting request: urn:request:49e9c2e5-1008-4837-a036-121427703f82 (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:44,447 [527] [DEBUG] [app] Starting request: urn:request:49e9c2e5-1008-4837-a036-121427703f82 (/health/instance) 2020-03-31 09:31:44,450 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:44,450 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:44,455 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:44,455 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:44,458 [556] [DEBUG] [app] Starting request: urn:request:e63d96a2-e6ec-4df8-a8c8-07c5db4380e4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:44,458 [556] [DEBUG] [app] Starting request: urn:request:e63d96a2-e6ec-4df8-a8c8-07c5db4380e4 (/v1/_internal_ping) 2020-03-31 09:31:44,460 [556] [DEBUG] [app] Ending request: urn:request:e63d96a2-e6ec-4df8-a8c8-07c5db4380e4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:44,460 [556] [DEBUG] [app] Ending request: urn:request:e63d96a2-e6ec-4df8-a8c8-07c5db4380e4 (/v1/_internal_ping) 2020-03-31 09:31:44,460 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:31:44,460 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:44,460 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:44,461 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:31:44,460 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:44,461 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:44,463 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:44,463 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:44,464 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:44,464 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:44,465 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:44,465 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:44,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:44,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:44,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:44,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:44,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:44,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:44,490 [527] [DEBUG] [app] Starting request: urn:request:1bcc5830-c8bf-4c0c-8a35-9ab492da4516 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:44,490 [527] [DEBUG] [app] Starting request: urn:request:1bcc5830-c8bf-4c0c-8a35-9ab492da4516 (/_internal_ping) 2020-03-31 09:31:44,491 [527] [DEBUG] [app] Ending request: urn:request:1bcc5830-c8bf-4c0c-8a35-9ab492da4516 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:44,491 [527] [DEBUG] [app] Ending request: urn:request:1bcc5830-c8bf-4c0c-8a35-9ab492da4516 (/_internal_ping) 2020-03-31 09:31:44,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:31:44,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:31:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:31:44,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:44,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:44,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:31:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:44,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:44,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:44,507 [536] [DEBUG] [app] Starting request: urn:request:4e6ee544-cdb9-44f7-afbc-1b72a2648675 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:44,507 [536] [DEBUG] [app] Starting request: urn:request:4e6ee544-cdb9-44f7-afbc-1b72a2648675 (/c1/_internal_ping) 2020-03-31 09:31:44,508 [536] [DEBUG] [app] Ending request: urn:request:4e6ee544-cdb9-44f7-afbc-1b72a2648675 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:44,508 [536] [DEBUG] [app] Ending request: urn:request:4e6ee544-cdb9-44f7-afbc-1b72a2648675 (/c1/_internal_ping) 2020-03-31 09:31:44,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:44,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:44,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:44,510 [527] [DEBUG] [app] Ending request: urn:request:49e9c2e5-1008-4837-a036-121427703f82 (/health/instance) 2020-03-31 09:31:44,510 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:31:44,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:44,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:31:44,510 [527] [DEBUG] [app] Ending request: urn:request:49e9c2e5-1008-4837-a036-121427703f82 (/health/instance) 2020-03-31 09:31:44,510 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:44,511 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:31:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:31:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:15.064514+00:00 (in 29.996456 seconds) 2020-03-31 09:31:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:15 UTC)" (scheduled at 2020-03-31 09:31:45.064514+00:00) gcworker stdout | 2020-03-31 09:31:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:15.064514+00:00 (in 29.996456 seconds) 2020-03-31 09:31:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:15 UTC)" (scheduled at 2020-03-31 09:31:45.064514+00:00) 2020-03-31 09:31:45,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437505, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:31:45,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437505, 1209600, 500, 1, 0]) 2020-03-31 09:31:45,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:31:45,087 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:31:45,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:31:45,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:31:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:15 UTC)" executed successfully 2020-03-31 09:31:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:31:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:56.924497+00:00 (in 9.999030 seconds) storagereplication stdout | 2020-03-31 09:31:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:56.924497+00:00 (in 9.999030 seconds) 2020-03-31 09:31:48,273 [527] [DEBUG] [app] Starting request: urn:request:4782e926-112d-4f60-b810-07014404293a (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:48,273 [527] [DEBUG] [app] Starting request: urn:request:4782e926-112d-4f60-b810-07014404293a (/health/instance) 2020-03-31 09:31:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:48,278 [538] [DEBUG] [app] Starting request: urn:request:0d96361c-c691-416e-96e0-7ccb86df0bbc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:48,278 [538] [DEBUG] [app] Starting request: urn:request:0d96361c-c691-416e-96e0-7ccb86df0bbc (/v1/_internal_ping) 2020-03-31 09:31:48,279 [538] [DEBUG] [app] Ending request: urn:request:0d96361c-c691-416e-96e0-7ccb86df0bbc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:48,279 [538] [DEBUG] [app] Ending request: urn:request:0d96361c-c691-416e-96e0-7ccb86df0bbc (/v1/_internal_ping) 2020-03-31 09:31:48,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:48,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:31:48,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:48,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:48,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:31:48,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:48,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:48,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:48,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:48,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:48,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:48,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:48,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:48,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:48,311 [525] [DEBUG] [app] Starting request: urn:request:34a8b9c9-d5e5-4e89-9d66-940c7b6982ef (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:48,311 [525] [DEBUG] [app] Starting request: urn:request:34a8b9c9-d5e5-4e89-9d66-940c7b6982ef (/_internal_ping) 2020-03-31 09:31:48,311 [525] [DEBUG] [app] Ending request: urn:request:34a8b9c9-d5e5-4e89-9d66-940c7b6982ef (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:48,311 [525] [DEBUG] [app] Ending request: urn:request:34a8b9c9-d5e5-4e89-9d66-940c7b6982ef (/_internal_ping) 2020-03-31 09:31:48,312 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:48,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:31:48,312 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:48,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:31:48,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:48,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:31:48,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:31:48,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:48,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:48,327 [536] [DEBUG] [app] Starting request: urn:request:a017608b-1a77-4beb-bc5b-be8b02ce8dc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:48,327 [536] [DEBUG] [app] Starting request: urn:request:a017608b-1a77-4beb-bc5b-be8b02ce8dc2 (/c1/_internal_ping) 2020-03-31 09:31:48,327 [536] [DEBUG] [app] Ending request: urn:request:a017608b-1a77-4beb-bc5b-be8b02ce8dc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:48,327 [536] [DEBUG] [app] Ending request: urn:request:a017608b-1a77-4beb-bc5b-be8b02ce8dc2 (/c1/_internal_ping) 2020-03-31 09:31:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:31:48,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:31:48,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:48,330 [527] [DEBUG] [app] Ending request: urn:request:4782e926-112d-4f60-b810-07014404293a (/health/instance) 2020-03-31 09:31:48,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:48,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:31:48,330 [527] [DEBUG] [app] Ending request: urn:request:4782e926-112d-4f60-b810-07014404293a (/health/instance) 2020-03-31 09:31:48,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:48,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:31:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:31:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:54.323352+00:00 (in 4.823202 seconds) 2020-03-31 09:31:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:49 UTC)" (scheduled at 2020-03-31 09:31:49.498966+00:00) 2020-03-31 09:31:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:31:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:54.323352+00:00 (in 4.823202 seconds) 2020-03-31 09:31:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:49 UTC)" (scheduled at 2020-03-31 09:31:49.498966+00:00) 2020-03-31 09:31:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:49 UTC)" executed successfully 2020-03-31 09:31:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:31:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:59.560326+00:00 (in 8.447864 seconds) 2020-03-31 09:31:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:51 UTC)" (scheduled at 2020-03-31 09:31:51.110610+00:00) 2020-03-31 09:31:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:31:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:31:59.560326+00:00 (in 8.447864 seconds) 2020-03-31 09:31:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:51 UTC)" (scheduled at 2020-03-31 09:31:51.110610+00:00) 2020-03-31 09:31:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 51, 113558), True, datetime.datetime(2020, 3, 31, 9, 31, 51, 113558), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:31:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 51, 113558), True, datetime.datetime(2020, 3, 31, 9, 31, 51, 113558), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:31:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:51,130 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:31:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:31:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:51 UTC)" executed successfully 2020-03-31 09:31:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:31:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:01.440707+00:00 (in 9.999220 seconds) 2020-03-31 09:31:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:01 UTC)" (scheduled at 2020-03-31 09:31:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:31:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:01.440707+00:00 (in 9.999220 seconds) 2020-03-31 09:31:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:01 UTC)" (scheduled at 2020-03-31 09:31:51.440707+00:00) 2020-03-31 09:31:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:31:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:31:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:31:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:31:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:31:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:01 UTC)" executed successfully 2020-03-31 09:31:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:31:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:22.113221+00:00 (in 29.998899 seconds) 2020-03-31 09:31:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:22 UTC)" (scheduled at 2020-03-31 09:31:52.113221+00:00) securityworker stdout | 2020-03-31 09:31:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:22.113221+00:00 (in 29.998899 seconds) 2020-03-31 09:31:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:22 UTC)" (scheduled at 2020-03-31 09:31:52.113221+00:00) 2020-03-31 09:31:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:31:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:31:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:31:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:31:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:31:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:31:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:31:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:52,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:22 UTC)" executed successfully 2020-03-31 09:31:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:31:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:04.323352+00:00 (in 9.998195 seconds) 2020-03-31 09:31:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:04 UTC)" (scheduled at 2020-03-31 09:31:54.323352+00:00) 2020-03-31 09:31:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:31:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:04.323352+00:00 (in 9.998195 seconds) 2020-03-31 09:31:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:04 UTC)" (scheduled at 2020-03-31 09:31:54.323352+00:00) 2020-03-31 09:31:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:31:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 54, 326107), True, datetime.datetime(2020, 3, 31, 9, 31, 54, 326107), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:31:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 54, 326107), True, datetime.datetime(2020, 3, 31, 9, 31, 54, 326107), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:31:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:54,344 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:31:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:31:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:31:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:04 UTC)" executed successfully 2020-03-31 09:31:54,446 [518] [DEBUG] [app] Starting request: urn:request:35318148-ad1e-4c71-96ca-6d4f88189ac3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:54,446 [518] [DEBUG] [app] Starting request: urn:request:35318148-ad1e-4c71-96ca-6d4f88189ac3 (/health/instance) 2020-03-31 09:31:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:54,454 [556] [DEBUG] [app] Starting request: urn:request:99317dce-d1be-4f4f-adce-45a67675a26c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:54,454 [556] [DEBUG] [app] Starting request: urn:request:99317dce-d1be-4f4f-adce-45a67675a26c (/v1/_internal_ping) 2020-03-31 09:31:54,455 [556] [DEBUG] [app] Ending request: urn:request:99317dce-d1be-4f4f-adce-45a67675a26c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:54,455 [556] [DEBUG] [app] Ending request: urn:request:99317dce-d1be-4f4f-adce-45a67675a26c (/v1/_internal_ping) 2020-03-31 09:31:54,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:31:54,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:31:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:54,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:54,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:54,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:54,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:54,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:54,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:54,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:54,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:54,487 [518] [DEBUG] [app] Starting request: urn:request:ea4328fe-55db-4bd0-8924-c25e4e65cc67 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:54,487 [518] [DEBUG] [app] Starting request: urn:request:ea4328fe-55db-4bd0-8924-c25e4e65cc67 (/_internal_ping) 2020-03-31 09:31:54,488 [518] [DEBUG] [app] Ending request: urn:request:ea4328fe-55db-4bd0-8924-c25e4e65cc67 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:54,488 [518] [DEBUG] [app] Ending request: urn:request:ea4328fe-55db-4bd0-8924-c25e4e65cc67 (/_internal_ping) 2020-03-31 09:31:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:31:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:31:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:31:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:31:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:54,500 [536] [DEBUG] [app] Starting request: urn:request:f37febe3-ff1a-486b-b39f-b0c92ed9d379 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:54,500 [536] [DEBUG] [app] Starting request: urn:request:f37febe3-ff1a-486b-b39f-b0c92ed9d379 (/c1/_internal_ping) 2020-03-31 09:31:54,501 [536] [DEBUG] [app] Ending request: urn:request:f37febe3-ff1a-486b-b39f-b0c92ed9d379 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:54,501 [536] [DEBUG] [app] Ending request: urn:request:f37febe3-ff1a-486b-b39f-b0c92ed9d379 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:54,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:31:54,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:54,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:54,503 [518] [DEBUG] [app] Ending request: urn:request:35318148-ad1e-4c71-96ca-6d4f88189ac3 (/health/instance) 2020-03-31 09:31:54,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:54,503 [518] [DEBUG] [app] Ending request: urn:request:35318148-ad1e-4c71-96ca-6d4f88189ac3 (/health/instance) 2020-03-31 09:31:54,503 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:31:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:31:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:31:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:31:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:03.010318+00:00 (in 6.084980 seconds) storagereplication stdout | 2020-03-31 09:31:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:31:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:31:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:03.010318+00:00 (in 6.084980 seconds) 2020-03-31 09:31:58,272 [527] [DEBUG] [app] Starting request: urn:request:752a3646-4be2-4af9-a4cf-b6ed84f512ce (/health/instance) gunicorn-web stdout | 2020-03-31 09:31:58,272 [527] [DEBUG] [app] Starting request: urn:request:752a3646-4be2-4af9-a4cf-b6ed84f512ce (/health/instance) 2020-03-31 09:31:58,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:58,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:58,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:58,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:58,280 [542] [DEBUG] [app] Starting request: urn:request:058e1b03-6624-48e7-bcbf-ff4b9e74d9e7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:58,280 [542] [DEBUG] [app] Starting request: urn:request:058e1b03-6624-48e7-bcbf-ff4b9e74d9e7 (/v1/_internal_ping) 2020-03-31 09:31:58,281 [542] [DEBUG] [app] Ending request: urn:request:058e1b03-6624-48e7-bcbf-ff4b9e74d9e7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:31:58,281 [542] [DEBUG] [app] Ending request: urn:request:058e1b03-6624-48e7-bcbf-ff4b9e74d9e7 (/v1/_internal_ping) 2020-03-31 09:31:58,281 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:31:58,281 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:31:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:31:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:31:58,285 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:31:58,285 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:31:58,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:31:58,286 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:31:58,287 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:31:58,287 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:31:58,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:31:58,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:31:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:31:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:31:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:58,313 [527] [DEBUG] [app] Starting request: urn:request:3e8d49a7-62ef-43f1-848d-89d667e0596a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:58,313 [527] [DEBUG] [app] Starting request: urn:request:3e8d49a7-62ef-43f1-848d-89d667e0596a (/_internal_ping) 2020-03-31 09:31:58,314 [527] [DEBUG] [app] Ending request: urn:request:3e8d49a7-62ef-43f1-848d-89d667e0596a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:31:58,314 [527] [DEBUG] [app] Ending request: urn:request:3e8d49a7-62ef-43f1-848d-89d667e0596a (/_internal_ping) 2020-03-31 09:31:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:31:58,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:31:58,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:31:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:31:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:31:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:31:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:31:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:31:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:31:58,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:31:58,327 [536] [DEBUG] [app] Starting request: urn:request:262c7eac-693c-4db1-a15b-8da052c27d40 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:58,327 [536] [DEBUG] [app] Starting request: urn:request:262c7eac-693c-4db1-a15b-8da052c27d40 (/c1/_internal_ping) 2020-03-31 09:31:58,328 [536] [DEBUG] [app] Ending request: urn:request:262c7eac-693c-4db1-a15b-8da052c27d40 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:31:58,328 [536] [DEBUG] [app] Ending request: urn:request:262c7eac-693c-4db1-a15b-8da052c27d40 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:58,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:31:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:31:58,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:31:58,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:31:58,330 [527] [DEBUG] [app] Ending request: urn:request:752a3646-4be2-4af9-a4cf-b6ed84f512ce (/health/instance) 2020-03-31 09:31:58,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:31:58,330 [527] [DEBUG] [app] Ending request: urn:request:752a3646-4be2-4af9-a4cf-b6ed84f512ce (/health/instance) 2020-03-31 09:31:58,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:31:58,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:31:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:31:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:31:58,467 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:31:58,467 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy 2020-03-31 09:31:58,468 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:31:58,468 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:31:58,468 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:31:58,468 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:31:58,477 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) storagereplication stdout | 2020-03-31 09:31:58,477 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) 2020-03-31 09:31:58,481 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 36, 58, 477231), True, u'43b1dced-dda3-48fd-81f0-713652abf8e2', 4]) storagereplication stdout | 2020-03-31 09:31:58,481 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 36, 58, 477231), True, u'43b1dced-dda3-48fd-81f0-713652abf8e2', 4]) 2020-03-31 09:31:58,488 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 58, 487005), True, datetime.datetime(2020, 3, 31, 9, 31, 58, 487005), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:31:58,488 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 31, 58, 487005), True, datetime.datetime(2020, 3, 31, 9, 31, 58, 487005), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:31:58,493 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 51, 58, 487005), 1, u'fb2e0b1e-11d0-48be-b29e-4650706df8ef', 2, u'5440e0e5-24a9-4eb6-b61a-88257c21c199']) storagereplication stdout | 2020-03-31 09:31:58,493 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 51, 58, 487005), 1, u'fb2e0b1e-11d0-48be-b29e-4650706df8ef', 2, u'5440e0e5-24a9-4eb6-b61a-88257c21c199']) 2020-03-31 09:31:58,499 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} storagereplication stdout | 2020-03-31 09:31:58,499 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:31:58,499 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:58,499 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 storagereplication stdout | 2020-03-31 09:31:58,499 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:58,499 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 2020-03-31 09:31:58,501 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:31:58,501 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:31:58,514 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:31:58,514 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:31:58,518 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) storagereplication stdout | 2020-03-31 09:31:58,518 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) 2020-03-31 09:31:58,522 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:31:58,522 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:31:58,522 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:77e43c1c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:31:58,522 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e43c1c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77e43c1c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:JnYbbvBLwYJDBcV3AGWntpSVJSOtSDFC+v5Ldz4QfWU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. storagereplication stdout | 2020-03-31 09:31:58,522 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:77e43c1c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:31:58,522 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e43c1c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77e43c1c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:JnYbbvBLwYJDBcV3AGWntpSVJSOtSDFC+v5Ldz4QfWU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. 2020-03-31 09:31:58,531 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:31:58,531 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:31:58,532 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e43c1c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7773056a-401e-009e-443f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77e43c1c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:JnYbbvBLwYJDBcV3AGWntpSVJSOtSDFC+v5Ldz4QfWU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. storagereplication stdout | 2020-03-31 09:31:58,532 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e43c1c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7773056a-401e-009e-443f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77e43c1c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:JnYbbvBLwYJDBcV3AGWntpSVJSOtSDFC+v5Ldz4QfWU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. 2020-03-31 09:31:58,533 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us storagereplication stdout | 2020-03-31 09:31:58,533 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us 2020-03-31 09:31:58,533 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:58,533 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:31:58,533 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:31:58,533 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:31:58,534 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:77e60042-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=98KqpkEMz0rVzWTTNRfkOGOmc8AW46hZXpVt6xPnhc0%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A58Z x-ms-date:Tue, 31 Mar 2020 09:31:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:31:58,534 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e60042-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=98KqpkEMz0rVzWTTNRfkOGOmc8AW46hZXpVt6xPnhc0%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0AUpoI+SqdTrV5v7BSS6faoGVlk1GQ3B6kpgl01bMck=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '77e60042-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. storagereplication stdout | 2020-03-31 09:31:58,534 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:77e60042-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=98KqpkEMz0rVzWTTNRfkOGOmc8AW46hZXpVt6xPnhc0%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A58Z x-ms-date:Tue, 31 Mar 2020 09:31:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:31:58,534 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e60042-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=98KqpkEMz0rVzWTTNRfkOGOmc8AW46hZXpVt6xPnhc0%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0AUpoI+SqdTrV5v7BSS6faoGVlk1GQ3B6kpgl01bMck=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '77e60042-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. 2020-03-31 09:31:58,566 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:31:58,566 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 2020-03-31 09:31:58,567 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e60042-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7773056d-401e-009e-453f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=98KqpkEMz0rVzWTTNRfkOGOmc8AW46hZXpVt6xPnhc0%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0AUpoI+SqdTrV5v7BSS6faoGVlk1GQ3B6kpgl01bMck=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '77e60042-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. storagereplication stdout | 2020-03-31 09:31:58,567 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77e60042-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7773056d-401e-009e-453f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=98KqpkEMz0rVzWTTNRfkOGOmc8AW46hZXpVt6xPnhc0%3D&sv=2017-07-29&se=2020-03-31T09%3A32%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:0AUpoI+SqdTrV5v7BSS6faoGVlk1GQ3B6kpgl01bMck=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '77e60042-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. 2020-03-31 09:31:58,568 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:31:58,568 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:77eb38fa-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:58 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:31:58,568 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77eb38fa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:M+RFtqYk0iSYMh860jROQqseAwToH/GpA7/xBuev1Jg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. storagereplication stdout | 2020-03-31 09:31:58,568 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:31:58,568 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:77eb38fa-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:31:58 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:31:58,568 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77eb38fa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:M+RFtqYk0iSYMh860jROQqseAwToH/GpA7/xBuev1Jg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. 2020-03-31 09:31:58,661 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:31:58,661 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:31:58,662 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7b7eef54-601e-0049-0b3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77eb38fa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:M+RFtqYk0iSYMh860jROQqseAwToH/GpA7/xBuev1Jg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. 2020-03-31 09:31:58,662 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7b7eef54-601e-0049-0b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:58,662 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7b7eef54-601e-0049-0b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:31:58,662 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7b7eef54-601e-0049-0b3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '77eb38fa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:M+RFtqYk0iSYMh860jROQqseAwToH/GpA7/xBuev1Jg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:31:58 GMT'}. 2020-03-31 09:31:58,662 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7b7eef54-601e-0049-0b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:58,662 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=77eb38fa-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:31:58 GMT, Server-Request-ID=7b7eef54-601e-0049-0b3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:31:58,662 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:31:58,662 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:31:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:31:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:31:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:51.110610+00:00 (in 51.549118 seconds) 2020-03-31 09:31:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:59 UTC)" (scheduled at 2020-03-31 09:31:59.560326+00:00) 2020-03-31 09:31:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:31:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:51.110610+00:00 (in 51.549118 seconds) 2020-03-31 09:31:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:59 UTC)" (scheduled at 2020-03-31 09:31:59.560326+00:00) 2020-03-31 09:31:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:31:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:32:59 UTC)" executed successfully 2020-03-31 09:32:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:32:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:30.337699+00:00 (in 29.999268 seconds) 2020-03-31 09:32:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:30 UTC)" (scheduled at 2020-03-31 09:32:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:32:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:30.337699+00:00 (in 29.999268 seconds) 2020-03-31 09:32:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:30 UTC)" (scheduled at 2020-03-31 09:32:00.337699+00:00) 2020-03-31 09:32:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 32, 0, 339224), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:32:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 32, 0, 339224), False, 50, 1, 0]) 2020-03-31 09:32:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:32:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:32:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:32:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:30 UTC)" executed successfully 2020-03-31 09:32:00,663 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:792ae3be-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:00 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:32:00,663 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:792ae3be-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:00 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:32:00,663 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '792ae3be-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:PpwFARtj4CPIMC0HodAB15SJQsbV1GS6JAT9bKhsJ7A=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:00 GMT'}. storagereplication stdout | 2020-03-31 09:32:00,663 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '792ae3be-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:PpwFARtj4CPIMC0HodAB15SJQsbV1GS6JAT9bKhsJ7A=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:00 GMT'}. 2020-03-31 09:32:00,753 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:32:00,753 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:32:00,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:00 GMT, Server-Request-ID=7b7ef57e-601e-0049-673f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '792ae3be-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:PpwFARtj4CPIMC0HodAB15SJQsbV1GS6JAT9bKhsJ7A=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:00 GMT'}. 2020-03-31 09:32:00,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:00 GMT, Server-Request-ID=7b7ef57e-601e-0049-673f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:32:00,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:00 GMT, Server-Request-ID=7b7ef57e-601e-0049-673f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '792ae3be-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:PpwFARtj4CPIMC0HodAB15SJQsbV1GS6JAT9bKhsJ7A=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:00 GMT'}. 2020-03-31 09:32:00,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:00 GMT, Server-Request-ID=7b7ef57e-601e-0049-673f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:00,754 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:00 GMT, Server-Request-ID=7b7ef57e-601e-0049-673f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:00,754 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:32:00,754 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=792ae3be-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:00 GMT, Server-Request-ID=7b7ef57e-601e-0049-673f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:00,754 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:32:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:32:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:11.440707+00:00 (in 9.999105 seconds) 2020-03-31 09:32:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:11 UTC)" (scheduled at 2020-03-31 09:32:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:32:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:11.440707+00:00 (in 9.999105 seconds) 2020-03-31 09:32:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:11 UTC)" (scheduled at 2020-03-31 09:32:01.440707+00:00) 2020-03-31 09:32:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:32:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:32:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:32:01,458 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:32:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:32:01,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:32:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:11 UTC)" executed successfully 2020-03-31 09:32:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:37:03 UTC)" (scheduled at 2020-03-31 09:32:03.010318+00:00) storagereplication stdout | 2020-03-31 09:32:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:37:03 UTC)" (scheduled at 2020-03-31 09:32:03.010318+00:00) 2020-03-31 09:32:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:06.924497+00:00 (in 3.913450 seconds) storagereplication stdout | 2020-03-31 09:32:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:06.924497+00:00 (in 3.913450 seconds) 2020-03-31 09:32:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:32:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), u'imagestoragereplication/%']) 2020-03-31 09:32:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), True, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:32:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), True, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), 0, u'imagestoragereplication/%']) 2020-03-31 09:32:03,033 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), True, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:32:03,033 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), True, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 32, 3, 11862), u'imagestoragereplication/%']) 2020-03-31 09:32:03,036 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:32:03,036 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:03,037 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:37:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:32:03,037 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:37:03 UTC)" executed successfully 2020-03-31 09:32:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:32:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:32:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:03,041 [191] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics storagereplication stdout | 2020-03-31 09:32:03,041 [191] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:32:03,043 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:32:03,043 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:03,044 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:03,045 [191] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics storagereplication stdout | 2020-03-31 09:32:03,044 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:32:03,045 [191] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:32:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:24.410927+00:00 (in 21.079596 seconds) 2020-03-31 09:32:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:33 UTC)" (scheduled at 2020-03-31 09:32:03.330112+00:00) 2020-03-31 09:32:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:32:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:24.410927+00:00 (in 21.079596 seconds) 2020-03-31 09:32:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:33 UTC)" (scheduled at 2020-03-31 09:32:03.330112+00:00) 2020-03-31 09:32:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 3, 332336), True, datetime.datetime(2020, 3, 31, 9, 32, 3, 332336), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:32:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 3, 332336), True, datetime.datetime(2020, 3, 31, 9, 32, 3, 332336), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:32:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:32:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:32:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:33 UTC)" executed successfully 2020-03-31 09:32:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:14.323352+00:00 (in 9.998840 seconds) 2020-03-31 09:32:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:14 UTC)" (scheduled at 2020-03-31 09:32:04.323352+00:00) 2020-03-31 09:32:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:32:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:14.323352+00:00 (in 9.998840 seconds) 2020-03-31 09:32:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:14 UTC)" (scheduled at 2020-03-31 09:32:04.323352+00:00) 2020-03-31 09:32:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 4, 325551), True, datetime.datetime(2020, 3, 31, 9, 32, 4, 325551), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:32:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 4, 325551), True, datetime.datetime(2020, 3, 31, 9, 32, 4, 325551), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:32:04,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:32:04,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:14 UTC)" executed successfully 2020-03-31 09:32:04,445 [518] [DEBUG] [app] Starting request: urn:request:968172b1-c79e-4fa3-97b3-fa1632a1e051 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:04,445 [518] [DEBUG] [app] Starting request: urn:request:968172b1-c79e-4fa3-97b3-fa1632a1e051 (/health/instance) 2020-03-31 09:32:04,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:04,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:04,453 [538] [DEBUG] [app] Starting request: urn:request:4c216c87-db51-4752-a57e-322b5313472f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:04,453 [538] [DEBUG] [app] Starting request: urn:request:4c216c87-db51-4752-a57e-322b5313472f (/v1/_internal_ping) 2020-03-31 09:32:04,454 [538] [DEBUG] [app] Ending request: urn:request:4c216c87-db51-4752-a57e-322b5313472f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:04,454 [538] [DEBUG] [app] Ending request: urn:request:4c216c87-db51-4752-a57e-322b5313472f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:04,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:04,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:32:04,454 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:04,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:04,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:32:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:32:04,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:04,458 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:04,458 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:04,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:04,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:04,464 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:04,464 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:04,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:04,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:04,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:04,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:04,483 [527] [DEBUG] [app] Starting request: urn:request:294c556c-4b4a-4210-baeb-d55f8e08e4ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:04,483 [527] [DEBUG] [app] Starting request: urn:request:294c556c-4b4a-4210-baeb-d55f8e08e4ce (/_internal_ping) 2020-03-31 09:32:04,484 [527] [DEBUG] [app] Ending request: urn:request:294c556c-4b4a-4210-baeb-d55f8e08e4ce (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:04,484 [527] [DEBUG] [app] Ending request: urn:request:294c556c-4b4a-4210-baeb-d55f8e08e4ce (/_internal_ping) 2020-03-31 09:32:04,484 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:04,484 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:04,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:04,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:04,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:04,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:32:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:32:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:04,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:32:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:04,497 [534] [DEBUG] [app] Starting request: urn:request:74295cb9-b855-44ac-8f79-406ebceecf7e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:04,497 [534] [DEBUG] [app] Starting request: urn:request:74295cb9-b855-44ac-8f79-406ebceecf7e (/c1/_internal_ping) 2020-03-31 09:32:04,498 [534] [DEBUG] [app] Ending request: urn:request:74295cb9-b855-44ac-8f79-406ebceecf7e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:04,498 [534] [DEBUG] [app] Ending request: urn:request:74295cb9-b855-44ac-8f79-406ebceecf7e (/c1/_internal_ping) 2020-03-31 09:32:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:04,499 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:32:04,499 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:04,500 [518] [DEBUG] [app] Ending request: urn:request:968172b1-c79e-4fa3-97b3-fa1632a1e051 (/health/instance) 2020-03-31 09:32:04,500 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:04,500 [518] [DEBUG] [app] Ending request: urn:request:968172b1-c79e-4fa3-97b3-fa1632a1e051 (/health/instance) 2020-03-31 09:32:04,500 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:32:04,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:32:04,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:32:04,756 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:7b9b6628-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:04 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:32:04,756 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:7b9b6628-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:04 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:32:04,756 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '7b9b6628-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XgMZRz8oOXcqzO0SxY62D8kLqImTT7fzE+npzEYG9AI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:04 GMT'}. storagereplication stdout | 2020-03-31 09:32:04,756 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '7b9b6628-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XgMZRz8oOXcqzO0SxY62D8kLqImTT7fzE+npzEYG9AI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:04 GMT'}. 2020-03-31 09:32:04,845 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:32:04,845 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:32:04,846 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:04 GMT, Server-Request-ID=7b7f00d7-601e-0049-243f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '7b9b6628-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XgMZRz8oOXcqzO0SxY62D8kLqImTT7fzE+npzEYG9AI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:04 GMT'}. 2020-03-31 09:32:04,846 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:04 GMT, Server-Request-ID=7b7f00d7-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:04,846 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:04 GMT, Server-Request-ID=7b7f00d7-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:32:04,846 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:04 GMT, Server-Request-ID=7b7f00d7-601e-0049-243f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '7b9b6628-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XgMZRz8oOXcqzO0SxY62D8kLqImTT7fzE+npzEYG9AI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:04 GMT'}. 2020-03-31 09:32:04,846 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:04 GMT, Server-Request-ID=7b7f00d7-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:04,846 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=7b9b6628-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:04 GMT, Server-Request-ID=7b7f00d7-601e-0049-243f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:04,846 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:32:04,846 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:32:06,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:06,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:06,927 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:16.924497+00:00 (in 9.996727 seconds) storagereplication stdout | 2020-03-31 09:32:06,927 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:16.924497+00:00 (in 9.996727 seconds) 2020-03-31 09:32:08,273 [525] [DEBUG] [app] Starting request: urn:request:60c41b47-5346-4c00-8f12-e2e7e453489d (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:08,273 [525] [DEBUG] [app] Starting request: urn:request:60c41b47-5346-4c00-8f12-e2e7e453489d (/health/instance) 2020-03-31 09:32:08,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:08,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:08,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:08,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:08,280 [550] [DEBUG] [app] Starting request: urn:request:bd0f9203-aab8-4be5-aaf8-f8d84689b019 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:08,280 [550] [DEBUG] [app] Starting request: urn:request:bd0f9203-aab8-4be5-aaf8-f8d84689b019 (/v1/_internal_ping) 2020-03-31 09:32:08,281 [550] [DEBUG] [app] Ending request: urn:request:bd0f9203-aab8-4be5-aaf8-f8d84689b019 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:08,281 [550] [DEBUG] [app] Ending request: urn:request:bd0f9203-aab8-4be5-aaf8-f8d84689b019 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:08,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:08,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:32:08,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:08,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:08,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:08,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:32:08,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:32:08,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:08,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:08,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:08,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:08,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:08,295 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:08,295 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:08,309 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:08,309 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:08,312 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:08,312 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:08,314 [519] [DEBUG] [app] Starting request: urn:request:1bd9c445-d67d-4ac4-bbb8-a5ab8e1b8e4e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:08,314 [519] [DEBUG] [app] Starting request: urn:request:1bd9c445-d67d-4ac4-bbb8-a5ab8e1b8e4e (/_internal_ping) 2020-03-31 09:32:08,315 [519] [DEBUG] [app] Ending request: urn:request:1bd9c445-d67d-4ac4-bbb8-a5ab8e1b8e4e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:08,315 [519] [DEBUG] [app] Ending request: urn:request:1bd9c445-d67d-4ac4-bbb8-a5ab8e1b8e4e (/_internal_ping) 2020-03-31 09:32:08,316 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:08,316 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:32:08,316 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:08,316 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:32:08,319 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:08,319 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:08,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:32:08,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:08,322 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:08,322 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:08,323 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:32:08,323 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:32:08,327 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:08,327 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:08,330 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:08,329 [536] [DEBUG] [app] Starting request: urn:request:25849134-ceeb-4106-a4d6-bc19f195146f (/c1/_internal_ping) 2020-03-31 09:32:08,329 [536] [DEBUG] [app] Ending request: urn:request:25849134-ceeb-4106-a4d6-bc19f195146f (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:08,330 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:32:08,329 [536] [DEBUG] [app] Starting request: urn:request:25849134-ceeb-4106-a4d6-bc19f195146f (/c1/_internal_ping) 2020-03-31 09:32:08,329 [536] [DEBUG] [app] Ending request: urn:request:25849134-ceeb-4106-a4d6-bc19f195146f (/c1/_internal_ping) 2020-03-31 09:32:08,331 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:08,332 [525] [DEBUG] [app] Ending request: urn:request:60c41b47-5346-4c00-8f12-e2e7e453489d (/health/instance) 2020-03-31 09:32:08,332 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:32:08,331 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:08,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:32:08,332 [525] [DEBUG] [app] Ending request: urn:request:60c41b47-5346-4c00-8f12-e2e7e453489d (/health/instance) 2020-03-31 09:32:08,332 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:08,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:32:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:32:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:21.440707+00:00 (in 9.999282 seconds) 2020-03-31 09:32:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:21 UTC)" (scheduled at 2020-03-31 09:32:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:32:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:21.440707+00:00 (in 9.999282 seconds) 2020-03-31 09:32:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:21 UTC)" (scheduled at 2020-03-31 09:32:11.440707+00:00) 2020-03-31 09:32:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:32:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:32:11,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:32:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:32:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:32:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:21 UTC)" executed successfully 2020-03-31 09:32:12,854 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:806f1f64-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:12 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:32:12,854 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:806f1f64-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:12 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:32:12,855 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '806f1f64-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:x/7fJoN1VNyFrBVB2UZFpNcV+fPYOkzmXKlvFTx8pLA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:12 GMT'}. storagereplication stdout | 2020-03-31 09:32:12,855 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '806f1f64-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:x/7fJoN1VNyFrBVB2UZFpNcV+fPYOkzmXKlvFTx8pLA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:12 GMT'}. 2020-03-31 09:32:12,943 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:32:12,943 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:32:12,944 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:12 GMT, Server-Request-ID=7b7f1565-601e-0049-5f3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '806f1f64-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:x/7fJoN1VNyFrBVB2UZFpNcV+fPYOkzmXKlvFTx8pLA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:12 GMT'}. 2020-03-31 09:32:12,944 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:12 GMT, Server-Request-ID=7b7f1565-601e-0049-5f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:12,944 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:12 GMT, Server-Request-ID=7b7f1565-601e-0049-5f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:12,944 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:32:12,944 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:12 GMT, Server-Request-ID=7b7f1565-601e-0049-5f3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '806f1f64-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:x/7fJoN1VNyFrBVB2UZFpNcV+fPYOkzmXKlvFTx8pLA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:12 GMT'}. 2020-03-31 09:32:12,944 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:12 GMT, Server-Request-ID=7b7f1565-601e-0049-5f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:12,944 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=806f1f64-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:12 GMT, Server-Request-ID=7b7f1565-601e-0049-5f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:12,944 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:32:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:14 UTC)" (scheduled at 2020-03-31 09:32:14.323352+00:00) 2020-03-31 09:32:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:24.323352+00:00 (in 9.998850 seconds) 2020-03-31 09:32:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:32:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:14 UTC)" (scheduled at 2020-03-31 09:32:14.323352+00:00) 2020-03-31 09:32:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:24.323352+00:00 (in 9.998850 seconds) 2020-03-31 09:32:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 14, 325693), True, datetime.datetime(2020, 3, 31, 9, 32, 14, 325693), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:32:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 14, 325693), True, datetime.datetime(2020, 3, 31, 9, 32, 14, 325693), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:32:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:32:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:24 UTC)" executed successfully 2020-03-31 09:32:14,446 [518] [DEBUG] [app] Starting request: urn:request:0cf02694-9d08-4b87-b6a4-6fc4452b75ca (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:14,446 [518] [DEBUG] [app] Starting request: urn:request:0cf02694-9d08-4b87-b6a4-6fc4452b75ca (/health/instance) 2020-03-31 09:32:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:14,455 [556] [DEBUG] [app] Starting request: urn:request:2c795acf-ed21-49f2-b9f2-6332314222d9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:14,455 [556] [DEBUG] [app] Starting request: urn:request:2c795acf-ed21-49f2-b9f2-6332314222d9 (/v1/_internal_ping) 2020-03-31 09:32:14,456 [556] [DEBUG] [app] Ending request: urn:request:2c795acf-ed21-49f2-b9f2-6332314222d9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:14,456 [556] [DEBUG] [app] Ending request: urn:request:2c795acf-ed21-49f2-b9f2-6332314222d9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:14,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:32:14,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:32:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:32:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:14,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:14,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:14,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:14,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:14,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:14,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:14,490 [527] [DEBUG] [app] Starting request: urn:request:0a9202ab-2334-4e78-a8bd-e8c1418e2444 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:14,490 [527] [DEBUG] [app] Starting request: urn:request:0a9202ab-2334-4e78-a8bd-e8c1418e2444 (/_internal_ping) 2020-03-31 09:32:14,491 [527] [DEBUG] [app] Ending request: urn:request:0a9202ab-2334-4e78-a8bd-e8c1418e2444 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:14,491 [527] [DEBUG] [app] Ending request: urn:request:0a9202ab-2334-4e78-a8bd-e8c1418e2444 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:32:14,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:32:14,491 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:14,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:32:14,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:14,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:32:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:14,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:14,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:14,504 [537] [DEBUG] [app] Starting request: urn:request:0f0d28fb-c761-4c42-baf9-01898d75c293 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:14,504 [537] [DEBUG] [app] Starting request: urn:request:0f0d28fb-c761-4c42-baf9-01898d75c293 (/c1/_internal_ping) 2020-03-31 09:32:14,505 [537] [DEBUG] [app] Ending request: urn:request:0f0d28fb-c761-4c42-baf9-01898d75c293 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:14,505 [537] [DEBUG] [app] Ending request: urn:request:0f0d28fb-c761-4c42-baf9-01898d75c293 (/c1/_internal_ping) 2020-03-31 09:32:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:14,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:14,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:32:14,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:14,507 [518] [DEBUG] [app] Ending request: urn:request:0cf02694-9d08-4b87-b6a4-6fc4452b75ca (/health/instance) 2020-03-31 09:32:14,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:14,507 [518] [DEBUG] [app] Ending request: urn:request:0cf02694-9d08-4b87-b6a4-6fc4452b75ca (/health/instance) 2020-03-31 09:32:14,507 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:32:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:32:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:32:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:32:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:45.064514+00:00 (in 29.999058 seconds) 2020-03-31 09:32:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:45 UTC)" (scheduled at 2020-03-31 09:32:15.064514+00:00) gcworker stdout | 2020-03-31 09:32:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:45.064514+00:00 (in 29.999058 seconds) 2020-03-31 09:32:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:45 UTC)" (scheduled at 2020-03-31 09:32:15.064514+00:00) 2020-03-31 09:32:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437535, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:32:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437535, 1209600, 500, 1, 0]) 2020-03-31 09:32:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:32:15,082 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:32:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:32:15,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:32:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:45 UTC)" executed successfully 2020-03-31 09:32:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:26.924497+00:00 (in 9.999079 seconds) storagereplication stdout | 2020-03-31 09:32:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:26.924497+00:00 (in 9.999079 seconds) 2020-03-31 09:32:18,273 [518] [DEBUG] [app] Starting request: urn:request:efbbcf4a-f5c5-49f9-8767-5fee59ef23d1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:18,273 [518] [DEBUG] [app] Starting request: urn:request:efbbcf4a-f5c5-49f9-8767-5fee59ef23d1 (/health/instance) 2020-03-31 09:32:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:18,277 [547] [DEBUG] [app] Starting request: urn:request:c949e5df-3b04-45b2-a28d-97cd8b3c84fe (/v1/_internal_ping) 2020-03-31 09:32:18,278 [547] [DEBUG] [app] Ending request: urn:request:c949e5df-3b04-45b2-a28d-97cd8b3c84fe (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:18,279 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:32:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:32:18,277 [547] [DEBUG] [app] Starting request: urn:request:c949e5df-3b04-45b2-a28d-97cd8b3c84fe (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:18,278 [547] [DEBUG] [app] Ending request: urn:request:c949e5df-3b04-45b2-a28d-97cd8b3c84fe (/v1/_internal_ping) 2020-03-31 09:32:18,282 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:18,283 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:18,279 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:18,284 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:32:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:32:18,282 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:32:18,283 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:18,284 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:18,311 [526] [DEBUG] [app] Starting request: urn:request:47f33a95-fce5-44fb-8c0b-1737c36d65dd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:18,311 [526] [DEBUG] [app] Starting request: urn:request:47f33a95-fce5-44fb-8c0b-1737c36d65dd (/_internal_ping) 2020-03-31 09:32:18,312 [526] [DEBUG] [app] Ending request: urn:request:47f33a95-fce5-44fb-8c0b-1737c36d65dd (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:32:18,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:32:18,312 [526] [DEBUG] [app] Ending request: urn:request:47f33a95-fce5-44fb-8c0b-1737c36d65dd (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:18,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:32:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:32:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:18,325 [543] [DEBUG] [app] Starting request: urn:request:d7bc344c-d427-491e-8de5-838a267148a9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:18,325 [543] [DEBUG] [app] Starting request: urn:request:d7bc344c-d427-491e-8de5-838a267148a9 (/c1/_internal_ping) 2020-03-31 09:32:18,326 [543] [DEBUG] [app] Ending request: urn:request:d7bc344c-d427-491e-8de5-838a267148a9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:18,326 [543] [DEBUG] [app] Ending request: urn:request:d7bc344c-d427-491e-8de5-838a267148a9 (/c1/_internal_ping) 2020-03-31 09:32:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:18,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:18,328 [518] [DEBUG] [app] Ending request: urn:request:efbbcf4a-f5c5-49f9-8767-5fee59ef23d1 (/health/instance) 2020-03-31 09:32:18,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:32:18,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:18,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:32:18,328 [518] [DEBUG] [app] Ending request: urn:request:efbbcf4a-f5c5-49f9-8767-5fee59ef23d1 (/health/instance) 2020-03-31 09:32:18,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:18,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:32:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:32:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:31.440707+00:00 (in 9.999211 seconds) 2020-03-31 09:32:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:31 UTC)" (scheduled at 2020-03-31 09:32:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:32:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:31.440707+00:00 (in 9.999211 seconds) 2020-03-31 09:32:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:31 UTC)" (scheduled at 2020-03-31 09:32:21.440707+00:00) 2020-03-31 09:32:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:32:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:32:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:32:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:32:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:32:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:31 UTC)" executed successfully 2020-03-31 09:32:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:32:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:52.113221+00:00 (in 29.998912 seconds) 2020-03-31 09:32:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:52 UTC)" (scheduled at 2020-03-31 09:32:22.113221+00:00) securityworker stdout | 2020-03-31 09:32:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:52.113221+00:00 (in 29.998912 seconds) 2020-03-31 09:32:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:52 UTC)" (scheduled at 2020-03-31 09:32:22.113221+00:00) 2020-03-31 09:32:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:32:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:32:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:32:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:32:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:32:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:32:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:32:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:32:52 UTC)" executed successfully 2020-03-31 09:32:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:31.498397+00:00 (in 7.173669 seconds) 2020-03-31 09:32:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:34 UTC)" (scheduled at 2020-03-31 09:32:24.323352+00:00) 2020-03-31 09:32:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:32:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:31.498397+00:00 (in 7.173669 seconds) 2020-03-31 09:32:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:34 UTC)" (scheduled at 2020-03-31 09:32:24.323352+00:00) 2020-03-31 09:32:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 24, 325581), True, datetime.datetime(2020, 3, 31, 9, 32, 24, 325581), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:32:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 24, 325581), True, datetime.datetime(2020, 3, 31, 9, 32, 24, 325581), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:32:24,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:32:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:34 UTC)" executed successfully 2020-03-31 09:32:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:32:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:33.330112+00:00 (in 8.918145 seconds) 2020-03-31 09:32:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:24 UTC)" (scheduled at 2020-03-31 09:32:24.410927+00:00) 2020-03-31 09:32:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:32:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:33.330112+00:00 (in 8.918145 seconds) 2020-03-31 09:32:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:24 UTC)" (scheduled at 2020-03-31 09:32:24.410927+00:00) 2020-03-31 09:32:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:32:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:24 UTC)" executed successfully 2020-03-31 09:32:24,445 [518] [DEBUG] [app] Starting request: urn:request:679bae36-f5a6-409a-b990-ac747bd82be1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:24,445 [518] [DEBUG] [app] Starting request: urn:request:679bae36-f5a6-409a-b990-ac747bd82be1 (/health/instance) 2020-03-31 09:32:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:24,453 [538] [DEBUG] [app] Starting request: urn:request:a62f4634-3daf-4c5c-8f59-fff29873dce1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:24,453 [538] [DEBUG] [app] Starting request: urn:request:a62f4634-3daf-4c5c-8f59-fff29873dce1 (/v1/_internal_ping) 2020-03-31 09:32:24,454 [538] [DEBUG] [app] Ending request: urn:request:a62f4634-3daf-4c5c-8f59-fff29873dce1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:24,454 [538] [DEBUG] [app] Ending request: urn:request:a62f4634-3daf-4c5c-8f59-fff29873dce1 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:32:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:32:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:32:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:32:24,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:24,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:24,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:24,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:24,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:24,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:24,485 [518] [DEBUG] [app] Starting request: urn:request:b5e6fcbf-8143-4c04-8a82-1b683736224b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:24,485 [518] [DEBUG] [app] Starting request: urn:request:b5e6fcbf-8143-4c04-8a82-1b683736224b (/_internal_ping) 2020-03-31 09:32:24,486 [518] [DEBUG] [app] Ending request: urn:request:b5e6fcbf-8143-4c04-8a82-1b683736224b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:24,486 [518] [DEBUG] [app] Ending request: urn:request:b5e6fcbf-8143-4c04-8a82-1b683736224b (/_internal_ping) 2020-03-31 09:32:24,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:24,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:32:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:32:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:32:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:32:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:24,499 [543] [DEBUG] [app] Starting request: urn:request:d60b89a7-b26d-4a58-88ae-f50379e84ffc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:24,499 [543] [DEBUG] [app] Starting request: urn:request:d60b89a7-b26d-4a58-88ae-f50379e84ffc (/c1/_internal_ping) 2020-03-31 09:32:24,500 [543] [DEBUG] [app] Ending request: urn:request:d60b89a7-b26d-4a58-88ae-f50379e84ffc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:24,500 [543] [DEBUG] [app] Ending request: urn:request:d60b89a7-b26d-4a58-88ae-f50379e84ffc (/c1/_internal_ping) 2020-03-31 09:32:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:24,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:32:24,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:24,503 [518] [DEBUG] [app] Ending request: urn:request:679bae36-f5a6-409a-b990-ac747bd82be1 (/health/instance) 2020-03-31 09:32:24,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:24,503 [518] [DEBUG] [app] Ending request: urn:request:679bae36-f5a6-409a-b990-ac747bd82be1 (/health/instance) 2020-03-31 09:32:24,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:24,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:32:24,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:32:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:36.924497+00:00 (in 9.998996 seconds) storagereplication stdout | 2020-03-31 09:32:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:36.924497+00:00 (in 9.998996 seconds) 2020-03-31 09:32:28,273 [518] [DEBUG] [app] Starting request: urn:request:c9358a39-87c9-4cc2-a72c-bea753e19a3d (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:28,273 [518] [DEBUG] [app] Starting request: urn:request:c9358a39-87c9-4cc2-a72c-bea753e19a3d (/health/instance) 2020-03-31 09:32:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:28,277 [538] [DEBUG] [app] Starting request: urn:request:922e2d1a-e408-4f02-be80-bddb1c233145 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:28,277 [538] [DEBUG] [app] Starting request: urn:request:922e2d1a-e408-4f02-be80-bddb1c233145 (/v1/_internal_ping) 2020-03-31 09:32:28,278 [538] [DEBUG] [app] Ending request: urn:request:922e2d1a-e408-4f02-be80-bddb1c233145 (/v1/_internal_ping) 2020-03-31 09:32:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:32:28,278 [538] [DEBUG] [app] Ending request: urn:request:922e2d1a-e408-4f02-be80-bddb1c233145 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:28,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:28,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:28,281 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:32:28,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:32:28,281 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:28,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:28,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:28,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:28,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:28,317 [518] [DEBUG] [app] Starting request: urn:request:8888f900-8744-473e-b3eb-a8156cbc35f2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:28,317 [518] [DEBUG] [app] Starting request: urn:request:8888f900-8744-473e-b3eb-a8156cbc35f2 (/_internal_ping) 2020-03-31 09:32:28,318 [518] [DEBUG] [app] Ending request: urn:request:8888f900-8744-473e-b3eb-a8156cbc35f2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:28,318 [518] [DEBUG] [app] Ending request: urn:request:8888f900-8744-473e-b3eb-a8156cbc35f2 (/_internal_ping) 2020-03-31 09:32:28,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:32:28,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:32:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:32:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:32:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:32:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:28,331 [536] [DEBUG] [app] Starting request: urn:request:8087d95f-7195-4b66-8ce6-8836b68388b6 (/c1/_internal_ping) 2020-03-31 09:32:28,332 [536] [DEBUG] [app] Ending request: urn:request:8087d95f-7195-4b66-8ce6-8836b68388b6 (/c1/_internal_ping) 2020-03-31 09:32:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:28,332 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:32:28,331 [536] [DEBUG] [app] Starting request: urn:request:8087d95f-7195-4b66-8ce6-8836b68388b6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:28,332 [536] [DEBUG] [app] Ending request: urn:request:8087d95f-7195-4b66-8ce6-8836b68388b6 (/c1/_internal_ping) 2020-03-31 09:32:28,334 [518] [DEBUG] [app] Ending request: urn:request:c9358a39-87c9-4cc2-a72c-bea753e19a3d (/health/instance) 2020-03-31 09:32:28,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:28,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:32:28,332 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:28,334 [518] [DEBUG] [app] Ending request: urn:request:c9358a39-87c9-4cc2-a72c-bea753e19a3d (/health/instance) 2020-03-31 09:32:28,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:28,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:32:28,960 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:32:28,960 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy 2020-03-31 09:32:28,960 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:32:28,960 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:32:28,960 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:32:28,960 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:32:28,968 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) storagereplication stdout | 2020-03-31 09:32:28,968 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) 2020-03-31 09:32:28,972 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 37, 28, 967885), True, u'ac14061e-bfca-44ec-8495-8cc1007b8301', 2]) storagereplication stdout | 2020-03-31 09:32:28,972 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 37, 28, 967885), True, u'ac14061e-bfca-44ec-8495-8cc1007b8301', 2]) 2020-03-31 09:32:28,978 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 28, 977394), True, datetime.datetime(2020, 3, 31, 9, 32, 28, 977394), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:32:28,978 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 28, 977394), True, datetime.datetime(2020, 3, 31, 9, 32, 28, 977394), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:32:28,982 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 52, 28, 977394), 1, u'd42e24fb-f709-4b0d-bc02-59fcab0b9b5f', 1, u'10381c13-b992-46bd-8bf9-d9ab86951c36']) storagereplication stdout | 2020-03-31 09:32:28,982 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 52, 28, 977394), 1, u'd42e24fb-f709-4b0d-bc02-59fcab0b9b5f', 1, u'10381c13-b992-46bd-8bf9-d9ab86951c36']) 2020-03-31 09:32:28,989 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} storagereplication stdout | 2020-03-31 09:32:28,989 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:32:28,989 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:28,989 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 storagereplication stdout | 2020-03-31 09:32:28,989 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:28,989 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 2020-03-31 09:32:28,991 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:32:28,991 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:32:29,003 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:32:29,003 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:32:29,007 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) storagereplication stdout | 2020-03-31 09:32:29,007 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:32:29,011 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:32:29,011 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:32:29,012 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8a1083aa-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:29 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:32:29,012 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1083aa-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1083aa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BPMgqkXrudSbS8mqimzz+N/myPfPwV/YL61oZxNYzAc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. storagereplication stdout | 2020-03-31 09:32:29,012 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8a1083aa-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:29 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:32:29,012 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1083aa-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1083aa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BPMgqkXrudSbS8mqimzz+N/myPfPwV/YL61oZxNYzAc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. 2020-03-31 09:32:29,021 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:32:29,021 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:32:29,022 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1083aa-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=77730b6d-401e-009e-253f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1083aa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BPMgqkXrudSbS8mqimzz+N/myPfPwV/YL61oZxNYzAc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. storagereplication stdout | 2020-03-31 09:32:29,022 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a1083aa-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=77730b6d-401e-009e-253f-07ccb1000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a1083aa-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BPMgqkXrudSbS8mqimzz+N/myPfPwV/YL61oZxNYzAc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. 2020-03-31 09:32:29,023 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:32:29,023 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:29,023 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:32:29,023 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:32:29,023 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:29,023 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:32:29,024 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:8a12635a-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=pYudaIR7Z7tnwtTsukpPlSsasqpqMc8cf/TACxD1ui0%3D&sv=2017-07-29&se=2020-03-31T09%3A33%3A29Z x-ms-date:Tue, 31 Mar 2020 09:32:29 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:32:29,024 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a12635a-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=pYudaIR7Z7tnwtTsukpPlSsasqpqMc8cf/TACxD1ui0%3D&sv=2017-07-29&se=2020-03-31T09%3A33%3A29Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:M5ff0/4FrqAsDlzousYFOezqtYWX5T13HUz/aptgaeQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8a12635a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. storagereplication stdout | 2020-03-31 09:32:29,024 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:8a12635a-7332-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=pYudaIR7Z7tnwtTsukpPlSsasqpqMc8cf/TACxD1ui0%3D&sv=2017-07-29&se=2020-03-31T09%3A33%3A29Z x-ms-date:Tue, 31 Mar 2020 09:32:29 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:32:29,024 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a12635a-7332-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=pYudaIR7Z7tnwtTsukpPlSsasqpqMc8cf/TACxD1ui0%3D&sv=2017-07-29&se=2020-03-31T09%3A33%3A29Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:M5ff0/4FrqAsDlzousYFOezqtYWX5T13HUz/aptgaeQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8a12635a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. 2020-03-31 09:32:29,071 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:32:29,071 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 2020-03-31 09:32:29,071 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a12635a-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=77730b71-401e-009e-273f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=pYudaIR7Z7tnwtTsukpPlSsasqpqMc8cf/TACxD1ui0%3D&sv=2017-07-29&se=2020-03-31T09%3A33%3A29Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:M5ff0/4FrqAsDlzousYFOezqtYWX5T13HUz/aptgaeQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8a12635a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. storagereplication stdout | 2020-03-31 09:32:29,071 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a12635a-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=77730b71-401e-009e-273f-07ccb1000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=pYudaIR7Z7tnwtTsukpPlSsasqpqMc8cf/TACxD1ui0%3D&sv=2017-07-29&se=2020-03-31T09%3A33%3A29Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:M5ff0/4FrqAsDlzousYFOezqtYWX5T13HUz/aptgaeQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '8a12635a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. 2020-03-31 09:32:29,072 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:32:29,072 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8a19ccd0-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:29 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:32:29,072 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a19ccd0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:e2443MugwqRoEvnHkuP8BE9tcE50RNJ+kwlsWtS1RBs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. storagereplication stdout | 2020-03-31 09:32:29,072 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:32:29,072 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8a19ccd0-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:29 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:32:29,072 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a19ccd0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:e2443MugwqRoEvnHkuP8BE9tcE50RNJ+kwlsWtS1RBs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. 2020-03-31 09:32:29,163 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:32:29,163 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:32:29,164 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=7b7f4272-601e-0049-6a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a19ccd0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:e2443MugwqRoEvnHkuP8BE9tcE50RNJ+kwlsWtS1RBs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. 2020-03-31 09:32:29,164 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=7b7f4272-601e-0049-6a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:29,164 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=7b7f4272-601e-0049-6a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:29,164 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:32:29,164 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=7b7f4272-601e-0049-6a3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8a19ccd0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:e2443MugwqRoEvnHkuP8BE9tcE50RNJ+kwlsWtS1RBs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:29 GMT'}. 2020-03-31 09:32:29,164 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=7b7f4272-601e-0049-6a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:29,164 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8a19ccd0-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:28 GMT, Server-Request-ID=7b7f4272-601e-0049-6a3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:29,164 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:32:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:32:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:00.337699+00:00 (in 29.999172 seconds) 2020-03-31 09:32:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:00 UTC)" (scheduled at 2020-03-31 09:32:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:32:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:00.337699+00:00 (in 29.999172 seconds) 2020-03-31 09:32:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:00 UTC)" (scheduled at 2020-03-31 09:32:30.337699+00:00) 2020-03-31 09:32:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 32, 30, 339241), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:32:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 32, 30, 339241), False, 50, 1, 0]) 2020-03-31 09:32:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:32:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:32:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:32:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:00 UTC)" executed successfully 2020-03-31 09:32:31,167 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8b59570a-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:31 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:32:31,167 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8b59570a-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:31 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:32:31,167 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8b59570a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XZIzFdvpEKP8qnJQqUrWZpj/3QA9Zvdv7DnIu2PjwXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:31 GMT'}. storagereplication stdout | 2020-03-31 09:32:31,167 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8b59570a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XZIzFdvpEKP8qnJQqUrWZpj/3QA9Zvdv7DnIu2PjwXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:31 GMT'}. 2020-03-31 09:32:31,254 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:32:31,254 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:32:31,255 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:31 GMT, Server-Request-ID=7b7f4803-601e-0049-183f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8b59570a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XZIzFdvpEKP8qnJQqUrWZpj/3QA9Zvdv7DnIu2PjwXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:31 GMT'}. 2020-03-31 09:32:31,255 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:31 GMT, Server-Request-ID=7b7f4803-601e-0049-183f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:31,256 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:31 GMT, Server-Request-ID=7b7f4803-601e-0049-183f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:31,256 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:32:31,255 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:31 GMT, Server-Request-ID=7b7f4803-601e-0049-183f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8b59570a-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:XZIzFdvpEKP8qnJQqUrWZpj/3QA9Zvdv7DnIu2PjwXI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:31 GMT'}. 2020-03-31 09:32:31,255 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:31 GMT, Server-Request-ID=7b7f4803-601e-0049-183f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:31,256 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8b59570a-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:31 GMT, Server-Request-ID=7b7f4803-601e-0049-183f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:31,256 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:32:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:32:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:41 UTC)" (scheduled at 2020-03-31 09:32:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:32:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:41 UTC)" (scheduled at 2020-03-31 09:32:31.440707+00:00) 2020-03-31 09:32:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:41.440707+00:00 (in 9.999177 seconds) repositoryactioncounter stdout | 2020-03-31 09:32:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:41.440707+00:00 (in 9.999177 seconds) 2020-03-31 09:32:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:32:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:32:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:32:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:32:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:32:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:41 UTC)" executed successfully 2020-03-31 09:32:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:31 UTC)" (scheduled at 2020-03-31 09:32:31.498397+00:00) notificationworker stdout | 2020-03-31 09:32:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:32:31 UTC)" (scheduled at 2020-03-31 09:32:31.498397+00:00) 2020-03-31 09:32:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), u'notification/%']) notificationworker stdout | 2020-03-31 09:32:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), u'notification/%']) 2020-03-31 09:32:31,502 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:34.323352+00:00 (in 2.824106 seconds) notificationworker stdout | 2020-03-31 09:32:31,502 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:34.323352+00:00 (in 2.824106 seconds) 2020-03-31 09:32:31,516 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), True, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:32:31,516 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), True, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), 0, u'notification/%']) 2020-03-31 09:32:31,521 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), True, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), u'notification/%']) notificationworker stdout | 2020-03-31 09:32:31,521 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), True, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 32, 31, 499684), u'notification/%']) 2020-03-31 09:32:31,524 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:32:31,524 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:31,524 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:37:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:31,524 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:37:31 UTC)" executed successfully 2020-03-31 09:32:31,527 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:32:31,527 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:31,528 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:32:31,528 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:31,529 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 09:32:31,529 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:32:33,335 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:33,335 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:03.330112+00:00 (in 29.994628 seconds) 2020-03-31 09:32:33,337 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:03 UTC)" (scheduled at 2020-03-31 09:32:33.330112+00:00) 2020-03-31 09:32:33,337 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:32:33,335 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:33,335 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:03.330112+00:00 (in 29.994628 seconds) 2020-03-31 09:32:33,337 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:03 UTC)" (scheduled at 2020-03-31 09:32:33.330112+00:00) 2020-03-31 09:32:33,337 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:33,340 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 33, 337829), True, datetime.datetime(2020, 3, 31, 9, 32, 33, 337829), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:32:33,340 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 33, 337829), True, datetime.datetime(2020, 3, 31, 9, 32, 33, 337829), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:32:33,356 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:32:33,356 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:33,356 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:33,357 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:32:33,356 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:33,357 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:03 UTC)" executed successfully 2020-03-31 09:32:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:32:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:43.539999+00:00 (in 10.172058 seconds) 2020-03-31 09:32:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:33 UTC)" (scheduled at 2020-03-31 09:32:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:32:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:43.539999+00:00 (in 10.172058 seconds) 2020-03-31 09:32:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:33 UTC)" (scheduled at 2020-03-31 09:32:33.366791+00:00) 2020-03-31 09:32:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:32:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 33, 368795), True, datetime.datetime(2020, 3, 31, 9, 32, 33, 368795), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:32:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 33, 368795), True, datetime.datetime(2020, 3, 31, 9, 32, 33, 368795), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:32:33,387 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:32:33,387 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:32:33,387 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:33,387 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:33 UTC)" executed successfully 2020-03-31 09:32:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:44.323352+00:00 (in 9.996795 seconds) 2020-03-31 09:32:34,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:44 UTC)" (scheduled at 2020-03-31 09:32:34.323352+00:00) 2020-03-31 09:32:34,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:32:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:44.323352+00:00 (in 9.996795 seconds) 2020-03-31 09:32:34,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:44 UTC)" (scheduled at 2020-03-31 09:32:34.323352+00:00) 2020-03-31 09:32:34,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 34, 327410), True, datetime.datetime(2020, 3, 31, 9, 32, 34, 327410), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:32:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 34, 327410), True, datetime.datetime(2020, 3, 31, 9, 32, 34, 327410), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:32:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:32:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:44 UTC)" executed successfully 2020-03-31 09:32:34,446 [527] [DEBUG] [app] Starting request: urn:request:1273f867-a1e2-4ced-9170-ee9f5dbeff85 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:34,446 [527] [DEBUG] [app] Starting request: urn:request:1273f867-a1e2-4ced-9170-ee9f5dbeff85 (/health/instance) 2020-03-31 09:32:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:34,453 [538] [DEBUG] [app] Starting request: urn:request:b0c5775c-a7cf-4150-b421-86cd34865636 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:34,453 [538] [DEBUG] [app] Starting request: urn:request:b0c5775c-a7cf-4150-b421-86cd34865636 (/v1/_internal_ping) 2020-03-31 09:32:34,455 [538] [DEBUG] [app] Ending request: urn:request:b0c5775c-a7cf-4150-b421-86cd34865636 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:34,455 [538] [DEBUG] [app] Ending request: urn:request:b0c5775c-a7cf-4150-b421-86cd34865636 (/v1/_internal_ping) 2020-03-31 09:32:34,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:32:34,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:32:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:32:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:32:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:34,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:34,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:34,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:34,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:34,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:34,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:34,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:34,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:34,489 [518] [DEBUG] [app] Starting request: urn:request:f063be25-9fbb-40b9-8bac-94aff7f6ceb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:34,489 [518] [DEBUG] [app] Starting request: urn:request:f063be25-9fbb-40b9-8bac-94aff7f6ceb2 (/_internal_ping) 2020-03-31 09:32:34,490 [518] [DEBUG] [app] Ending request: urn:request:f063be25-9fbb-40b9-8bac-94aff7f6ceb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:34,490 [518] [DEBUG] [app] Ending request: urn:request:f063be25-9fbb-40b9-8bac-94aff7f6ceb2 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:32:34,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:32:34,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:32:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:34,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:32:34,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:34,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:34,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:34,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:34,505 [537] [DEBUG] [app] Starting request: urn:request:510f7783-a3ee-43e5-8ce5-4d8cc700532e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:34,505 [537] [DEBUG] [app] Starting request: urn:request:510f7783-a3ee-43e5-8ce5-4d8cc700532e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:34,506 [537] [DEBUG] [app] Ending request: urn:request:510f7783-a3ee-43e5-8ce5-4d8cc700532e (/c1/_internal_ping) 2020-03-31 09:32:34,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:34,507 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:32:34,506 [537] [DEBUG] [app] Ending request: urn:request:510f7783-a3ee-43e5-8ce5-4d8cc700532e (/c1/_internal_ping) 2020-03-31 09:32:34,509 [527] [DEBUG] [app] Ending request: urn:request:1273f867-a1e2-4ced-9170-ee9f5dbeff85 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:34,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:34,509 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:34,510 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-verbs stdout | 2020-03-31 09:32:34,507 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:34,509 [527] [DEBUG] [app] Ending request: urn:request:1273f867-a1e2-4ced-9170-ee9f5dbeff85 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:34,509 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:34,510 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:32:35,257 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8dc96e6c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:35 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:32:35,257 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:8dc96e6c-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:35 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:32:35,257 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8dc96e6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:O3LuI3CpwseSNB7ToQAdsLZwz906ENd4Zlgb/hak3As=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:35 GMT'}. storagereplication stdout | 2020-03-31 09:32:35,257 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8dc96e6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:O3LuI3CpwseSNB7ToQAdsLZwz906ENd4Zlgb/hak3As=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:35 GMT'}. 2020-03-31 09:32:35,346 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:32:35,346 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:32:35,347 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:35 GMT, Server-Request-ID=7b7f51fc-601e-0049-493f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8dc96e6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:O3LuI3CpwseSNB7ToQAdsLZwz906ENd4Zlgb/hak3As=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:35 GMT'}. 2020-03-31 09:32:35,347 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:35 GMT, Server-Request-ID=7b7f51fc-601e-0049-493f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:35,347 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:35 GMT, Server-Request-ID=7b7f51fc-601e-0049-493f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:32:35,347 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:35 GMT, Server-Request-ID=7b7f51fc-601e-0049-493f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '8dc96e6c-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:O3LuI3CpwseSNB7ToQAdsLZwz906ENd4Zlgb/hak3As=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:35 GMT'}. 2020-03-31 09:32:35,347 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:35 GMT, Server-Request-ID=7b7f51fc-601e-0049-493f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:35,347 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=8dc96e6c-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:35 GMT, Server-Request-ID=7b7f51fc-601e-0049-493f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:35,347 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:32:35,347 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:32:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:38.010934+00:00 (in 1.085667 seconds) storagereplication stdout | 2020-03-31 09:32:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:38.010934+00:00 (in 1.085667 seconds) 2020-03-31 09:32:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:46.924497+00:00 (in 8.912783 seconds) 2020-03-31 09:32:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:38 UTC)" (scheduled at 2020-03-31 09:32:38.010934+00:00) 2020-03-31 09:32:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:32:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:46.924497+00:00 (in 8.912783 seconds) 2020-03-31 09:32:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:38 UTC)" (scheduled at 2020-03-31 09:32:38.010934+00:00) 2020-03-31 09:32:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:38 UTC)" executed successfully 2020-03-31 09:32:38,273 [527] [DEBUG] [app] Starting request: urn:request:9b99da9c-3090-4b9b-9e3b-8b12586256d2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:38,273 [527] [DEBUG] [app] Starting request: urn:request:9b99da9c-3090-4b9b-9e3b-8b12586256d2 (/health/instance) 2020-03-31 09:32:38,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:38,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:38,278 [554] [DEBUG] [app] Starting request: urn:request:ca03703f-84a8-4aa0-b7c1-fcaf3a4a3e1a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:38,278 [554] [DEBUG] [app] Starting request: urn:request:ca03703f-84a8-4aa0-b7c1-fcaf3a4a3e1a (/v1/_internal_ping) 2020-03-31 09:32:38,279 [554] [DEBUG] [app] Ending request: urn:request:ca03703f-84a8-4aa0-b7c1-fcaf3a4a3e1a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:38,279 [554] [DEBUG] [app] Ending request: urn:request:ca03703f-84a8-4aa0-b7c1-fcaf3a4a3e1a (/v1/_internal_ping) 2020-03-31 09:32:38,280 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:32:38,280 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:32:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:38,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:32:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:32:38,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:38,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:38,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:38,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:38,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:38,312 [518] [DEBUG] [app] Starting request: urn:request:7f1518c4-612e-4ed2-a95b-e10c5da64cad (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:38,312 [518] [DEBUG] [app] Starting request: urn:request:7f1518c4-612e-4ed2-a95b-e10c5da64cad (/_internal_ping) 2020-03-31 09:32:38,313 [518] [DEBUG] [app] Ending request: urn:request:7f1518c4-612e-4ed2-a95b-e10c5da64cad (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:38,313 [518] [DEBUG] [app] Ending request: urn:request:7f1518c4-612e-4ed2-a95b-e10c5da64cad (/_internal_ping) 2020-03-31 09:32:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:38,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:38,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:32:38,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:38,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:32:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:32:38,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:38,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:32:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:38,325 [537] [DEBUG] [app] Starting request: urn:request:10625624-0fd4-44e9-a354-e564ed27b2db (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:38,325 [537] [DEBUG] [app] Starting request: urn:request:10625624-0fd4-44e9-a354-e564ed27b2db (/c1/_internal_ping) 2020-03-31 09:32:38,326 [537] [DEBUG] [app] Ending request: urn:request:10625624-0fd4-44e9-a354-e564ed27b2db (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:38,326 [537] [DEBUG] [app] Ending request: urn:request:10625624-0fd4-44e9-a354-e564ed27b2db (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:38,327 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:32:38,327 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:38,329 [527] [DEBUG] [app] Ending request: urn:request:9b99da9c-3090-4b9b-9e3b-8b12586256d2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:38,329 [527] [DEBUG] [app] Ending request: urn:request:9b99da9c-3090-4b9b-9e3b-8b12586256d2 (/health/instance) 2020-03-31 09:32:38,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:38,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:32:38,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:38,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:32:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:32:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:51.440707+00:00 (in 9.998094 seconds) repositoryactioncounter stdout | 2020-03-31 09:32:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:51.440707+00:00 (in 9.998094 seconds) 2020-03-31 09:32:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:51 UTC)" (scheduled at 2020-03-31 09:32:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:32:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:51 UTC)" (scheduled at 2020-03-31 09:32:41.440707+00:00) 2020-03-31 09:32:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:32:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:32:41,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:32:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:32:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:32:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:51 UTC)" executed successfully 2020-03-31 09:32:43,355 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:929d2be0-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:43 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:32:43,355 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:929d2be0-7332-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:32:43 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:32:43,355 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '929d2be0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ibO5VsPDsw9NC/+G5j/HKgHaPtWWO+jwYV4YkS7Sq10=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:43 GMT'}. storagereplication stdout | 2020-03-31 09:32:43,355 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '929d2be0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ibO5VsPDsw9NC/+G5j/HKgHaPtWWO+jwYV4YkS7Sq10=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:43 GMT'}. 2020-03-31 09:32:43,444 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:32:43,444 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:32:43,445 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:43 GMT, Server-Request-ID=7b7f6768-601e-0049-7f3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '929d2be0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ibO5VsPDsw9NC/+G5j/HKgHaPtWWO+jwYV4YkS7Sq10=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:43 GMT'}. 2020-03-31 09:32:43,445 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:43 GMT, Server-Request-ID=7b7f6768-601e-0049-7f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:43,445 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:43 GMT, Server-Request-ID=7b7f6768-601e-0049-7f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:43,445 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:32:43,445 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:32:43 GMT, Server-Request-ID=7b7f6768-601e-0049-7f3f-079b25000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '929d2be0-7332-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ibO5VsPDsw9NC/+G5j/HKgHaPtWWO+jwYV4YkS7Sq10=', 'x-ms-date': 'Tue, 31 Mar 2020 09:32:43 GMT'}. 2020-03-31 09:32:43,445 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:32:43 GMT, Server-Request-ID=7b7f6768-601e-0049-7f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:43,445 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=929d2be0-7332-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:32:43 GMT, Server-Request-ID=7b7f6768-601e-0049-7f3f-079b25000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:32:43,445 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:32:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:32:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:33.366791+00:00 (in 49.826049 seconds) 2020-03-31 09:32:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:43 UTC)" (scheduled at 2020-03-31 09:32:43.539999+00:00) 2020-03-31 09:32:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:32:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:33.366791+00:00 (in 49.826049 seconds) 2020-03-31 09:32:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:43 UTC)" (scheduled at 2020-03-31 09:32:43.539999+00:00) 2020-03-31 09:32:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:43 UTC)" executed successfully 2020-03-31 09:32:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:49.498966+00:00 (in 5.174817 seconds) 2020-03-31 09:32:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:54 UTC)" (scheduled at 2020-03-31 09:32:44.323352+00:00) notificationworker stdout | 2020-03-31 09:32:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:49.498966+00:00 (in 5.174817 seconds) 2020-03-31 09:32:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:54 UTC)" (scheduled at 2020-03-31 09:32:44.323352+00:00) 2020-03-31 09:32:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:32:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 44, 325117), True, datetime.datetime(2020, 3, 31, 9, 32, 44, 325117), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:32:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 44, 325117), True, datetime.datetime(2020, 3, 31, 9, 32, 44, 325117), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:32:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:32:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:54 UTC)" executed successfully 2020-03-31 09:32:44,446 [526] [DEBUG] [app] Starting request: urn:request:eb4ffede-2567-4b74-91ce-7cb70cd878c1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:44,446 [526] [DEBUG] [app] Starting request: urn:request:eb4ffede-2567-4b74-91ce-7cb70cd878c1 (/health/instance) 2020-03-31 09:32:44,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:44,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:44,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:44,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:44,453 [551] [DEBUG] [app] Starting request: urn:request:5e40685c-5672-4a30-b7ea-9b58a136993e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:44,453 [551] [DEBUG] [app] Starting request: urn:request:5e40685c-5672-4a30-b7ea-9b58a136993e (/v1/_internal_ping) 2020-03-31 09:32:44,454 [551] [DEBUG] [app] Ending request: urn:request:5e40685c-5672-4a30-b7ea-9b58a136993e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:44,454 [551] [DEBUG] [app] Ending request: urn:request:5e40685c-5672-4a30-b7ea-9b58a136993e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:44,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:32:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:44,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:44,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:44,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:32:44,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:32:44,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:44,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:44,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:44,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:44,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:44,465 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:44,465 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:44,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:44,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:44,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:44,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:44,488 [527] [DEBUG] [app] Starting request: urn:request:8f3a1cf8-3c2a-47c1-9a2b-b96fb3b28b31 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:44,488 [527] [DEBUG] [app] Starting request: urn:request:8f3a1cf8-3c2a-47c1-9a2b-b96fb3b28b31 (/_internal_ping) 2020-03-31 09:32:44,489 [527] [DEBUG] [app] Ending request: urn:request:8f3a1cf8-3c2a-47c1-9a2b-b96fb3b28b31 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:44,489 [527] [DEBUG] [app] Ending request: urn:request:8f3a1cf8-3c2a-47c1-9a2b-b96fb3b28b31 (/_internal_ping) 2020-03-31 09:32:44,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:44,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:44,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:32:44,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:44,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:44,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:44,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:32:44,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:44,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:44,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:32:44,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:44,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:44,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:44,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:44,502 [536] [DEBUG] [app] Starting request: urn:request:6c58611e-4e48-4016-9a3b-18d5e32fcfc5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:44,502 [536] [DEBUG] [app] Starting request: urn:request:6c58611e-4e48-4016-9a3b-18d5e32fcfc5 (/c1/_internal_ping) 2020-03-31 09:32:44,503 [536] [DEBUG] [app] Ending request: urn:request:6c58611e-4e48-4016-9a3b-18d5e32fcfc5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:44,503 [536] [DEBUG] [app] Ending request: urn:request:6c58611e-4e48-4016-9a3b-18d5e32fcfc5 (/c1/_internal_ping) 2020-03-31 09:32:44,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:44,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:44,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:32:44,503 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:44,505 [526] [DEBUG] [app] Ending request: urn:request:eb4ffede-2567-4b74-91ce-7cb70cd878c1 (/health/instance) 2020-03-31 09:32:44,506 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:44,505 [526] [DEBUG] [app] Ending request: urn:request:eb4ffede-2567-4b74-91ce-7cb70cd878c1 (/health/instance) 2020-03-31 09:32:44,506 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:44,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:32:44,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:32:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:32:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:15.064514+00:00 (in 29.996742 seconds) 2020-03-31 09:32:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:15 UTC)" (scheduled at 2020-03-31 09:32:45.064514+00:00) gcworker stdout | 2020-03-31 09:32:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:15.064514+00:00 (in 29.996742 seconds) 2020-03-31 09:32:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:15 UTC)" (scheduled at 2020-03-31 09:32:45.064514+00:00) 2020-03-31 09:32:45,068 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2020-03-31 09:32:45,068 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) 2020-03-31 09:32:45,088 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437565, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:32:45,088 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437565, 1209600, 500, 1, 0]) 2020-03-31 09:32:45,094 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:32:45,094 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:32:45,094 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:45,094 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:32:45,094 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:45,094 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:15 UTC)" executed successfully 2020-03-31 09:32:46,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:56.924497+00:00 (in 9.999378 seconds) storagereplication stdout | 2020-03-31 09:32:46,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:56.924497+00:00 (in 9.999378 seconds) 2020-03-31 09:32:48,273 [518] [DEBUG] [app] Starting request: urn:request:956200fc-c198-4078-a123-21fdb4ec9d14 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:48,273 [518] [DEBUG] [app] Starting request: urn:request:956200fc-c198-4078-a123-21fdb4ec9d14 (/health/instance) 2020-03-31 09:32:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:48,281 [538] [DEBUG] [app] Starting request: urn:request:aca581ef-51ad-4bba-82ac-79e72f6d43cd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:48,281 [538] [DEBUG] [app] Starting request: urn:request:aca581ef-51ad-4bba-82ac-79e72f6d43cd (/v1/_internal_ping) 2020-03-31 09:32:48,282 [538] [DEBUG] [app] Ending request: urn:request:aca581ef-51ad-4bba-82ac-79e72f6d43cd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:48,282 [538] [DEBUG] [app] Ending request: urn:request:aca581ef-51ad-4bba-82ac-79e72f6d43cd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:48,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:32:48,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:48,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:32:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:32:48,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:48,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:48,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:48,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:48,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:48,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:48,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:48,312 [527] [DEBUG] [app] Starting request: urn:request:91272a06-68f7-458f-aa18-12cd5d4e94f6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:48,312 [527] [DEBUG] [app] Starting request: urn:request:91272a06-68f7-458f-aa18-12cd5d4e94f6 (/_internal_ping) 2020-03-31 09:32:48,313 [527] [DEBUG] [app] Ending request: urn:request:91272a06-68f7-458f-aa18-12cd5d4e94f6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:48,313 [527] [DEBUG] [app] Ending request: urn:request:91272a06-68f7-458f-aa18-12cd5d4e94f6 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:32:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:32:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:32:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:32:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:32:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:48,325 [537] [DEBUG] [app] Starting request: urn:request:aa14a033-17b8-412e-8678-9c09992d5bc0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:48,325 [537] [DEBUG] [app] Starting request: urn:request:aa14a033-17b8-412e-8678-9c09992d5bc0 (/c1/_internal_ping) 2020-03-31 09:32:48,325 [537] [DEBUG] [app] Ending request: urn:request:aa14a033-17b8-412e-8678-9c09992d5bc0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:48,325 [537] [DEBUG] [app] Ending request: urn:request:aa14a033-17b8-412e-8678-9c09992d5bc0 (/c1/_internal_ping) 2020-03-31 09:32:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:48,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:48,327 [518] [DEBUG] [app] Ending request: urn:request:956200fc-c198-4078-a123-21fdb4ec9d14 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:32:48,326 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:48,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:48,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:32:48,327 [518] [DEBUG] [app] Ending request: urn:request:956200fc-c198-4078-a123-21fdb4ec9d14 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:48,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:48,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:32:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:49 UTC)" (scheduled at 2020-03-31 09:32:49.498966+00:00) 2020-03-31 09:32:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:54.323352+00:00 (in 4.823368 seconds) notificationworker stdout | 2020-03-31 09:32:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:49 UTC)" (scheduled at 2020-03-31 09:32:49.498966+00:00) 2020-03-31 09:32:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:54.323352+00:00 (in 4.823368 seconds) 2020-03-31 09:32:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:49 UTC)" executed successfully 2020-03-31 09:32:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:32:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:59.560326+00:00 (in 8.448785 seconds) 2020-03-31 09:32:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:51 UTC)" (scheduled at 2020-03-31 09:32:51.110610+00:00) 2020-03-31 09:32:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:32:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:32:59.560326+00:00 (in 8.448785 seconds) 2020-03-31 09:32:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:51 UTC)" (scheduled at 2020-03-31 09:32:51.110610+00:00) 2020-03-31 09:32:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 51, 112425), True, datetime.datetime(2020, 3, 31, 9, 32, 51, 112425), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:32:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 51, 112425), True, datetime.datetime(2020, 3, 31, 9, 32, 51, 112425), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:32:51,131 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:32:51,131 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:32:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:51 UTC)" executed successfully 2020-03-31 09:32:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:32:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:01.440707+00:00 (in 9.998221 seconds) 2020-03-31 09:32:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:01 UTC)" (scheduled at 2020-03-31 09:32:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:32:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:01.440707+00:00 (in 9.998221 seconds) 2020-03-31 09:32:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:01 UTC)" (scheduled at 2020-03-31 09:32:51.440707+00:00) 2020-03-31 09:32:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:32:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:32:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:32:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:32:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:32:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:01 UTC)" executed successfully 2020-03-31 09:32:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:32:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:22.113221+00:00 (in 29.999326 seconds) 2020-03-31 09:32:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:22 UTC)" (scheduled at 2020-03-31 09:32:52.113221+00:00) securityworker stdout | 2020-03-31 09:32:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:22.113221+00:00 (in 29.999326 seconds) 2020-03-31 09:32:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:22 UTC)" (scheduled at 2020-03-31 09:32:52.113221+00:00) 2020-03-31 09:32:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:32:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:32:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:32:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:32:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:32:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:32:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:52,129 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:32:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:52,129 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:22 UTC)" executed successfully 2020-03-31 09:32:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:32:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:04.323352+00:00 (in 9.998302 seconds) 2020-03-31 09:32:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:04 UTC)" (scheduled at 2020-03-31 09:32:54.323352+00:00) 2020-03-31 09:32:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:32:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:04.323352+00:00 (in 9.998302 seconds) 2020-03-31 09:32:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:04 UTC)" (scheduled at 2020-03-31 09:32:54.323352+00:00) 2020-03-31 09:32:54,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:32:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 54, 326382), True, datetime.datetime(2020, 3, 31, 9, 32, 54, 326382), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:32:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 54, 326382), True, datetime.datetime(2020, 3, 31, 9, 32, 54, 326382), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:32:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:54,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:32:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:32:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:04 UTC)" executed successfully 2020-03-31 09:32:54,446 [527] [DEBUG] [app] Starting request: urn:request:d5f481ca-3c47-4905-bed1-a9068d1df2a1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:54,446 [527] [DEBUG] [app] Starting request: urn:request:d5f481ca-3c47-4905-bed1-a9068d1df2a1 (/health/instance) 2020-03-31 09:32:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:54,453 [555] [DEBUG] [app] Starting request: urn:request:e04c1e61-9672-4146-855f-c2a41cd44d4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:54,453 [555] [DEBUG] [app] Starting request: urn:request:e04c1e61-9672-4146-855f-c2a41cd44d4c (/v1/_internal_ping) 2020-03-31 09:32:54,454 [555] [DEBUG] [app] Ending request: urn:request:e04c1e61-9672-4146-855f-c2a41cd44d4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:54,454 [555] [DEBUG] [app] Ending request: urn:request:e04c1e61-9672-4146-855f-c2a41cd44d4c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:54,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:32:54,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:32:54,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:32:54,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:54,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:54,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:54,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:54,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:54,487 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:54,487 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:54,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:54,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:54,492 [526] [DEBUG] [app] Starting request: urn:request:507f525a-c716-4939-afba-939a8769008e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:54,492 [526] [DEBUG] [app] Starting request: urn:request:507f525a-c716-4939-afba-939a8769008e (/_internal_ping) 2020-03-31 09:32:54,493 [526] [DEBUG] [app] Ending request: urn:request:507f525a-c716-4939-afba-939a8769008e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:54,493 [526] [DEBUG] [app] Ending request: urn:request:507f525a-c716-4939-afba-939a8769008e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:32:54,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:32:54,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:32:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:54,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:54,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:32:54,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:54,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:54,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:54,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:54,507 [536] [DEBUG] [app] Starting request: urn:request:dc57a87d-74e3-4dcd-bb44-4b0a4221b623 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:54,507 [536] [DEBUG] [app] Starting request: urn:request:dc57a87d-74e3-4dcd-bb44-4b0a4221b623 (/c1/_internal_ping) 2020-03-31 09:32:54,508 [536] [DEBUG] [app] Ending request: urn:request:dc57a87d-74e3-4dcd-bb44-4b0a4221b623 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:54,508 [536] [DEBUG] [app] Ending request: urn:request:dc57a87d-74e3-4dcd-bb44-4b0a4221b623 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:32:54,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:54,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:54,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:32:54,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:54,511 [527] [DEBUG] [app] Ending request: urn:request:d5f481ca-3c47-4905-bed1-a9068d1df2a1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:54,511 [527] [DEBUG] [app] Ending request: urn:request:d5f481ca-3c47-4905-bed1-a9068d1df2a1 (/health/instance) 2020-03-31 09:32:54,511 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:32:54,511 [527] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:32:54,512 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:32:54,512 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:32:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:32:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:06.924497+00:00 (in 9.999115 seconds) storagereplication stdout | 2020-03-31 09:32:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:32:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:32:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:06.924497+00:00 (in 9.999115 seconds) 2020-03-31 09:32:58,272 [527] [DEBUG] [app] Starting request: urn:request:0b876fdb-0ad5-4163-af11-9bbbb3bbd6e5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:32:58,272 [527] [DEBUG] [app] Starting request: urn:request:0b876fdb-0ad5-4163-af11-9bbbb3bbd6e5 (/health/instance) 2020-03-31 09:32:58,275 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:58,275 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:58,277 [538] [DEBUG] [app] Starting request: urn:request:eda831f4-6d98-4762-913d-dfbcdeba073f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:58,277 [538] [DEBUG] [app] Starting request: urn:request:eda831f4-6d98-4762-913d-dfbcdeba073f (/v1/_internal_ping) 2020-03-31 09:32:58,279 [538] [DEBUG] [app] Ending request: urn:request:eda831f4-6d98-4762-913d-dfbcdeba073f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:32:58,279 [538] [DEBUG] [app] Ending request: urn:request:eda831f4-6d98-4762-913d-dfbcdeba073f (/v1/_internal_ping) 2020-03-31 09:32:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:58,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:32:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:58,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:32:58,279 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:32:58,280 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:32:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:32:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:32:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:32:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:32:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:32:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:32:58,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:32:58,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:32:58,302 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:32:58,302 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:32:58,306 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:58,306 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:58,307 [527] [DEBUG] [app] Starting request: urn:request:f6d9425a-0c17-4017-b372-811dd11e62c7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:58,307 [527] [DEBUG] [app] Starting request: urn:request:f6d9425a-0c17-4017-b372-811dd11e62c7 (/_internal_ping) 2020-03-31 09:32:58,308 [527] [DEBUG] [app] Ending request: urn:request:f6d9425a-0c17-4017-b372-811dd11e62c7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:32:58,308 [527] [DEBUG] [app] Ending request: urn:request:f6d9425a-0c17-4017-b372-811dd11e62c7 (/_internal_ping) 2020-03-31 09:32:58,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:32:58,309 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:58,309 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:32:58,309 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:32:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:32:58,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:32:58,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:32:58,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:58,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:32:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:32:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:32:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:32:58,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:32:58,322 [541] [DEBUG] [app] Starting request: urn:request:135a4a71-0e78-4282-b0a5-a5b835329803 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:58,322 [541] [DEBUG] [app] Starting request: urn:request:135a4a71-0e78-4282-b0a5-a5b835329803 (/c1/_internal_ping) 2020-03-31 09:32:58,323 [541] [DEBUG] [app] Ending request: urn:request:135a4a71-0e78-4282-b0a5-a5b835329803 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:32:58,323 [541] [DEBUG] [app] Ending request: urn:request:135a4a71-0e78-4282-b0a5-a5b835329803 (/c1/_internal_ping) 2020-03-31 09:32:58,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:32:58,324 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:32:58,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:32:58,324 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:32:58,325 [527] [DEBUG] [app] Ending request: urn:request:0b876fdb-0ad5-4163-af11-9bbbb3bbd6e5 (/health/instance) 2020-03-31 09:32:58,325 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:32:58,325 [527] [DEBUG] [app] Ending request: urn:request:0b876fdb-0ad5-4163-af11-9bbbb3bbd6e5 (/health/instance) 2020-03-31 09:32:58,325 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:32:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:32:58,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:32:58,326 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:32:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:32:59,456 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:32:59,456 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy 2020-03-31 09:32:59,457 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:32:59,457 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:32:59,457 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:32:59,457 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:32:59,466 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) storagereplication stdout | 2020-03-31 09:32:59,466 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) 2020-03-31 09:32:59,470 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 37, 59, 465830), True, u'bc7bff38-5bf4-4070-aec4-28ee90f350b5', 1]) storagereplication stdout | 2020-03-31 09:32:59,470 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 37, 59, 465830), True, u'bc7bff38-5bf4-4070-aec4-28ee90f350b5', 1]) 2020-03-31 09:32:59,477 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 59, 476312), True, datetime.datetime(2020, 3, 31, 9, 32, 59, 476312), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:32:59,477 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 32, 59, 476312), True, datetime.datetime(2020, 3, 31, 9, 32, 59, 476312), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:32:59,482 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:32:59,482 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:32:59,483 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:59,483 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:32:59,483 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:32:59,483 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:06 UTC)" executed successfully 2020-03-31 09:32:59,564 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:32:59,564 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:32:59,565 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:51.110610+00:00 (in 51.545743 seconds) 2020-03-31 09:32:59,565 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:59 UTC)" (scheduled at 2020-03-31 09:32:59.560326+00:00) 2020-03-31 09:32:59,565 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:59,566 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:32:59,565 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:51.110610+00:00 (in 51.545743 seconds) 2020-03-31 09:32:59,565 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:59 UTC)" (scheduled at 2020-03-31 09:32:59.560326+00:00) 2020-03-31 09:32:59,565 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:32:59,566 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:33:59 UTC)" executed successfully 2020-03-31 09:33:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:33:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:30.337699+00:00 (in 29.999134 seconds) 2020-03-31 09:33:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:30 UTC)" (scheduled at 2020-03-31 09:33:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:33:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:30.337699+00:00 (in 29.999134 seconds) 2020-03-31 09:33:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:30 UTC)" (scheduled at 2020-03-31 09:33:00.337699+00:00) 2020-03-31 09:33:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 33, 0, 339163), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:33:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 33, 0, 339163), False, 50, 1, 0]) 2020-03-31 09:33:00,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:33:00,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:33:00,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:00,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:33:00,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:00,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:30 UTC)" executed successfully 2020-03-31 09:33:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:33:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:11.440707+00:00 (in 9.999151 seconds) 2020-03-31 09:33:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:11 UTC)" (scheduled at 2020-03-31 09:33:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:33:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:11.440707+00:00 (in 9.999151 seconds) 2020-03-31 09:33:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:11 UTC)" (scheduled at 2020-03-31 09:33:01.440707+00:00) 2020-03-31 09:33:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:33:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:33:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:33:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:33:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:33:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:11 UTC)" executed successfully 2020-03-31 09:33:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:33:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:24.410927+00:00 (in 21.080112 seconds) 2020-03-31 09:33:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:33 UTC)" (scheduled at 2020-03-31 09:33:03.330112+00:00) 2020-03-31 09:33:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:33:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:24.410927+00:00 (in 21.080112 seconds) 2020-03-31 09:33:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:33 UTC)" (scheduled at 2020-03-31 09:33:03.330112+00:00) 2020-03-31 09:33:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 3, 331527), True, datetime.datetime(2020, 3, 31, 9, 33, 3, 331527), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:33:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 3, 331527), True, datetime.datetime(2020, 3, 31, 9, 33, 3, 331527), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:33:03,361 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:33:03,361 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:03,361 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:03,362 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:33:03,361 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:03,362 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:33 UTC)" executed successfully 2020-03-31 09:33:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:33:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:14.323352+00:00 (in 9.998354 seconds) 2020-03-31 09:33:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:14 UTC)" (scheduled at 2020-03-31 09:33:04.323352+00:00) notificationworker stdout | 2020-03-31 09:33:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:14.323352+00:00 (in 9.998354 seconds) 2020-03-31 09:33:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:14 UTC)" (scheduled at 2020-03-31 09:33:04.323352+00:00) 2020-03-31 09:33:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:33:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 4, 325974), True, datetime.datetime(2020, 3, 31, 9, 33, 4, 325974), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:33:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 4, 325974), True, datetime.datetime(2020, 3, 31, 9, 33, 4, 325974), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:33:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:33:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:33:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:14 UTC)" executed successfully 2020-03-31 09:33:04,446 [526] [DEBUG] [app] Starting request: urn:request:ac330c6a-c4d2-4327-9fef-3d07b71d243f (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:04,446 [526] [DEBUG] [app] Starting request: urn:request:ac330c6a-c4d2-4327-9fef-3d07b71d243f (/health/instance) 2020-03-31 09:33:04,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:04,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:04,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:04,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:04,453 [555] [DEBUG] [app] Starting request: urn:request:0b1881a3-ef32-4ab9-986e-79fa03116cc7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:04,453 [555] [DEBUG] [app] Starting request: urn:request:0b1881a3-ef32-4ab9-986e-79fa03116cc7 (/v1/_internal_ping) 2020-03-31 09:33:04,454 [555] [DEBUG] [app] Ending request: urn:request:0b1881a3-ef32-4ab9-986e-79fa03116cc7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:04,454 [555] [DEBUG] [app] Ending request: urn:request:0b1881a3-ef32-4ab9-986e-79fa03116cc7 (/v1/_internal_ping) 2020-03-31 09:33:04,454 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:33:04,454 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:04,454 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:04,455 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:04,454 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:04,456 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:33:04,455 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:33:04,456 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:04,458 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:04,458 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:04,459 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:04,459 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:04,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:04,472 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:04,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:04,487 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:04,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:04,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:04,492 [527] [DEBUG] [app] Starting request: urn:request:9fa52652-c096-4e85-82a1-0a2704c8a526 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:04,492 [527] [DEBUG] [app] Starting request: urn:request:9fa52652-c096-4e85-82a1-0a2704c8a526 (/_internal_ping) 2020-03-31 09:33:04,493 [527] [DEBUG] [app] Ending request: urn:request:9fa52652-c096-4e85-82a1-0a2704c8a526 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:04,493 [527] [DEBUG] [app] Ending request: urn:request:9fa52652-c096-4e85-82a1-0a2704c8a526 (/_internal_ping) 2020-03-31 09:33:04,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:04,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:04,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:04,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:04,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:04,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:04,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:04,502 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:04,506 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:04,506 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:04,508 [537] [DEBUG] [app] Starting request: urn:request:2d8bf45c-0db9-455f-b016-d7fa6bfa44e0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:04,508 [537] [DEBUG] [app] Starting request: urn:request:2d8bf45c-0db9-455f-b016-d7fa6bfa44e0 (/c1/_internal_ping) 2020-03-31 09:33:04,508 [537] [DEBUG] [app] Ending request: urn:request:2d8bf45c-0db9-455f-b016-d7fa6bfa44e0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:04,508 [537] [DEBUG] [app] Ending request: urn:request:2d8bf45c-0db9-455f-b016-d7fa6bfa44e0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:04,509 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:04,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:33:04,509 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:04,509 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:04,511 [526] [DEBUG] [app] Ending request: urn:request:ac330c6a-c4d2-4327-9fef-3d07b71d243f (/health/instance) 2020-03-31 09:33:04,511 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:04,511 [526] [DEBUG] [app] Ending request: urn:request:ac330c6a-c4d2-4327-9fef-3d07b71d243f (/health/instance) 2020-03-31 09:33:04,511 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:04,511 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:33:04,511 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:33:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:33:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:16.924497+00:00 (in 9.999068 seconds) 2020-03-31 09:33:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:16 UTC)" (scheduled at 2020-03-31 09:33:06.924497+00:00) 2020-03-31 09:33:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:33:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:16.924497+00:00 (in 9.999068 seconds) 2020-03-31 09:33:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:16 UTC)" (scheduled at 2020-03-31 09:33:06.924497+00:00) 2020-03-31 09:33:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 6, 926289), True, datetime.datetime(2020, 3, 31, 9, 33, 6, 926289), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:33:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 6, 926289), True, datetime.datetime(2020, 3, 31, 9, 33, 6, 926289), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:33:06,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:33:06,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:33:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:16 UTC)" executed successfully 2020-03-31 09:33:08,273 [518] [DEBUG] [app] Starting request: urn:request:3a04a425-0805-420f-bc66-5bbbc032436b (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:08,273 [518] [DEBUG] [app] Starting request: urn:request:3a04a425-0805-420f-bc66-5bbbc032436b (/health/instance) 2020-03-31 09:33:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:08,281 [547] [DEBUG] [app] Starting request: urn:request:89628740-53b0-41a1-a3e2-514bd04079bf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:08,281 [547] [DEBUG] [app] Starting request: urn:request:89628740-53b0-41a1-a3e2-514bd04079bf (/v1/_internal_ping) 2020-03-31 09:33:08,282 [547] [DEBUG] [app] Ending request: urn:request:89628740-53b0-41a1-a3e2-514bd04079bf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:08,282 [547] [DEBUG] [app] Ending request: urn:request:89628740-53b0-41a1-a3e2-514bd04079bf (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:08,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:33:08,283 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:33:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:08,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:08,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:08,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:08,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:08,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:08,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:08,315 [525] [DEBUG] [app] Starting request: urn:request:dadb785c-1201-4c43-b83b-c5454ac3ee7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:08,315 [525] [DEBUG] [app] Starting request: urn:request:dadb785c-1201-4c43-b83b-c5454ac3ee7b (/_internal_ping) 2020-03-31 09:33:08,316 [525] [DEBUG] [app] Ending request: urn:request:dadb785c-1201-4c43-b83b-c5454ac3ee7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:08,316 [525] [DEBUG] [app] Ending request: urn:request:dadb785c-1201-4c43-b83b-c5454ac3ee7b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:33:08,316 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:33:08,316 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:33:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:33:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:33:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:08,329 [543] [DEBUG] [app] Starting request: urn:request:ebde8e48-6052-467c-aa18-16af603fdef3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:08,329 [543] [DEBUG] [app] Starting request: urn:request:ebde8e48-6052-467c-aa18-16af603fdef3 (/c1/_internal_ping) 2020-03-31 09:33:08,329 [543] [DEBUG] [app] Ending request: urn:request:ebde8e48-6052-467c-aa18-16af603fdef3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:08,329 [543] [DEBUG] [app] Ending request: urn:request:ebde8e48-6052-467c-aa18-16af603fdef3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:08,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:33:08,330 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:08,331 [518] [DEBUG] [app] Ending request: urn:request:3a04a425-0805-420f-bc66-5bbbc032436b (/health/instance) 2020-03-31 09:33:08,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:08,331 [518] [DEBUG] [app] Ending request: urn:request:3a04a425-0805-420f-bc66-5bbbc032436b (/health/instance) 2020-03-31 09:33:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:33:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:33:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:33:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:33:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:21.440707+00:00 (in 9.999275 seconds) 2020-03-31 09:33:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:21 UTC)" (scheduled at 2020-03-31 09:33:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:33:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:21.440707+00:00 (in 9.999275 seconds) 2020-03-31 09:33:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:21 UTC)" (scheduled at 2020-03-31 09:33:11.440707+00:00) 2020-03-31 09:33:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:33:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:33:11,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:33:11,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:33:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:33:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:21 UTC)" executed successfully 2020-03-31 09:33:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:33:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:24.323352+00:00 (in 9.998746 seconds) 2020-03-31 09:33:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:24 UTC)" (scheduled at 2020-03-31 09:33:14.323352+00:00) 2020-03-31 09:33:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:33:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:24.323352+00:00 (in 9.998746 seconds) 2020-03-31 09:33:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:24 UTC)" (scheduled at 2020-03-31 09:33:14.323352+00:00) 2020-03-31 09:33:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 14, 325580), True, datetime.datetime(2020, 3, 31, 9, 33, 14, 325580), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:33:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 14, 325580), True, datetime.datetime(2020, 3, 31, 9, 33, 14, 325580), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:33:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:33:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:33:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:24 UTC)" executed successfully 2020-03-31 09:33:14,446 [527] [DEBUG] [app] Starting request: urn:request:154e88b7-0b67-4361-890f-449cfe03706d (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:14,446 [527] [DEBUG] [app] Starting request: urn:request:154e88b7-0b67-4361-890f-449cfe03706d (/health/instance) 2020-03-31 09:33:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:14,454 [550] [DEBUG] [app] Starting request: urn:request:23a47722-695c-4921-b41b-d1230ce6d3ab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:14,454 [550] [DEBUG] [app] Starting request: urn:request:23a47722-695c-4921-b41b-d1230ce6d3ab (/v1/_internal_ping) 2020-03-31 09:33:14,455 [550] [DEBUG] [app] Ending request: urn:request:23a47722-695c-4921-b41b-d1230ce6d3ab (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:14,455 [550] [DEBUG] [app] Ending request: urn:request:23a47722-695c-4921-b41b-d1230ce6d3ab (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:14,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:33:14,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:14,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:33:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:33:14,459 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:14,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:14,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:14,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:14,461 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:14,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:14,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:14,487 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:14,487 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:14,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:14,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:14,493 [525] [DEBUG] [app] Starting request: urn:request:e6049422-94e0-4fbe-932d-dedb2e1b5fa0 (/_internal_ping) 2020-03-31 09:33:14,494 [525] [DEBUG] [app] Ending request: urn:request:e6049422-94e0-4fbe-932d-dedb2e1b5fa0 (/_internal_ping) 2020-03-31 09:33:14,494 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:33:14,493 [525] [DEBUG] [app] Starting request: urn:request:e6049422-94e0-4fbe-932d-dedb2e1b5fa0 (/_internal_ping) 2020-03-31 09:33:14,494 [525] [DEBUG] [app] Ending request: urn:request:e6049422-94e0-4fbe-932d-dedb2e1b5fa0 (/_internal_ping) 2020-03-31 09:33:14,494 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:33:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:14,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:14,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:14,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:14,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:33:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:33:14,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:14,509 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:14,510 [537] [DEBUG] [app] Starting request: urn:request:5921bb10-c0fe-41da-a6e1-9afa29d4472b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:14,510 [537] [DEBUG] [app] Starting request: urn:request:5921bb10-c0fe-41da-a6e1-9afa29d4472b (/c1/_internal_ping) 2020-03-31 09:33:14,511 [537] [DEBUG] [app] Ending request: urn:request:5921bb10-c0fe-41da-a6e1-9afa29d4472b (/c1/_internal_ping) 2020-03-31 09:33:14,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:33:14,511 [537] [DEBUG] [app] Ending request: urn:request:5921bb10-c0fe-41da-a6e1-9afa29d4472b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:33:14,512 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:14,514 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:33:14,514 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:14,516 [527] [DEBUG] [app] Ending request: urn:request:154e88b7-0b67-4361-890f-449cfe03706d (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:14,516 [527] [DEBUG] [app] Ending request: urn:request:154e88b7-0b67-4361-890f-449cfe03706d (/health/instance) 2020-03-31 09:33:14,516 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:14,516 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:14,516 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 09:33:14,516 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:33:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:33:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:45.064514+00:00 (in 29.999211 seconds) 2020-03-31 09:33:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:45 UTC)" (scheduled at 2020-03-31 09:33:15.064514+00:00) gcworker stdout | 2020-03-31 09:33:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:45.064514+00:00 (in 29.999211 seconds) 2020-03-31 09:33:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:45 UTC)" (scheduled at 2020-03-31 09:33:15.064514+00:00) 2020-03-31 09:33:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437595, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:33:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437595, 1209600, 500, 1, 0]) 2020-03-31 09:33:15,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:33:15,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:33:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:33:15,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:15,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:45 UTC)" executed successfully 2020-03-31 09:33:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:26 UTC)" (scheduled at 2020-03-31 09:33:16.924497+00:00) 2020-03-31 09:33:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 16, 925948), True, datetime.datetime(2020, 3, 31, 9, 33, 16, 925948), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:33:16,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:26.924497+00:00 (in 9.999146 seconds) storagereplication stdout | 2020-03-31 09:33:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:33:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:26 UTC)" (scheduled at 2020-03-31 09:33:16.924497+00:00) 2020-03-31 09:33:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:33:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 16, 925948), True, datetime.datetime(2020, 3, 31, 9, 33, 16, 925948), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:33:16,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:26.924497+00:00 (in 9.999146 seconds) 2020-03-31 09:33:16,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:33:16,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:16,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:33:16,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:16,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:26 UTC)" executed successfully 2020-03-31 09:33:18,273 [527] [DEBUG] [app] Starting request: urn:request:33aca5db-d15c-4f32-b568-753913cdba07 (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:18,273 [527] [DEBUG] [app] Starting request: urn:request:33aca5db-d15c-4f32-b568-753913cdba07 (/health/instance) 2020-03-31 09:33:18,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:18,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:18,278 [549] [DEBUG] [app] Starting request: urn:request:cd8e3b9f-312f-4721-99f1-465fb088396d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:18,278 [549] [DEBUG] [app] Starting request: urn:request:cd8e3b9f-312f-4721-99f1-465fb088396d (/v1/_internal_ping) 2020-03-31 09:33:18,279 [549] [DEBUG] [app] Ending request: urn:request:cd8e3b9f-312f-4721-99f1-465fb088396d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:18,279 [549] [DEBUG] [app] Ending request: urn:request:cd8e3b9f-312f-4721-99f1-465fb088396d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:18,280 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:18,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:33:18,280 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:18,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:18,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:18,282 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:33:18,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:33:18,282 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:18,283 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:18,283 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:18,284 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:18,284 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:18,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:18,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:18,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:18,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:18,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:18,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:18,314 [518] [DEBUG] [app] Starting request: urn:request:e45e3bfa-36ac-46d8-87b0-b40404137285 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:18,314 [518] [DEBUG] [app] Starting request: urn:request:e45e3bfa-36ac-46d8-87b0-b40404137285 (/_internal_ping) 2020-03-31 09:33:18,315 [518] [DEBUG] [app] Ending request: urn:request:e45e3bfa-36ac-46d8-87b0-b40404137285 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:18,315 [518] [DEBUG] [app] Ending request: urn:request:e45e3bfa-36ac-46d8-87b0-b40404137285 (/_internal_ping) 2020-03-31 09:33:18,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:18,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:33:18,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:18,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:33:18,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:18,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:18,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:33:18,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:33:18,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:18,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:18,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:33:18,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:33:18,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:18,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:18,328 [541] [DEBUG] [app] Starting request: urn:request:22c0fb10-19dd-474e-83e2-0137bb1b603c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:18,328 [541] [DEBUG] [app] Starting request: urn:request:22c0fb10-19dd-474e-83e2-0137bb1b603c (/c1/_internal_ping) 2020-03-31 09:33:18,329 [541] [DEBUG] [app] Ending request: urn:request:22c0fb10-19dd-474e-83e2-0137bb1b603c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:18,329 [541] [DEBUG] [app] Ending request: urn:request:22c0fb10-19dd-474e-83e2-0137bb1b603c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:18,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:18,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:18,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:33:18,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:18,331 [527] [DEBUG] [app] Ending request: urn:request:33aca5db-d15c-4f32-b568-753913cdba07 (/health/instance) 2020-03-31 09:33:18,331 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:18,331 [527] [DEBUG] [app] Ending request: urn:request:33aca5db-d15c-4f32-b568-753913cdba07 (/health/instance) 2020-03-31 09:33:18,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:18,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:33:18,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:33:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:33:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:31.440707+00:00 (in 9.998945 seconds) 2020-03-31 09:33:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:31 UTC)" (scheduled at 2020-03-31 09:33:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:33:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:31.440707+00:00 (in 9.998945 seconds) 2020-03-31 09:33:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:31 UTC)" (scheduled at 2020-03-31 09:33:21.440707+00:00) 2020-03-31 09:33:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:33:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:33:21,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:33:21,458 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:33:21,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:33:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:33:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:31 UTC)" executed successfully 2020-03-31 09:33:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:33:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:52.113221+00:00 (in 29.999237 seconds) 2020-03-31 09:33:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:52 UTC)" (scheduled at 2020-03-31 09:33:22.113221+00:00) securityworker stdout | 2020-03-31 09:33:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:52.113221+00:00 (in 29.999237 seconds) 2020-03-31 09:33:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:52 UTC)" (scheduled at 2020-03-31 09:33:22.113221+00:00) 2020-03-31 09:33:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:33:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:33:22,126 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:33:22,126 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:33:22,128 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:33:22,128 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:33:22,128 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:22,128 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:33:22,128 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:22,128 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:33:52 UTC)" executed successfully 2020-03-31 09:33:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:33:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:34.323352+00:00 (in 9.998965 seconds) 2020-03-31 09:33:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:34 UTC)" (scheduled at 2020-03-31 09:33:24.323352+00:00) 2020-03-31 09:33:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:33:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:34.323352+00:00 (in 9.998965 seconds) 2020-03-31 09:33:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:34 UTC)" (scheduled at 2020-03-31 09:33:24.323352+00:00) 2020-03-31 09:33:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 24, 325318), True, datetime.datetime(2020, 3, 31, 9, 33, 24, 325318), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:33:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 24, 325318), True, datetime.datetime(2020, 3, 31, 9, 33, 24, 325318), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:33:24,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:33:24,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:24,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:24,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:33:24,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:24,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:34 UTC)" executed successfully 2020-03-31 09:33:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:33:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:33.330112+00:00 (in 8.918491 seconds) 2020-03-31 09:33:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:24 UTC)" (scheduled at 2020-03-31 09:33:24.410927+00:00) 2020-03-31 09:33:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:33:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:33.330112+00:00 (in 8.918491 seconds) 2020-03-31 09:33:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:24 UTC)" (scheduled at 2020-03-31 09:33:24.410927+00:00) 2020-03-31 09:33:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:24 UTC)" executed successfully 2020-03-31 09:33:24,446 [518] [DEBUG] [app] Starting request: urn:request:2038bbe8-ae88-4262-8333-884d73fbee49 (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:24,446 [518] [DEBUG] [app] Starting request: urn:request:2038bbe8-ae88-4262-8333-884d73fbee49 (/health/instance) 2020-03-31 09:33:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:24,454 [548] [DEBUG] [app] Starting request: urn:request:bf778cac-0ecf-459b-93ec-18d1ba6f2280 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:24,454 [548] [DEBUG] [app] Starting request: urn:request:bf778cac-0ecf-459b-93ec-18d1ba6f2280 (/v1/_internal_ping) 2020-03-31 09:33:24,456 [548] [DEBUG] [app] Ending request: urn:request:bf778cac-0ecf-459b-93ec-18d1ba6f2280 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:24,456 [548] [DEBUG] [app] Ending request: urn:request:bf778cac-0ecf-459b-93ec-18d1ba6f2280 (/v1/_internal_ping) 2020-03-31 09:33:24,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:33:24,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:33:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:24,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:24,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:24,461 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:24,461 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:24,461 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:24,461 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:24,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:24,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:24,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:24,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:24,483 [518] [DEBUG] [app] Starting request: urn:request:74ae2b67-f554-4074-8f5f-d711ebff328d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:24,483 [518] [DEBUG] [app] Starting request: urn:request:74ae2b67-f554-4074-8f5f-d711ebff328d (/_internal_ping) 2020-03-31 09:33:24,484 [518] [DEBUG] [app] Ending request: urn:request:74ae2b67-f554-4074-8f5f-d711ebff328d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:24,484 [518] [DEBUG] [app] Ending request: urn:request:74ae2b67-f554-4074-8f5f-d711ebff328d (/_internal_ping) 2020-03-31 09:33:24,484 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:33:24,484 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:24,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:33:24,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:24,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:24,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:24,496 [537] [DEBUG] [app] Starting request: urn:request:16c3c01c-0c74-42e5-ac07-0804aacac015 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:24,496 [537] [DEBUG] [app] Starting request: urn:request:16c3c01c-0c74-42e5-ac07-0804aacac015 (/c1/_internal_ping) 2020-03-31 09:33:24,497 [537] [DEBUG] [app] Ending request: urn:request:16c3c01c-0c74-42e5-ac07-0804aacac015 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:24,497 [537] [DEBUG] [app] Ending request: urn:request:16c3c01c-0c74-42e5-ac07-0804aacac015 (/c1/_internal_ping) 2020-03-31 09:33:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:24,498 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:24,499 [518] [DEBUG] [app] Ending request: urn:request:2038bbe8-ae88-4262-8333-884d73fbee49 (/health/instance) 2020-03-31 09:33:24,499 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:33:24,498 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:24,500 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) gunicorn-web stdout | 2020-03-31 09:33:24,499 [518] [DEBUG] [app] Ending request: urn:request:2038bbe8-ae88-4262-8333-884d73fbee49 (/health/instance) 2020-03-31 09:33:24,499 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:24,500 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) 2020-03-31 09:33:26,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:33:26,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:26,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:26 UTC)" (scheduled at 2020-03-31 09:33:26.924497+00:00) 2020-03-31 09:33:26,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:33:26,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:26 UTC)" (scheduled at 2020-03-31 09:33:26.924497+00:00) 2020-03-31 09:33:26,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:26,930 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:36.924497+00:00 (in 9.996563 seconds) storagereplication stdout | 2020-03-31 09:33:26,930 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:36.924497+00:00 (in 9.996563 seconds) 2020-03-31 09:33:26,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 26, 928662), True, datetime.datetime(2020, 3, 31, 9, 33, 26, 928662), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:33:26,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 26, 928662), True, datetime.datetime(2020, 3, 31, 9, 33, 26, 928662), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:33:26,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:33:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:33:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:36 UTC)" executed successfully 2020-03-31 09:33:28,273 [518] [DEBUG] [app] Starting request: urn:request:006dbe26-a0f1-44ac-bf77-22a6b1d9277a (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:28,273 [518] [DEBUG] [app] Starting request: urn:request:006dbe26-a0f1-44ac-bf77-22a6b1d9277a (/health/instance) 2020-03-31 09:33:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:28,278 [555] [DEBUG] [app] Starting request: urn:request:0b6bd566-d82a-45b9-a15e-aaede201f008 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:28,278 [555] [DEBUG] [app] Starting request: urn:request:0b6bd566-d82a-45b9-a15e-aaede201f008 (/v1/_internal_ping) 2020-03-31 09:33:28,279 [555] [DEBUG] [app] Ending request: urn:request:0b6bd566-d82a-45b9-a15e-aaede201f008 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:28,279 [555] [DEBUG] [app] Ending request: urn:request:0b6bd566-d82a-45b9-a15e-aaede201f008 (/v1/_internal_ping) 2020-03-31 09:33:28,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:33:28,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:33:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:28,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:28,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:28,285 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:28,285 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:28,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:28,286 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:28,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:28,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:28,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:28,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:28,310 [525] [DEBUG] [app] Starting request: urn:request:a50040d4-9e86-44d5-a953-09c8623a91fe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:28,310 [525] [DEBUG] [app] Starting request: urn:request:a50040d4-9e86-44d5-a953-09c8623a91fe (/_internal_ping) 2020-03-31 09:33:28,311 [525] [DEBUG] [app] Ending request: urn:request:a50040d4-9e86-44d5-a953-09c8623a91fe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:28,311 [525] [DEBUG] [app] Ending request: urn:request:a50040d4-9e86-44d5-a953-09c8623a91fe (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:33:28,312 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:33:28,312 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:33:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:28,324 [541] [DEBUG] [app] Starting request: urn:request:d5e50bf2-cbae-4afe-aad4-da9eb99b4746 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:28,324 [541] [DEBUG] [app] Starting request: urn:request:d5e50bf2-cbae-4afe-aad4-da9eb99b4746 (/c1/_internal_ping) 2020-03-31 09:33:28,325 [541] [DEBUG] [app] Ending request: urn:request:d5e50bf2-cbae-4afe-aad4-da9eb99b4746 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:28,325 [541] [DEBUG] [app] Ending request: urn:request:d5e50bf2-cbae-4afe-aad4-da9eb99b4746 (/c1/_internal_ping) 2020-03-31 09:33:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:28,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:33:28,326 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:28,328 [518] [DEBUG] [app] Ending request: urn:request:006dbe26-a0f1-44ac-bf77-22a6b1d9277a (/health/instance) 2020-03-31 09:33:28,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:28,328 [518] [DEBUG] [app] Ending request: urn:request:006dbe26-a0f1-44ac-bf77-22a6b1d9277a (/health/instance) 2020-03-31 09:33:28,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:28,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:33:28,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:33:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:33:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:00.337699+00:00 (in 29.999135 seconds) 2020-03-31 09:33:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:00 UTC)" (scheduled at 2020-03-31 09:33:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:33:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:00.337699+00:00 (in 29.999135 seconds) 2020-03-31 09:33:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:00 UTC)" (scheduled at 2020-03-31 09:33:30.337699+00:00) 2020-03-31 09:33:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 33, 30, 339392), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:33:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 33, 30, 339392), False, 50, 1, 0]) 2020-03-31 09:33:30,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:33:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:33:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:33:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:00 UTC)" executed successfully 2020-03-31 09:33:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:33:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:41.440707+00:00 (in 9.999020 seconds) 2020-03-31 09:33:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:41 UTC)" (scheduled at 2020-03-31 09:33:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:33:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:41.440707+00:00 (in 9.999020 seconds) 2020-03-31 09:33:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:41 UTC)" (scheduled at 2020-03-31 09:33:31.440707+00:00) 2020-03-31 09:33:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:33:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:33:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:33:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:33:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:33:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:41 UTC)" executed successfully 2020-03-31 09:33:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:33:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:03.330112+00:00 (in 29.999225 seconds) 2020-03-31 09:33:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:03 UTC)" (scheduled at 2020-03-31 09:33:33.330112+00:00) security_notification_worker stdout | 2020-03-31 09:33:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:03.330112+00:00 (in 29.999225 seconds) 2020-03-31 09:33:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:03 UTC)" (scheduled at 2020-03-31 09:33:33.330112+00:00) 2020-03-31 09:33:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:33:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 33, 331877), True, datetime.datetime(2020, 3, 31, 9, 33, 33, 331877), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:33:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 33, 331877), True, datetime.datetime(2020, 3, 31, 9, 33, 33, 331877), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:33:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:33:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:33:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:03 UTC)" executed successfully 2020-03-31 09:33:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:33:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:43.539999+00:00 (in 10.172294 seconds) 2020-03-31 09:33:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:33 UTC)" (scheduled at 2020-03-31 09:33:33.366791+00:00) 2020-03-31 09:33:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:33:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:43.539999+00:00 (in 10.172294 seconds) 2020-03-31 09:33:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:33 UTC)" (scheduled at 2020-03-31 09:33:33.366791+00:00) 2020-03-31 09:33:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 33, 368424), True, datetime.datetime(2020, 3, 31, 9, 33, 33, 368424), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:33:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 33, 368424), True, datetime.datetime(2020, 3, 31, 9, 33, 33, 368424), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:33:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:33:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:33:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:33 UTC)" executed successfully 2020-03-31 09:33:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:33:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:44.323352+00:00 (in 9.999136 seconds) 2020-03-31 09:33:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:44 UTC)" (scheduled at 2020-03-31 09:33:34.323352+00:00) 2020-03-31 09:33:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:33:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:44.323352+00:00 (in 9.999136 seconds) 2020-03-31 09:33:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:44 UTC)" (scheduled at 2020-03-31 09:33:34.323352+00:00) 2020-03-31 09:33:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 34, 325165), True, datetime.datetime(2020, 3, 31, 9, 33, 34, 325165), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:33:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 34, 325165), True, datetime.datetime(2020, 3, 31, 9, 33, 34, 325165), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:33:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:33:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:33:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:44 UTC)" executed successfully 2020-03-31 09:33:34,446 [518] [DEBUG] [app] Starting request: urn:request:f6087930-cd0e-4f0c-9b0f-646f76d01a32 (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:34,446 [518] [DEBUG] [app] Starting request: urn:request:f6087930-cd0e-4f0c-9b0f-646f76d01a32 (/health/instance) 2020-03-31 09:33:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:34,454 [549] [DEBUG] [app] Starting request: urn:request:8c858567-9488-4803-8424-aef6c42fcce8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:34,454 [549] [DEBUG] [app] Starting request: urn:request:8c858567-9488-4803-8424-aef6c42fcce8 (/v1/_internal_ping) 2020-03-31 09:33:34,455 [549] [DEBUG] [app] Ending request: urn:request:8c858567-9488-4803-8424-aef6c42fcce8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:34,455 [549] [DEBUG] [app] Ending request: urn:request:8c858567-9488-4803-8424-aef6c42fcce8 (/v1/_internal_ping) 2020-03-31 09:33:34,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:33:34,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:33:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:34,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:34,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:34,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:34,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:34,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:34,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:34,485 [527] [DEBUG] [app] Starting request: urn:request:01004317-d71f-4de3-88a8-95f2d9edf16b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:34,485 [527] [DEBUG] [app] Starting request: urn:request:01004317-d71f-4de3-88a8-95f2d9edf16b (/_internal_ping) 2020-03-31 09:33:34,486 [527] [DEBUG] [app] Ending request: urn:request:01004317-d71f-4de3-88a8-95f2d9edf16b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:34,486 [527] [DEBUG] [app] Ending request: urn:request:01004317-d71f-4de3-88a8-95f2d9edf16b (/_internal_ping) 2020-03-31 09:33:34,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:34,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:33:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:34,501 [541] [DEBUG] [app] Starting request: urn:request:6258d305-5015-49b1-b3e1-1cd057eb372a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:34,501 [541] [DEBUG] [app] Starting request: urn:request:6258d305-5015-49b1-b3e1-1cd057eb372a (/c1/_internal_ping) 2020-03-31 09:33:34,502 [541] [DEBUG] [app] Ending request: urn:request:6258d305-5015-49b1-b3e1-1cd057eb372a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:34,502 [541] [DEBUG] [app] Ending request: urn:request:6258d305-5015-49b1-b3e1-1cd057eb372a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:34,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:33:34,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:34,505 [518] [DEBUG] [app] Ending request: urn:request:f6087930-cd0e-4f0c-9b0f-646f76d01a32 (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:34,505 [518] [DEBUG] [app] Ending request: urn:request:f6087930-cd0e-4f0c-9b0f-646f76d01a32 (/health/instance) 2020-03-31 09:33:34,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:33:34,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:33:36,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:33:36,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:36,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:38.010934+00:00 (in 1.083567 seconds) 2020-03-31 09:33:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:46 UTC)" (scheduled at 2020-03-31 09:33:36.924497+00:00) storagereplication stdout | 2020-03-31 09:33:36,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:38.010934+00:00 (in 1.083567 seconds) 2020-03-31 09:33:36,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:46 UTC)" (scheduled at 2020-03-31 09:33:36.924497+00:00) 2020-03-31 09:33:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:33:36,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 36, 928513), True, datetime.datetime(2020, 3, 31, 9, 33, 36, 928513), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:33:36,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 36, 928513), True, datetime.datetime(2020, 3, 31, 9, 33, 36, 928513), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:33:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:33:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:33:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:46 UTC)" executed successfully 2020-03-31 09:33:38,015 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:33:38,015 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:38,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:46.924497+00:00 (in 8.909189 seconds) 2020-03-31 09:33:38,015 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:38 UTC)" (scheduled at 2020-03-31 09:33:38.010934+00:00) 2020-03-31 09:33:38,015 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:38,016 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:33:38,015 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:46.924497+00:00 (in 8.909189 seconds) 2020-03-31 09:33:38,015 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:38 UTC)" (scheduled at 2020-03-31 09:33:38.010934+00:00) 2020-03-31 09:33:38,015 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:38,016 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:38 UTC)" executed successfully 2020-03-31 09:33:38,273 [527] [DEBUG] [app] Starting request: urn:request:ac25089e-a16f-42a8-bcce-fb13f7ba91fa (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:38,273 [527] [DEBUG] [app] Starting request: urn:request:ac25089e-a16f-42a8-bcce-fb13f7ba91fa (/health/instance) 2020-03-31 09:33:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:38,281 [555] [DEBUG] [app] Starting request: urn:request:c1f285ee-9765-454d-b49e-c348520a06ec (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:38,281 [555] [DEBUG] [app] Starting request: urn:request:c1f285ee-9765-454d-b49e-c348520a06ec (/v1/_internal_ping) 2020-03-31 09:33:38,282 [555] [DEBUG] [app] Ending request: urn:request:c1f285ee-9765-454d-b49e-c348520a06ec (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:38,282 [555] [DEBUG] [app] Ending request: urn:request:c1f285ee-9765-454d-b49e-c348520a06ec (/v1/_internal_ping) 2020-03-31 09:33:38,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:33:38,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:38,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:33:38,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:38,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:38,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:33:38,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:38,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:33:38,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:38,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:38,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:38,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:38,305 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:38,305 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:38,319 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:38,319 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:38,324 [527] [DEBUG] [app] Starting request: urn:request:1c379911-37a4-4c51-b1e0-244e5d91ebfe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:38,324 [527] [DEBUG] [app] Starting request: urn:request:1c379911-37a4-4c51-b1e0-244e5d91ebfe (/_internal_ping) 2020-03-31 09:33:38,326 [527] [DEBUG] [app] Ending request: urn:request:1c379911-37a4-4c51-b1e0-244e5d91ebfe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:38,326 [527] [DEBUG] [app] Ending request: urn:request:1c379911-37a4-4c51-b1e0-244e5d91ebfe (/_internal_ping) 2020-03-31 09:33:38,326 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:38,326 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:38,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:38,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:38,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:38,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:38,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:38,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:38,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:38,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:38,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:38,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:38,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:38,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:38,341 [536] [DEBUG] [app] Starting request: urn:request:0277c432-5ea4-490c-b8df-5f622e981860 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:38,341 [536] [DEBUG] [app] Starting request: urn:request:0277c432-5ea4-490c-b8df-5f622e981860 (/c1/_internal_ping) 2020-03-31 09:33:38,342 [536] [DEBUG] [app] Ending request: urn:request:0277c432-5ea4-490c-b8df-5f622e981860 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:38,342 [536] [DEBUG] [app] Ending request: urn:request:0277c432-5ea4-490c-b8df-5f622e981860 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:38,343 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:38,343 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:33:38,343 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:38,343 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:38,344 [527] [DEBUG] [app] Ending request: urn:request:ac25089e-a16f-42a8-bcce-fb13f7ba91fa (/health/instance) 2020-03-31 09:33:38,344 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:38,344 [527] [DEBUG] [app] Ending request: urn:request:ac25089e-a16f-42a8-bcce-fb13f7ba91fa (/health/instance) 2020-03-31 09:33:38,344 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:38,345 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:33:38,345 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:33:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:33:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:51.440707+00:00 (in 9.998899 seconds) 2020-03-31 09:33:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:51 UTC)" (scheduled at 2020-03-31 09:33:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:33:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:51.440707+00:00 (in 9.998899 seconds) 2020-03-31 09:33:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:51 UTC)" (scheduled at 2020-03-31 09:33:41.440707+00:00) 2020-03-31 09:33:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:33:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:33:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:33:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:33:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:33:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:51 UTC)" executed successfully 2020-03-31 09:33:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:33:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:33.366791+00:00 (in 49.825986 seconds) 2020-03-31 09:33:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:43 UTC)" (scheduled at 2020-03-31 09:33:43.539999+00:00) 2020-03-31 09:33:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:33:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:33.366791+00:00 (in 49.825986 seconds) 2020-03-31 09:33:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:43 UTC)" (scheduled at 2020-03-31 09:33:43.539999+00:00) 2020-03-31 09:33:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:33:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:43 UTC)" executed successfully 2020-03-31 09:33:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:33:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:49.498966+00:00 (in 5.174452 seconds) 2020-03-31 09:33:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:54 UTC)" (scheduled at 2020-03-31 09:33:44.323352+00:00) 2020-03-31 09:33:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:33:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:49.498966+00:00 (in 5.174452 seconds) 2020-03-31 09:33:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:54 UTC)" (scheduled at 2020-03-31 09:33:44.323352+00:00) 2020-03-31 09:33:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 44, 325524), True, datetime.datetime(2020, 3, 31, 9, 33, 44, 325524), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:33:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 44, 325524), True, datetime.datetime(2020, 3, 31, 9, 33, 44, 325524), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:33:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:33:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:33:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:54 UTC)" executed successfully 2020-03-31 09:33:44,446 [518] [DEBUG] [app] Starting request: urn:request:17399b88-f9ed-46f9-b63e-0a2b8efc122f (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:44,446 [518] [DEBUG] [app] Starting request: urn:request:17399b88-f9ed-46f9-b63e-0a2b8efc122f (/health/instance) 2020-03-31 09:33:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:44,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:44,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:44,453 [556] [DEBUG] [app] Starting request: urn:request:357a416b-6730-4579-ba3a-bbd7746eef2f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:44,453 [556] [DEBUG] [app] Starting request: urn:request:357a416b-6730-4579-ba3a-bbd7746eef2f (/v1/_internal_ping) 2020-03-31 09:33:44,454 [556] [DEBUG] [app] Ending request: urn:request:357a416b-6730-4579-ba3a-bbd7746eef2f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:44,454 [556] [DEBUG] [app] Ending request: urn:request:357a416b-6730-4579-ba3a-bbd7746eef2f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:44,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:33:44,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:33:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:44,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:44,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:44,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:44,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:44,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:44,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:44,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:44,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:44,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:44,489 [518] [DEBUG] [app] Starting request: urn:request:2d283c4b-cef3-4fea-8da9-c6eef53a91c4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:44,489 [518] [DEBUG] [app] Starting request: urn:request:2d283c4b-cef3-4fea-8da9-c6eef53a91c4 (/_internal_ping) 2020-03-31 09:33:44,490 [518] [DEBUG] [app] Ending request: urn:request:2d283c4b-cef3-4fea-8da9-c6eef53a91c4 (/_internal_ping) 2020-03-31 09:33:44,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:44,490 [518] [DEBUG] [app] Ending request: urn:request:2d283c4b-cef3-4fea-8da9-c6eef53a91c4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:44,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:33:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:33:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:44,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:44,504 [541] [DEBUG] [app] Starting request: urn:request:bcbde0ff-b4c5-4bd4-9097-fe71456a5f25 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:44,504 [541] [DEBUG] [app] Starting request: urn:request:bcbde0ff-b4c5-4bd4-9097-fe71456a5f25 (/c1/_internal_ping) 2020-03-31 09:33:44,505 [541] [DEBUG] [app] Ending request: urn:request:bcbde0ff-b4c5-4bd4-9097-fe71456a5f25 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:44,505 [541] [DEBUG] [app] Ending request: urn:request:bcbde0ff-b4c5-4bd4-9097-fe71456a5f25 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:44,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:33:44,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:44,507 [518] [DEBUG] [app] Ending request: urn:request:17399b88-f9ed-46f9-b63e-0a2b8efc122f (/health/instance) 2020-03-31 09:33:44,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:44,507 [518] [DEBUG] [app] Ending request: urn:request:17399b88-f9ed-46f9-b63e-0a2b8efc122f (/health/instance) 2020-03-31 09:33:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:33:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:33:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:33:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:15.064514+00:00 (in 29.999050 seconds) 2020-03-31 09:33:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:15 UTC)" (scheduled at 2020-03-31 09:33:45.064514+00:00) gcworker stdout | 2020-03-31 09:33:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:15.064514+00:00 (in 29.999050 seconds) 2020-03-31 09:33:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:15 UTC)" (scheduled at 2020-03-31 09:33:45.064514+00:00) 2020-03-31 09:33:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437625, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:33:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437625, 1209600, 500, 1, 0]) 2020-03-31 09:33:45,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:33:45,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:33:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:33:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:15 UTC)" executed successfully 2020-03-31 09:33:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:33:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:56.924497+00:00 (in 9.999115 seconds) 2020-03-31 09:33:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:56 UTC)" (scheduled at 2020-03-31 09:33:46.924497+00:00) 2020-03-31 09:33:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:33:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:56.924497+00:00 (in 9.999115 seconds) 2020-03-31 09:33:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:56 UTC)" (scheduled at 2020-03-31 09:33:46.924497+00:00) 2020-03-31 09:33:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 46, 926129), True, datetime.datetime(2020, 3, 31, 9, 33, 46, 926129), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:33:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 46, 926129), True, datetime.datetime(2020, 3, 31, 9, 33, 46, 926129), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:33:46,949 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:33:46,949 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:46,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:33:46,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:46,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:33:56 UTC)" executed successfully 2020-03-31 09:33:48,273 [527] [DEBUG] [app] Starting request: urn:request:2738a0d9-120e-4b8c-8ab6-d009a14add3f (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:48,273 [527] [DEBUG] [app] Starting request: urn:request:2738a0d9-120e-4b8c-8ab6-d009a14add3f (/health/instance) 2020-03-31 09:33:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:48,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:48,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:48,281 [554] [DEBUG] [app] Starting request: urn:request:985f6efa-b52f-4fb7-bda1-7af6c9c920c8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:48,281 [554] [DEBUG] [app] Starting request: urn:request:985f6efa-b52f-4fb7-bda1-7af6c9c920c8 (/v1/_internal_ping) 2020-03-31 09:33:48,282 [554] [DEBUG] [app] Ending request: urn:request:985f6efa-b52f-4fb7-bda1-7af6c9c920c8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:48,282 [554] [DEBUG] [app] Ending request: urn:request:985f6efa-b52f-4fb7-bda1-7af6c9c920c8 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:33:48,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:33:48,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:48,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:33:48,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:48,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:48,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:48,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:48,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:48,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:48,287 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:48,311 [527] [DEBUG] [app] Starting request: urn:request:36a21735-c13f-492e-8286-21d3a9fa723b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:48,311 [527] [DEBUG] [app] Starting request: urn:request:36a21735-c13f-492e-8286-21d3a9fa723b (/_internal_ping) 2020-03-31 09:33:48,312 [527] [DEBUG] [app] Ending request: urn:request:36a21735-c13f-492e-8286-21d3a9fa723b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:48,312 [527] [DEBUG] [app] Ending request: urn:request:36a21735-c13f-492e-8286-21d3a9fa723b (/_internal_ping) 2020-03-31 09:33:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:33:48,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:33:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:48,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:33:48,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:33:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:33:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:48,325 [536] [DEBUG] [app] Starting request: urn:request:a5f385a9-9ddd-40d9-8929-ce1c0188f505 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:48,325 [536] [DEBUG] [app] Starting request: urn:request:a5f385a9-9ddd-40d9-8929-ce1c0188f505 (/c1/_internal_ping) 2020-03-31 09:33:48,326 [536] [DEBUG] [app] Ending request: urn:request:a5f385a9-9ddd-40d9-8929-ce1c0188f505 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:48,326 [536] [DEBUG] [app] Ending request: urn:request:a5f385a9-9ddd-40d9-8929-ce1c0188f505 (/c1/_internal_ping) 2020-03-31 09:33:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:48,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:48,328 [527] [DEBUG] [app] Ending request: urn:request:2738a0d9-120e-4b8c-8ab6-d009a14add3f (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:33:48,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:48,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:48,328 [527] [DEBUG] [app] Ending request: urn:request:2738a0d9-120e-4b8c-8ab6-d009a14add3f (/health/instance) 2020-03-31 09:33:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:33:48,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:48,328 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:33:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:33:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:54.323352+00:00 (in 4.823598 seconds) 2020-03-31 09:33:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:49 UTC)" (scheduled at 2020-03-31 09:33:49.498966+00:00) 2020-03-31 09:33:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:33:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:54.323352+00:00 (in 4.823598 seconds) 2020-03-31 09:33:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:49 UTC)" (scheduled at 2020-03-31 09:33:49.498966+00:00) 2020-03-31 09:33:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:33:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:49 UTC)" executed successfully 2020-03-31 09:33:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:33:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:51 UTC)" (scheduled at 2020-03-31 09:33:51.110610+00:00) 2020-03-31 09:33:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:33:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:51 UTC)" (scheduled at 2020-03-31 09:33:51.110610+00:00) 2020-03-31 09:33:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:59.560326+00:00 (in 8.448489 seconds) namespacegcworker stdout | 2020-03-31 09:33:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:33:59.560326+00:00 (in 8.448489 seconds) 2020-03-31 09:33:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 51, 112593), True, datetime.datetime(2020, 3, 31, 9, 33, 51, 112593), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:33:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 51, 112593), True, datetime.datetime(2020, 3, 31, 9, 33, 51, 112593), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:33:51,128 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:33:51,128 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:51,128 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:33:51,128 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:51 UTC)" executed successfully 2020-03-31 09:33:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:33:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:01.440707+00:00 (in 9.998694 seconds) 2020-03-31 09:33:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:01 UTC)" (scheduled at 2020-03-31 09:33:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:33:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:01.440707+00:00 (in 9.998694 seconds) 2020-03-31 09:33:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:01 UTC)" (scheduled at 2020-03-31 09:33:51.440707+00:00) 2020-03-31 09:33:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:33:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:33:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:33:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:33:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:33:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:01 UTC)" executed successfully 2020-03-31 09:33:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:33:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:22.113221+00:00 (in 29.998841 seconds) 2020-03-31 09:33:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:22 UTC)" (scheduled at 2020-03-31 09:33:52.113221+00:00) securityworker stdout | 2020-03-31 09:33:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:22.113221+00:00 (in 29.998841 seconds) 2020-03-31 09:33:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:22 UTC)" (scheduled at 2020-03-31 09:33:52.113221+00:00) 2020-03-31 09:33:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:33:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:33:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:33:52,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:33:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:33:52,135 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:33:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:33:52,135 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:52,135 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:22 UTC)" executed successfully 2020-03-31 09:33:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:33:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:04.323352+00:00 (in 9.999128 seconds) 2020-03-31 09:33:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:04 UTC)" (scheduled at 2020-03-31 09:33:54.323352+00:00) 2020-03-31 09:33:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:33:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:04.323352+00:00 (in 9.999128 seconds) 2020-03-31 09:33:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:04 UTC)" (scheduled at 2020-03-31 09:33:54.323352+00:00) 2020-03-31 09:33:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 54, 324927), True, datetime.datetime(2020, 3, 31, 9, 33, 54, 324927), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:33:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 54, 324927), True, datetime.datetime(2020, 3, 31, 9, 33, 54, 324927), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:33:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:33:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:33:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:04 UTC)" executed successfully 2020-03-31 09:33:54,446 [518] [DEBUG] [app] Starting request: urn:request:f6034a2f-f3e8-4d01-b774-7e01302f3854 (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:54,446 [518] [DEBUG] [app] Starting request: urn:request:f6034a2f-f3e8-4d01-b774-7e01302f3854 (/health/instance) 2020-03-31 09:33:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:54,454 [553] [DEBUG] [app] Starting request: urn:request:efc733bb-71bd-4eca-998a-28c08824cbe5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:54,454 [553] [DEBUG] [app] Starting request: urn:request:efc733bb-71bd-4eca-998a-28c08824cbe5 (/v1/_internal_ping) 2020-03-31 09:33:54,455 [553] [DEBUG] [app] Ending request: urn:request:efc733bb-71bd-4eca-998a-28c08824cbe5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:54,455 [553] [DEBUG] [app] Ending request: urn:request:efc733bb-71bd-4eca-998a-28c08824cbe5 (/v1/_internal_ping) 2020-03-31 09:33:54,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:33:54,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:33:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:54,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:54,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:54,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:54,461 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:54,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:54,462 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:54,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:54,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:54,493 [518] [DEBUG] [app] Starting request: urn:request:ad123028-2fd6-471f-82b2-dfd0e06e90e7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:54,493 [518] [DEBUG] [app] Starting request: urn:request:ad123028-2fd6-471f-82b2-dfd0e06e90e7 (/_internal_ping) 2020-03-31 09:33:54,493 [518] [DEBUG] [app] Ending request: urn:request:ad123028-2fd6-471f-82b2-dfd0e06e90e7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:54,493 [518] [DEBUG] [app] Ending request: urn:request:ad123028-2fd6-471f-82b2-dfd0e06e90e7 (/_internal_ping) 2020-03-31 09:33:54,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:54,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:33:54,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:33:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:54,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:54,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:54,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:54,507 [536] [DEBUG] [app] Starting request: urn:request:a884d067-4b03-4ef2-b3c9-1e0c89db20e3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:54,507 [536] [DEBUG] [app] Starting request: urn:request:a884d067-4b03-4ef2-b3c9-1e0c89db20e3 (/c1/_internal_ping) 2020-03-31 09:33:54,508 [536] [DEBUG] [app] Ending request: urn:request:a884d067-4b03-4ef2-b3c9-1e0c89db20e3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:54,508 [536] [DEBUG] [app] Ending request: urn:request:a884d067-4b03-4ef2-b3c9-1e0c89db20e3 (/c1/_internal_ping) 2020-03-31 09:33:54,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:54,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:54,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:54,510 [518] [DEBUG] [app] Ending request: urn:request:f6034a2f-f3e8-4d01-b774-7e01302f3854 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:33:54,509 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:54,510 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:54,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) gunicorn-web stdout | 2020-03-31 09:33:54,510 [518] [DEBUG] [app] Ending request: urn:request:f6034a2f-f3e8-4d01-b774-7e01302f3854 (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:54,510 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:54,511 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:33:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:33:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:06.924497+00:00 (in 9.999216 seconds) 2020-03-31 09:33:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:06 UTC)" (scheduled at 2020-03-31 09:33:56.924497+00:00) 2020-03-31 09:33:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:33:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:06.924497+00:00 (in 9.999216 seconds) 2020-03-31 09:33:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:06 UTC)" (scheduled at 2020-03-31 09:33:56.924497+00:00) 2020-03-31 09:33:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:33:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 56, 925977), True, datetime.datetime(2020, 3, 31, 9, 33, 56, 925977), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:33:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 33, 56, 925977), True, datetime.datetime(2020, 3, 31, 9, 33, 56, 925977), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:33:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:33:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:33:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:33:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:06 UTC)" executed successfully 2020-03-31 09:33:58,273 [527] [DEBUG] [app] Starting request: urn:request:65c723f9-ec1a-4d11-8b4e-d86a7ff50fe8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:33:58,273 [527] [DEBUG] [app] Starting request: urn:request:65c723f9-ec1a-4d11-8b4e-d86a7ff50fe8 (/health/instance) 2020-03-31 09:33:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:58,280 [549] [DEBUG] [app] Starting request: urn:request:f472fce8-2fa7-4a2e-be68-65710f13a38b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:58,280 [549] [DEBUG] [app] Starting request: urn:request:f472fce8-2fa7-4a2e-be68-65710f13a38b (/v1/_internal_ping) 2020-03-31 09:33:58,281 [549] [DEBUG] [app] Ending request: urn:request:f472fce8-2fa7-4a2e-be68-65710f13a38b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:33:58,281 [549] [DEBUG] [app] Ending request: urn:request:f472fce8-2fa7-4a2e-be68-65710f13a38b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:58,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:33:58,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:33:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:33:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:33:58,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:33:58,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:33:58,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:33:58,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:33:58,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:33:58,287 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:33:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:33:58,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:33:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:33:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:33:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:58,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:58,313 [527] [DEBUG] [app] Starting request: urn:request:3537d59b-d916-431d-b4d6-17154fddb404 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:58,313 [527] [DEBUG] [app] Starting request: urn:request:3537d59b-d916-431d-b4d6-17154fddb404 (/_internal_ping) 2020-03-31 09:33:58,314 [527] [DEBUG] [app] Ending request: urn:request:3537d59b-d916-431d-b4d6-17154fddb404 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:33:58,314 [527] [DEBUG] [app] Ending request: urn:request:3537d59b-d916-431d-b4d6-17154fddb404 (/_internal_ping) 2020-03-31 09:33:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:33:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:58,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:33:58,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:33:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:33:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:33:58,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:33:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:58,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:33:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:33:58,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:33:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:33:58,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:33:58,327 [532] [DEBUG] [app] Starting request: urn:request:052b2a70-8533-4492-b473-fb302d8d6962 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:58,327 [532] [DEBUG] [app] Starting request: urn:request:052b2a70-8533-4492-b473-fb302d8d6962 (/c1/_internal_ping) 2020-03-31 09:33:58,328 [532] [DEBUG] [app] Ending request: urn:request:052b2a70-8533-4492-b473-fb302d8d6962 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:33:58,328 [532] [DEBUG] [app] Ending request: urn:request:052b2a70-8533-4492-b473-fb302d8d6962 (/c1/_internal_ping) 2020-03-31 09:33:58,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:33:58,329 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:33:58,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:33:58,329 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:33:58,330 [527] [DEBUG] [app] Ending request: urn:request:65c723f9-ec1a-4d11-8b4e-d86a7ff50fe8 (/health/instance) 2020-03-31 09:33:58,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:33:58,330 [527] [DEBUG] [app] Ending request: urn:request:65c723f9-ec1a-4d11-8b4e-d86a7ff50fe8 (/health/instance) 2020-03-31 09:33:58,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:33:58,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:33:58,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:33:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:33:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:33:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:33:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:33:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:59 UTC)" (scheduled at 2020-03-31 09:33:59.560326+00:00) 2020-03-31 09:33:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:51.110610+00:00 (in 51.548724 seconds) namespacegcworker stdout | 2020-03-31 09:33:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:59 UTC)" (scheduled at 2020-03-31 09:33:59.560326+00:00) 2020-03-31 09:33:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:33:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:51.110610+00:00 (in 51.548724 seconds) 2020-03-31 09:33:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:33:59,563 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:34:59 UTC)" executed successfully 2020-03-31 09:34:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:34:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:30.337699+00:00 (in 29.999106 seconds) 2020-03-31 09:34:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:30 UTC)" (scheduled at 2020-03-31 09:34:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:34:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:30.337699+00:00 (in 29.999106 seconds) 2020-03-31 09:34:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:30 UTC)" (scheduled at 2020-03-31 09:34:00.337699+00:00) 2020-03-31 09:34:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 34, 0, 339299), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:34:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 34, 0, 339299), False, 50, 1, 0]) 2020-03-31 09:34:00,352 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:34:00,352 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:34:00,352 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:00,353 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:34:00,352 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:00,353 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:30 UTC)" executed successfully 2020-03-31 09:34:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:34:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:11 UTC)" (scheduled at 2020-03-31 09:34:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:34:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:11 UTC)" (scheduled at 2020-03-31 09:34:01.440707+00:00) 2020-03-31 09:34:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:11.440707+00:00 (in 9.998912 seconds) repositoryactioncounter stdout | 2020-03-31 09:34:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:11.440707+00:00 (in 9.998912 seconds) 2020-03-31 09:34:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:34:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:34:01,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:34:01,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:34:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:34:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:11 UTC)" executed successfully 2020-03-31 09:34:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:34:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:23.410371+00:00 (in 20.079013 seconds) 2020-03-31 09:34:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:33 UTC)" (scheduled at 2020-03-31 09:34:03.330112+00:00) 2020-03-31 09:34:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:34:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:23.410371+00:00 (in 20.079013 seconds) 2020-03-31 09:34:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:33 UTC)" (scheduled at 2020-03-31 09:34:03.330112+00:00) 2020-03-31 09:34:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 3, 332304), True, datetime.datetime(2020, 3, 31, 9, 34, 3, 332304), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:34:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 3, 332304), True, datetime.datetime(2020, 3, 31, 9, 34, 3, 332304), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:34:03,353 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:34:03,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:34:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:33 UTC)" executed successfully 2020-03-31 09:34:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:34:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:14.323352+00:00 (in 9.999249 seconds) 2020-03-31 09:34:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:14 UTC)" (scheduled at 2020-03-31 09:34:04.323352+00:00) 2020-03-31 09:34:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:34:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:14.323352+00:00 (in 9.999249 seconds) 2020-03-31 09:34:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:14 UTC)" (scheduled at 2020-03-31 09:34:04.323352+00:00) 2020-03-31 09:34:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 4, 325139), True, datetime.datetime(2020, 3, 31, 9, 34, 4, 325139), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:34:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 4, 325139), True, datetime.datetime(2020, 3, 31, 9, 34, 4, 325139), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:34:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:34:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:34:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:14 UTC)" executed successfully 2020-03-31 09:34:04,446 [518] [DEBUG] [app] Starting request: urn:request:742a0007-d6e0-47b0-a64f-df01a91ac570 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:04,446 [518] [DEBUG] [app] Starting request: urn:request:742a0007-d6e0-47b0-a64f-df01a91ac570 (/health/instance) 2020-03-31 09:34:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:04,455 [540] [DEBUG] [app] Starting request: urn:request:c28736ff-9994-4369-a7dd-f53b8e44fe88 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:04,455 [540] [DEBUG] [app] Starting request: urn:request:c28736ff-9994-4369-a7dd-f53b8e44fe88 (/v1/_internal_ping) 2020-03-31 09:34:04,456 [540] [DEBUG] [app] Ending request: urn:request:c28736ff-9994-4369-a7dd-f53b8e44fe88 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:04,456 [540] [DEBUG] [app] Ending request: urn:request:c28736ff-9994-4369-a7dd-f53b8e44fe88 (/v1/_internal_ping) 2020-03-31 09:34:04,460 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:04,461 [518] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:34:04,460 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:04,461 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:04,463 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:34:04,463 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:04,468 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:04,468 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:04,469 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:04,469 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:04,470 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:04,470 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:04,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:04,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:04,491 [527] [DEBUG] [app] Starting request: urn:request:b9a529f8-7132-4edb-87c1-033d7aa51826 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:04,491 [527] [DEBUG] [app] Starting request: urn:request:b9a529f8-7132-4edb-87c1-033d7aa51826 (/_internal_ping) 2020-03-31 09:34:04,492 [527] [DEBUG] [app] Ending request: urn:request:b9a529f8-7132-4edb-87c1-033d7aa51826 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:04,492 [527] [DEBUG] [app] Ending request: urn:request:b9a529f8-7132-4edb-87c1-033d7aa51826 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:34:04,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:34:04,493 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:34:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:34:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:04,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:04,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:04,505 [536] [DEBUG] [app] Starting request: urn:request:f59f7903-5e7a-4824-8219-71c1089df63b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:04,505 [536] [DEBUG] [app] Starting request: urn:request:f59f7903-5e7a-4824-8219-71c1089df63b (/c1/_internal_ping) 2020-03-31 09:34:04,506 [536] [DEBUG] [app] Ending request: urn:request:f59f7903-5e7a-4824-8219-71c1089df63b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:04,506 [536] [DEBUG] [app] Ending request: urn:request:f59f7903-5e7a-4824-8219-71c1089df63b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:04,507 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:34:04,507 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:04,509 [518] [DEBUG] [app] Ending request: urn:request:742a0007-d6e0-47b0-a64f-df01a91ac570 (/health/instance) 2020-03-31 09:34:04,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:04,509 [518] [DEBUG] [app] Ending request: urn:request:742a0007-d6e0-47b0-a64f-df01a91ac570 (/health/instance) 2020-03-31 09:34:04,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:34:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:34:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:34:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:16.924497+00:00 (in 9.999239 seconds) 2020-03-31 09:34:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:16 UTC)" (scheduled at 2020-03-31 09:34:06.924497+00:00) 2020-03-31 09:34:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:34:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:16.924497+00:00 (in 9.999239 seconds) 2020-03-31 09:34:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:16 UTC)" (scheduled at 2020-03-31 09:34:06.924497+00:00) 2020-03-31 09:34:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 6, 925943), True, datetime.datetime(2020, 3, 31, 9, 34, 6, 925943), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:34:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 6, 925943), True, datetime.datetime(2020, 3, 31, 9, 34, 6, 925943), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:34:06,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:34:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:34:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:16 UTC)" executed successfully 2020-03-31 09:34:08,273 [527] [DEBUG] [app] Starting request: urn:request:3188187b-1420-48b8-a04c-66c36c992322 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:08,273 [527] [DEBUG] [app] Starting request: urn:request:3188187b-1420-48b8-a04c-66c36c992322 (/health/instance) 2020-03-31 09:34:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:08,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:08,282 [553] [DEBUG] [app] Starting request: urn:request:1a300aff-ddf1-41da-8f7e-efc7b981f2bc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:08,282 [553] [DEBUG] [app] Starting request: urn:request:1a300aff-ddf1-41da-8f7e-efc7b981f2bc (/v1/_internal_ping) 2020-03-31 09:34:08,283 [553] [DEBUG] [app] Ending request: urn:request:1a300aff-ddf1-41da-8f7e-efc7b981f2bc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:08,283 [553] [DEBUG] [app] Ending request: urn:request:1a300aff-ddf1-41da-8f7e-efc7b981f2bc (/v1/_internal_ping) 2020-03-31 09:34:08,283 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:34:08,283 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:08,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:34:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:08,287 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:08,285 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:08,287 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:08,288 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:08,288 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:08,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:08,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:08,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:08,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:08,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:08,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:08,316 [527] [DEBUG] [app] Starting request: urn:request:fdac82e9-9e72-437f-ad9f-ea76fef20a75 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:08,316 [527] [DEBUG] [app] Starting request: urn:request:fdac82e9-9e72-437f-ad9f-ea76fef20a75 (/_internal_ping) 2020-03-31 09:34:08,317 [527] [DEBUG] [app] Ending request: urn:request:fdac82e9-9e72-437f-ad9f-ea76fef20a75 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:08,317 [527] [DEBUG] [app] Ending request: urn:request:fdac82e9-9e72-437f-ad9f-ea76fef20a75 (/_internal_ping) 2020-03-31 09:34:08,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:34:08,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:08,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:34:08,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:08,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:08,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:34:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:08,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:34:08,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:08,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:08,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:08,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:08,333 [543] [DEBUG] [app] Starting request: urn:request:192340b2-3487-4871-b049-c6e1a3379e1f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:08,333 [543] [DEBUG] [app] Starting request: urn:request:192340b2-3487-4871-b049-c6e1a3379e1f (/c1/_internal_ping) 2020-03-31 09:34:08,334 [543] [DEBUG] [app] Ending request: urn:request:192340b2-3487-4871-b049-c6e1a3379e1f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:08,334 [543] [DEBUG] [app] Ending request: urn:request:192340b2-3487-4871-b049-c6e1a3379e1f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:08,334 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:08,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:34:08,334 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:08,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:08,337 [527] [DEBUG] [app] Ending request: urn:request:3188187b-1420-48b8-a04c-66c36c992322 (/health/instance) 2020-03-31 09:34:08,337 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:08,337 [527] [DEBUG] [app] Ending request: urn:request:3188187b-1420-48b8-a04c-66c36c992322 (/health/instance) 2020-03-31 09:34:08,337 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:08,338 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:34:08,338 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:34:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:34:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:21.440707+00:00 (in 9.997092 seconds) 2020-03-31 09:34:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:21 UTC)" (scheduled at 2020-03-31 09:34:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:34:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:21.440707+00:00 (in 9.997092 seconds) 2020-03-31 09:34:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:21 UTC)" (scheduled at 2020-03-31 09:34:11.440707+00:00) 2020-03-31 09:34:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:34:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:34:11,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:34:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:34:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:34:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:21 UTC)" executed successfully 2020-03-31 09:34:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:34:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:24.323352+00:00 (in 9.998940 seconds) 2020-03-31 09:34:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:24 UTC)" (scheduled at 2020-03-31 09:34:14.323352+00:00) 2020-03-31 09:34:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:34:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:24.323352+00:00 (in 9.998940 seconds) 2020-03-31 09:34:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:24 UTC)" (scheduled at 2020-03-31 09:34:14.323352+00:00) 2020-03-31 09:34:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 14, 325279), True, datetime.datetime(2020, 3, 31, 9, 34, 14, 325279), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:34:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 14, 325279), True, datetime.datetime(2020, 3, 31, 9, 34, 14, 325279), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:34:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:34:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:34:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:24 UTC)" executed successfully 2020-03-31 09:34:14,446 [527] [DEBUG] [app] Starting request: urn:request:42c4ba80-414a-4d24-b782-b144f94db7ea (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:14,446 [527] [DEBUG] [app] Starting request: urn:request:42c4ba80-414a-4d24-b782-b144f94db7ea (/health/instance) 2020-03-31 09:34:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:14,453 [550] [DEBUG] [app] Starting request: urn:request:c853086b-cb98-49e6-8dc0-7e27c8d351b2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:14,453 [550] [DEBUG] [app] Starting request: urn:request:c853086b-cb98-49e6-8dc0-7e27c8d351b2 (/v1/_internal_ping) 2020-03-31 09:34:14,454 [550] [DEBUG] [app] Ending request: urn:request:c853086b-cb98-49e6-8dc0-7e27c8d351b2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:14,454 [550] [DEBUG] [app] Ending request: urn:request:c853086b-cb98-49e6-8dc0-7e27c8d351b2 (/v1/_internal_ping) 2020-03-31 09:34:14,454 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:34:14,454 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:34:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:14,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:14,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:14,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:14,460 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:14,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:14,465 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:14,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:14,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:14,488 [518] [DEBUG] [app] Starting request: urn:request:db520993-0cb9-4f85-bf6b-cb9b20311f1a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:14,488 [518] [DEBUG] [app] Starting request: urn:request:db520993-0cb9-4f85-bf6b-cb9b20311f1a (/_internal_ping) 2020-03-31 09:34:14,490 [518] [DEBUG] [app] Ending request: urn:request:db520993-0cb9-4f85-bf6b-cb9b20311f1a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:14,490 [518] [DEBUG] [app] Ending request: urn:request:db520993-0cb9-4f85-bf6b-cb9b20311f1a (/_internal_ping) 2020-03-31 09:34:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:14,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:34:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:14,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:34:14,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:14,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:14,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:34:14,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:34:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:14,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:34:14,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:14,502 [532] [DEBUG] [app] Starting request: urn:request:df339cf4-f2ba-46b0-99af-df1027049b9c (/c1/_internal_ping) 2020-03-31 09:34:14,503 [532] [DEBUG] [app] Ending request: urn:request:df339cf4-f2ba-46b0-99af-df1027049b9c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:14,502 [532] [DEBUG] [app] Starting request: urn:request:df339cf4-f2ba-46b0-99af-df1027049b9c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:34:14,503 [532] [DEBUG] [app] Ending request: urn:request:df339cf4-f2ba-46b0-99af-df1027049b9c (/c1/_internal_ping) 2020-03-31 09:34:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:14,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:34:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:34:14,504 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:14,506 [527] [DEBUG] [app] Ending request: urn:request:42c4ba80-414a-4d24-b782-b144f94db7ea (/health/instance) 2020-03-31 09:34:14,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:14,506 [527] [DEBUG] [app] Ending request: urn:request:42c4ba80-414a-4d24-b782-b144f94db7ea (/health/instance) 2020-03-31 09:34:14,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:14,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:34:14,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:34:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:34:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:45.064514+00:00 (in 29.999082 seconds) 2020-03-31 09:34:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:45 UTC)" (scheduled at 2020-03-31 09:34:15.064514+00:00) gcworker stdout | 2020-03-31 09:34:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:45.064514+00:00 (in 29.999082 seconds) 2020-03-31 09:34:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:45 UTC)" (scheduled at 2020-03-31 09:34:15.064514+00:00) 2020-03-31 09:34:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437655, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:34:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437655, 1209600, 500, 1, 0]) 2020-03-31 09:34:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:34:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:34:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:34:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:45 UTC)" executed successfully 2020-03-31 09:34:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:34:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:26.924497+00:00 (in 9.999225 seconds) 2020-03-31 09:34:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:26 UTC)" (scheduled at 2020-03-31 09:34:16.924497+00:00) 2020-03-31 09:34:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:34:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:26.924497+00:00 (in 9.999225 seconds) 2020-03-31 09:34:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:26 UTC)" (scheduled at 2020-03-31 09:34:16.924497+00:00) 2020-03-31 09:34:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 16, 926008), True, datetime.datetime(2020, 3, 31, 9, 34, 16, 926008), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:34:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 16, 926008), True, datetime.datetime(2020, 3, 31, 9, 34, 16, 926008), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:34:16,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:34:16,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:16,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:34:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:16,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:26 UTC)" executed successfully 2020-03-31 09:34:18,273 [518] [DEBUG] [app] Starting request: urn:request:cd5ac1a9-a85c-4ab8-a5f1-72b7b360de10 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:18,273 [518] [DEBUG] [app] Starting request: urn:request:cd5ac1a9-a85c-4ab8-a5f1-72b7b360de10 (/health/instance) 2020-03-31 09:34:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:18,281 [551] [DEBUG] [app] Starting request: urn:request:e88cb7d0-60fe-4cc3-834e-66860725f15d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:18,281 [551] [DEBUG] [app] Starting request: urn:request:e88cb7d0-60fe-4cc3-834e-66860725f15d (/v1/_internal_ping) 2020-03-31 09:34:18,282 [551] [DEBUG] [app] Ending request: urn:request:e88cb7d0-60fe-4cc3-834e-66860725f15d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:18,282 [551] [DEBUG] [app] Ending request: urn:request:e88cb7d0-60fe-4cc3-834e-66860725f15d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:18,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:34:18,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:18,286 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:18,286 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:18,288 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:18,288 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:18,289 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:18,289 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:18,316 [527] [DEBUG] [app] Starting request: urn:request:e5331181-9fc9-4b6d-816f-80da5eca206c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:18,316 [527] [DEBUG] [app] Starting request: urn:request:e5331181-9fc9-4b6d-816f-80da5eca206c (/_internal_ping) 2020-03-31 09:34:18,317 [527] [DEBUG] [app] Ending request: urn:request:e5331181-9fc9-4b6d-816f-80da5eca206c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:18,317 [527] [DEBUG] [app] Ending request: urn:request:e5331181-9fc9-4b6d-816f-80da5eca206c (/_internal_ping) 2020-03-31 09:34:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:34:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:34:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:18,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:34:18,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:34:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:18,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:18,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:18,331 [543] [DEBUG] [app] Starting request: urn:request:7df8dd05-78a0-498e-95f5-fbca42ff29f2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:18,331 [543] [DEBUG] [app] Starting request: urn:request:7df8dd05-78a0-498e-95f5-fbca42ff29f2 (/c1/_internal_ping) 2020-03-31 09:34:18,332 [543] [DEBUG] [app] Ending request: urn:request:7df8dd05-78a0-498e-95f5-fbca42ff29f2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:18,332 [543] [DEBUG] [app] Ending request: urn:request:7df8dd05-78a0-498e-95f5-fbca42ff29f2 (/c1/_internal_ping) 2020-03-31 09:34:18,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:18,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:18,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:34:18,333 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:18,334 [518] [DEBUG] [app] Ending request: urn:request:cd5ac1a9-a85c-4ab8-a5f1-72b7b360de10 (/health/instance) 2020-03-31 09:34:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:18,334 [518] [DEBUG] [app] Ending request: urn:request:cd5ac1a9-a85c-4ab8-a5f1-72b7b360de10 (/health/instance) 2020-03-31 09:34:18,335 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:34:18,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:34:18,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:34:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:34:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:31 UTC)" (scheduled at 2020-03-31 09:34:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:34:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:31 UTC)" (scheduled at 2020-03-31 09:34:21.440707+00:00) 2020-03-31 09:34:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:31.440707+00:00 (in 9.999092 seconds) repositoryactioncounter stdout | 2020-03-31 09:34:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:31.440707+00:00 (in 9.999092 seconds) 2020-03-31 09:34:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:34:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:34:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:34:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:34:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:34:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:31 UTC)" executed successfully 2020-03-31 09:34:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:34:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:52 UTC)" (scheduled at 2020-03-31 09:34:22.113221+00:00) securityworker stdout | 2020-03-31 09:34:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:52 UTC)" (scheduled at 2020-03-31 09:34:22.113221+00:00) 2020-03-31 09:34:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:52.113221+00:00 (in 29.998547 seconds) 2020-03-31 09:34:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:34:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:52.113221+00:00 (in 29.998547 seconds) 2020-03-31 09:34:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:34:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:34:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:34:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:34:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:34:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:34:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:34:52 UTC)" executed successfully 2020-03-31 09:34:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:34:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:23 UTC)" (scheduled at 2020-03-31 09:34:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:34:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:34:23 UTC)" (scheduled at 2020-03-31 09:34:23.410371+00:00) 2020-03-31 09:34:23,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:24.410927+00:00 (in 0.999628 seconds) security_notification_worker stdout | 2020-03-31 09:34:23,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:24.410927+00:00 (in 0.999628 seconds) 2020-03-31 09:34:23,414 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:34:23,414 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), u'security_notification/%']) 2020-03-31 09:34:23,435 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), True, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:34:23,435 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), True, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), 0, u'security_notification/%']) 2020-03-31 09:34:23,440 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), True, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:34:23,440 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), True, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 34, 23, 411975), u'security_notification/%']) 2020-03-31 09:34:23,443 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:34:23,443 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:23,444 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:34:23,444 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:23 UTC)" executed successfully 2020-03-31 09:34:23,446 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:34:23,446 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:23,447 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:34:23,447 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:23,448 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:34:23,448 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:34:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:34:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:34.323352+00:00 (in 9.999198 seconds) 2020-03-31 09:34:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:34 UTC)" (scheduled at 2020-03-31 09:34:24.323352+00:00) notificationworker stdout | 2020-03-31 09:34:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:34.323352+00:00 (in 9.999198 seconds) 2020-03-31 09:34:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:34 UTC)" (scheduled at 2020-03-31 09:34:24.323352+00:00) 2020-03-31 09:34:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:34:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 24, 325415), True, datetime.datetime(2020, 3, 31, 9, 34, 24, 325415), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:34:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 24, 325415), True, datetime.datetime(2020, 3, 31, 9, 34, 24, 325415), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:34:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:24,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:34:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:34:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:34 UTC)" executed successfully 2020-03-31 09:34:24,414 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:34:24,414 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:24,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:33.330112+00:00 (in 8.915943 seconds) 2020-03-31 09:34:24,414 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:24 UTC)" (scheduled at 2020-03-31 09:34:24.410927+00:00) 2020-03-31 09:34:24,415 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:24,415 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:34:24,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:33.330112+00:00 (in 8.915943 seconds) 2020-03-31 09:34:24,414 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:24 UTC)" (scheduled at 2020-03-31 09:34:24.410927+00:00) 2020-03-31 09:34:24,415 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:24,415 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:24 UTC)" executed successfully 2020-03-31 09:34:24,446 [518] [DEBUG] [app] Starting request: urn:request:e5c52c20-658e-4622-80e1-02b7adfca20c (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:24,446 [518] [DEBUG] [app] Starting request: urn:request:e5c52c20-658e-4622-80e1-02b7adfca20c (/health/instance) 2020-03-31 09:34:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:24,454 [549] [DEBUG] [app] Starting request: urn:request:daff4eea-936b-420d-a63b-7b10f6dc616b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:24,454 [549] [DEBUG] [app] Starting request: urn:request:daff4eea-936b-420d-a63b-7b10f6dc616b (/v1/_internal_ping) 2020-03-31 09:34:24,455 [549] [DEBUG] [app] Ending request: urn:request:daff4eea-936b-420d-a63b-7b10f6dc616b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:24,455 [549] [DEBUG] [app] Ending request: urn:request:daff4eea-936b-420d-a63b-7b10f6dc616b (/v1/_internal_ping) 2020-03-31 09:34:24,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:34:24,456 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:24,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:34:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:24,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:24,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:24,459 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:24,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:24,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:24,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:24,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:24,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:24,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:24,490 [518] [DEBUG] [app] Starting request: urn:request:9c5b2cb8-70e3-4d90-b9ce-d48b9dfbdf5e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:24,490 [518] [DEBUG] [app] Starting request: urn:request:9c5b2cb8-70e3-4d90-b9ce-d48b9dfbdf5e (/_internal_ping) 2020-03-31 09:34:24,491 [518] [DEBUG] [app] Ending request: urn:request:9c5b2cb8-70e3-4d90-b9ce-d48b9dfbdf5e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:24,491 [518] [DEBUG] [app] Ending request: urn:request:9c5b2cb8-70e3-4d90-b9ce-d48b9dfbdf5e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:34:24,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:34:24,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:34:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:34:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:34:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:24,505 [543] [DEBUG] [app] Starting request: urn:request:d69d0e35-af3e-47ba-a037-ea198da93b43 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:24,505 [543] [DEBUG] [app] Starting request: urn:request:d69d0e35-af3e-47ba-a037-ea198da93b43 (/c1/_internal_ping) 2020-03-31 09:34:24,505 [543] [DEBUG] [app] Ending request: urn:request:d69d0e35-af3e-47ba-a037-ea198da93b43 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:24,505 [543] [DEBUG] [app] Ending request: urn:request:d69d0e35-af3e-47ba-a037-ea198da93b43 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:24,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:34:24,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:24,507 [518] [DEBUG] [app] Ending request: urn:request:e5c52c20-658e-4622-80e1-02b7adfca20c (/health/instance) 2020-03-31 09:34:24,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:24,507 [518] [DEBUG] [app] Ending request: urn:request:e5c52c20-658e-4622-80e1-02b7adfca20c (/health/instance) 2020-03-31 09:34:24,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:34:24,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:34:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:36.924497+00:00 (in 9.999262 seconds) storagereplication stdout | 2020-03-31 09:34:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:36.924497+00:00 (in 9.999262 seconds) 2020-03-31 09:34:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:36 UTC)" (scheduled at 2020-03-31 09:34:26.924497+00:00) 2020-03-31 09:34:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:34:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:36 UTC)" (scheduled at 2020-03-31 09:34:26.924497+00:00) 2020-03-31 09:34:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 26, 926017), True, datetime.datetime(2020, 3, 31, 9, 34, 26, 926017), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:34:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 26, 926017), True, datetime.datetime(2020, 3, 31, 9, 34, 26, 926017), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:34:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:34:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:34:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:36 UTC)" executed successfully 2020-03-31 09:34:28,273 [518] [DEBUG] [app] Starting request: urn:request:02ed9b22-bb2a-44b7-bf34-51afe821f394 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:28,273 [518] [DEBUG] [app] Starting request: urn:request:02ed9b22-bb2a-44b7-bf34-51afe821f394 (/health/instance) 2020-03-31 09:34:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:28,278 [556] [DEBUG] [app] Starting request: urn:request:8ab536ba-8ef8-4977-90e0-cd1db535ed06 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:28,278 [556] [DEBUG] [app] Starting request: urn:request:8ab536ba-8ef8-4977-90e0-cd1db535ed06 (/v1/_internal_ping) 2020-03-31 09:34:28,279 [556] [DEBUG] [app] Ending request: urn:request:8ab536ba-8ef8-4977-90e0-cd1db535ed06 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:28,279 [556] [DEBUG] [app] Ending request: urn:request:8ab536ba-8ef8-4977-90e0-cd1db535ed06 (/v1/_internal_ping) 2020-03-31 09:34:28,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:34:28,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:34:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:28,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:28,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:28,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:28,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:28,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:28,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:28,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:28,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:28,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:28,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:28,311 [520] [DEBUG] [app] Starting request: urn:request:8713cd41-4ab0-4c90-a450-e9158c49114e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:28,311 [520] [DEBUG] [app] Starting request: urn:request:8713cd41-4ab0-4c90-a450-e9158c49114e (/_internal_ping) 2020-03-31 09:34:28,311 [520] [DEBUG] [app] Ending request: urn:request:8713cd41-4ab0-4c90-a450-e9158c49114e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:28,311 [520] [DEBUG] [app] Ending request: urn:request:8713cd41-4ab0-4c90-a450-e9158c49114e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:34:28,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:34:28,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:34:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:34:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:34:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:34:28,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:28,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:28,326 [537] [DEBUG] [app] Starting request: urn:request:359c106f-9e10-4df4-8c3e-7cf48b8ec048 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:28,326 [537] [DEBUG] [app] Starting request: urn:request:359c106f-9e10-4df4-8c3e-7cf48b8ec048 (/c1/_internal_ping) 2020-03-31 09:34:28,326 [537] [DEBUG] [app] Ending request: urn:request:359c106f-9e10-4df4-8c3e-7cf48b8ec048 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:28,326 [537] [DEBUG] [app] Ending request: urn:request:359c106f-9e10-4df4-8c3e-7cf48b8ec048 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:28,327 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:34:28,327 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:28,329 [518] [DEBUG] [app] Ending request: urn:request:02ed9b22-bb2a-44b7-bf34-51afe821f394 (/health/instance) 2020-03-31 09:34:28,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:28,329 [518] [DEBUG] [app] Ending request: urn:request:02ed9b22-bb2a-44b7-bf34-51afe821f394 (/health/instance) 2020-03-31 09:34:28,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:28,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:34:28,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:34:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:34:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:00.337699+00:00 (in 29.999215 seconds) 2020-03-31 09:34:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:00 UTC)" (scheduled at 2020-03-31 09:34:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:34:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:00.337699+00:00 (in 29.999215 seconds) 2020-03-31 09:34:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:00 UTC)" (scheduled at 2020-03-31 09:34:30.337699+00:00) 2020-03-31 09:34:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 34, 30, 339368), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:34:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 34, 30, 339368), False, 50, 1, 0]) 2020-03-31 09:34:30,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:34:30,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:34:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:34:30,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:30,354 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:00 UTC)" executed successfully 2020-03-31 09:34:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:34:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:41.440707+00:00 (in 9.996265 seconds) 2020-03-31 09:34:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:41 UTC)" (scheduled at 2020-03-31 09:34:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:34:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:41.440707+00:00 (in 9.996265 seconds) 2020-03-31 09:34:31,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:41 UTC)" (scheduled at 2020-03-31 09:34:31.440707+00:00) 2020-03-31 09:34:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:34:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:34:31,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:34:31,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:34:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:34:31,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:41 UTC)" executed successfully 2020-03-31 09:34:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:34:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:03.330112+00:00 (in 29.998840 seconds) 2020-03-31 09:34:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:03 UTC)" (scheduled at 2020-03-31 09:34:33.330112+00:00) 2020-03-31 09:34:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:34:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:03.330112+00:00 (in 29.998840 seconds) 2020-03-31 09:34:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:03 UTC)" (scheduled at 2020-03-31 09:34:33.330112+00:00) 2020-03-31 09:34:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 33, 332126), True, datetime.datetime(2020, 3, 31, 9, 34, 33, 332126), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:34:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 33, 332126), True, datetime.datetime(2020, 3, 31, 9, 34, 33, 332126), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:34:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:34:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:34:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:03 UTC)" executed successfully 2020-03-31 09:34:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:34:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:43.539999+00:00 (in 10.172461 seconds) 2020-03-31 09:34:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:33 UTC)" (scheduled at 2020-03-31 09:34:33.366791+00:00) 2020-03-31 09:34:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:34:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:43.539999+00:00 (in 10.172461 seconds) 2020-03-31 09:34:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:33 UTC)" (scheduled at 2020-03-31 09:34:33.366791+00:00) 2020-03-31 09:34:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 33, 368410), True, datetime.datetime(2020, 3, 31, 9, 34, 33, 368410), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:34:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 33, 368410), True, datetime.datetime(2020, 3, 31, 9, 34, 33, 368410), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:34:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:34:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:34:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:33 UTC)" executed successfully 2020-03-31 09:34:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:34:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:44.323352+00:00 (in 9.998854 seconds) 2020-03-31 09:34:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:44 UTC)" (scheduled at 2020-03-31 09:34:34.323352+00:00) 2020-03-31 09:34:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:34:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:44.323352+00:00 (in 9.998854 seconds) 2020-03-31 09:34:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:44 UTC)" (scheduled at 2020-03-31 09:34:34.323352+00:00) 2020-03-31 09:34:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 34, 325622), True, datetime.datetime(2020, 3, 31, 9, 34, 34, 325622), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:34:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 34, 325622), True, datetime.datetime(2020, 3, 31, 9, 34, 34, 325622), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:34:34,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:34:34,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:34,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:34,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:34:34,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:34,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:44 UTC)" executed successfully 2020-03-31 09:34:34,446 [518] [DEBUG] [app] Starting request: urn:request:979145b6-04d9-46da-8f81-56074cc6d96d (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:34,446 [518] [DEBUG] [app] Starting request: urn:request:979145b6-04d9-46da-8f81-56074cc6d96d (/health/instance) 2020-03-31 09:34:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:34,452 [544] [DEBUG] [app] Starting request: urn:request:ffdafd54-c903-4703-9d91-cd6858fb2bde (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:34,452 [544] [DEBUG] [app] Starting request: urn:request:ffdafd54-c903-4703-9d91-cd6858fb2bde (/v1/_internal_ping) 2020-03-31 09:34:34,453 [544] [DEBUG] [app] Ending request: urn:request:ffdafd54-c903-4703-9d91-cd6858fb2bde (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:34,453 [544] [DEBUG] [app] Ending request: urn:request:ffdafd54-c903-4703-9d91-cd6858fb2bde (/v1/_internal_ping) 2020-03-31 09:34:34,454 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:34:34,454 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:34,456 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:34:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:34:34,456 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:34,457 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:34,457 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:34,458 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:34,458 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:34,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:34,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:34,485 [527] [DEBUG] [app] Starting request: urn:request:72136745-16bf-46e6-b334-c683f2d94027 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:34,485 [527] [DEBUG] [app] Starting request: urn:request:72136745-16bf-46e6-b334-c683f2d94027 (/_internal_ping) 2020-03-31 09:34:34,486 [527] [DEBUG] [app] Ending request: urn:request:72136745-16bf-46e6-b334-c683f2d94027 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:34,486 [527] [DEBUG] [app] Ending request: urn:request:72136745-16bf-46e6-b334-c683f2d94027 (/_internal_ping) 2020-03-31 09:34:34,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:34:34,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:34:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:34,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:34:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:34:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:34,500 [532] [DEBUG] [app] Starting request: urn:request:461fc494-255d-4069-8296-87c8997308e7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:34,500 [532] [DEBUG] [app] Starting request: urn:request:461fc494-255d-4069-8296-87c8997308e7 (/c1/_internal_ping) 2020-03-31 09:34:34,501 [532] [DEBUG] [app] Ending request: urn:request:461fc494-255d-4069-8296-87c8997308e7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:34,501 [532] [DEBUG] [app] Ending request: urn:request:461fc494-255d-4069-8296-87c8997308e7 (/c1/_internal_ping) 2020-03-31 09:34:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:34,501 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:34,503 [518] [DEBUG] [app] Ending request: urn:request:979145b6-04d9-46da-8f81-56074cc6d96d (/health/instance) 2020-03-31 09:34:34,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:34:34,501 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:34:34,503 [518] [DEBUG] [app] Ending request: urn:request:979145b6-04d9-46da-8f81-56074cc6d96d (/health/instance) 2020-03-31 09:34:34,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:34:36,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:34:36,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:38.010934+00:00 (in 1.085835 seconds) 2020-03-31 09:34:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:46 UTC)" (scheduled at 2020-03-31 09:34:36.924497+00:00) 2020-03-31 09:34:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 36, 926086), True, datetime.datetime(2020, 3, 31, 9, 34, 36, 926086), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:34:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:38.010934+00:00 (in 1.085835 seconds) 2020-03-31 09:34:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:46 UTC)" (scheduled at 2020-03-31 09:34:36.924497+00:00) 2020-03-31 09:34:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:34:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 36, 926086), True, datetime.datetime(2020, 3, 31, 9, 34, 36, 926086), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:34:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:34:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:34:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:46 UTC)" executed successfully 2020-03-31 09:34:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:34:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:46.924497+00:00 (in 8.912759 seconds) 2020-03-31 09:34:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:38 UTC)" (scheduled at 2020-03-31 09:34:38.010934+00:00) 2020-03-31 09:34:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:34:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:46.924497+00:00 (in 8.912759 seconds) 2020-03-31 09:34:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:38 UTC)" (scheduled at 2020-03-31 09:34:38.010934+00:00) 2020-03-31 09:34:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:38 UTC)" executed successfully 2020-03-31 09:34:38,273 [518] [DEBUG] [app] Starting request: urn:request:5564600a-e756-4eff-8808-54697ec4f819 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:38,273 [518] [DEBUG] [app] Starting request: urn:request:5564600a-e756-4eff-8808-54697ec4f819 (/health/instance) 2020-03-31 09:34:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:38,279 [544] [DEBUG] [app] Starting request: urn:request:5739a93b-11cd-4c2d-8a51-194b6320921b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:38,279 [544] [DEBUG] [app] Starting request: urn:request:5739a93b-11cd-4c2d-8a51-194b6320921b (/v1/_internal_ping) 2020-03-31 09:34:38,280 [544] [DEBUG] [app] Ending request: urn:request:5739a93b-11cd-4c2d-8a51-194b6320921b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:38,280 [544] [DEBUG] [app] Ending request: urn:request:5739a93b-11cd-4c2d-8a51-194b6320921b (/v1/_internal_ping) 2020-03-31 09:34:38,281 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:38,282 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:34:38,281 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:38,282 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:34:38,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:38,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:38,284 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:34:38,284 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:38,285 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:38,285 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:38,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:38,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:38,312 [520] [DEBUG] [app] Starting request: urn:request:d4e1ff2f-f606-48de-bb60-bcaa0a2bce86 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:38,312 [520] [DEBUG] [app] Starting request: urn:request:d4e1ff2f-f606-48de-bb60-bcaa0a2bce86 (/_internal_ping) 2020-03-31 09:34:38,313 [520] [DEBUG] [app] Ending request: urn:request:d4e1ff2f-f606-48de-bb60-bcaa0a2bce86 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:38,313 [520] [DEBUG] [app] Ending request: urn:request:d4e1ff2f-f606-48de-bb60-bcaa0a2bce86 (/_internal_ping) 2020-03-31 09:34:38,314 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:34:38,314 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:34:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:34:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:38,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:38,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:38,328 [541] [DEBUG] [app] Starting request: urn:request:43bf8dbb-7008-4aa2-9380-fec0ecb29dc6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:38,328 [541] [DEBUG] [app] Starting request: urn:request:43bf8dbb-7008-4aa2-9380-fec0ecb29dc6 (/c1/_internal_ping) 2020-03-31 09:34:38,329 [541] [DEBUG] [app] Ending request: urn:request:43bf8dbb-7008-4aa2-9380-fec0ecb29dc6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:38,329 [541] [DEBUG] [app] Ending request: urn:request:43bf8dbb-7008-4aa2-9380-fec0ecb29dc6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:38,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:38,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:38,331 [518] [DEBUG] [app] Ending request: urn:request:5564600a-e756-4eff-8808-54697ec4f819 (/health/instance) 2020-03-31 09:34:38,332 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:34:38,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:38,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-verbs stdout | 2020-03-31 09:34:38,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:38,331 [518] [DEBUG] [app] Ending request: urn:request:5564600a-e756-4eff-8808-54697ec4f819 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:38,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:38,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:34:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:34:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:51.440707+00:00 (in 9.999193 seconds) 2020-03-31 09:34:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:51 UTC)" (scheduled at 2020-03-31 09:34:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:34:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:51.440707+00:00 (in 9.999193 seconds) 2020-03-31 09:34:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:51 UTC)" (scheduled at 2020-03-31 09:34:41.440707+00:00) 2020-03-31 09:34:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:34:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:34:41,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:34:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:34:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:34:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:51 UTC)" executed successfully 2020-03-31 09:34:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:34:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:14.539408+00:00 (in 30.998649 seconds) 2020-03-31 09:34:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:43 UTC)" (scheduled at 2020-03-31 09:34:43.539999+00:00) 2020-03-31 09:34:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:34:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:14.539408+00:00 (in 30.998649 seconds) 2020-03-31 09:34:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:43 UTC)" (scheduled at 2020-03-31 09:34:43.539999+00:00) 2020-03-31 09:34:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:43 UTC)" executed successfully 2020-03-31 09:34:44,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:34:44,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:44,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:54 UTC)" (scheduled at 2020-03-31 09:34:44.323352+00:00) 2020-03-31 09:34:44,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:49.498966+00:00 (in 5.171584 seconds) 2020-03-31 09:34:44,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:34:44,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:54 UTC)" (scheduled at 2020-03-31 09:34:44.323352+00:00) 2020-03-31 09:34:44,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:49.498966+00:00 (in 5.171584 seconds) 2020-03-31 09:34:44,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:44,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 44, 328492), True, datetime.datetime(2020, 3, 31, 9, 34, 44, 328492), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:34:44,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 44, 328492), True, datetime.datetime(2020, 3, 31, 9, 34, 44, 328492), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:34:44,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:34:44,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:34:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:54 UTC)" executed successfully 2020-03-31 09:34:44,446 [527] [DEBUG] [app] Starting request: urn:request:d23dd457-ddd0-43f2-9546-dd89de292f47 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:44,446 [527] [DEBUG] [app] Starting request: urn:request:d23dd457-ddd0-43f2-9546-dd89de292f47 (/health/instance) 2020-03-31 09:34:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:44,453 [549] [DEBUG] [app] Starting request: urn:request:5253922d-213b-4f08-ae64-dcf2f4d41495 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:44,453 [549] [DEBUG] [app] Starting request: urn:request:5253922d-213b-4f08-ae64-dcf2f4d41495 (/v1/_internal_ping) 2020-03-31 09:34:44,455 [549] [DEBUG] [app] Ending request: urn:request:5253922d-213b-4f08-ae64-dcf2f4d41495 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:44,455 [549] [DEBUG] [app] Ending request: urn:request:5253922d-213b-4f08-ae64-dcf2f4d41495 (/v1/_internal_ping) 2020-03-31 09:34:44,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:34:44,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:44,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:44,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:44,458 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:44,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:34:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:34:44,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:44,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:44,481 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:44,481 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:44,494 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:44,494 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:44,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:44,500 [525] [DEBUG] [app] Starting request: urn:request:e5e9373e-8623-49ce-bc1d-c007dc0f466f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:44,500 [525] [DEBUG] [app] Starting request: urn:request:e5e9373e-8623-49ce-bc1d-c007dc0f466f (/_internal_ping) 2020-03-31 09:34:44,501 [525] [DEBUG] [app] Ending request: urn:request:e5e9373e-8623-49ce-bc1d-c007dc0f466f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:44,501 [525] [DEBUG] [app] Ending request: urn:request:e5e9373e-8623-49ce-bc1d-c007dc0f466f (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:34:44,501 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:34:44,501 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:44,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:44,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:44,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:44,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:34:44,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:44,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:44,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:44,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:34:44,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:44,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:44,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:44,514 [543] [DEBUG] [app] Starting request: urn:request:8369f51f-9814-4107-85e2-88502b2597e6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:44,514 [543] [DEBUG] [app] Starting request: urn:request:8369f51f-9814-4107-85e2-88502b2597e6 (/c1/_internal_ping) 2020-03-31 09:34:44,515 [543] [DEBUG] [app] Ending request: urn:request:8369f51f-9814-4107-85e2-88502b2597e6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:44,515 [543] [DEBUG] [app] Ending request: urn:request:8369f51f-9814-4107-85e2-88502b2597e6 (/c1/_internal_ping) 2020-03-31 09:34:44,516 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:44,515 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:44,516 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:34:44,515 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:44,517 [527] [DEBUG] [app] Ending request: urn:request:d23dd457-ddd0-43f2-9546-dd89de292f47 (/health/instance) 2020-03-31 09:34:44,517 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:44,517 [527] [DEBUG] [app] Ending request: urn:request:d23dd457-ddd0-43f2-9546-dd89de292f47 (/health/instance) 2020-03-31 09:34:44,517 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:44,518 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:34:44,518 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:34:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:34:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:15.064514+00:00 (in 29.999102 seconds) 2020-03-31 09:34:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:15 UTC)" (scheduled at 2020-03-31 09:34:45.064514+00:00) gcworker stdout | 2020-03-31 09:34:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:15.064514+00:00 (in 29.999102 seconds) 2020-03-31 09:34:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:15 UTC)" (scheduled at 2020-03-31 09:34:45.064514+00:00) 2020-03-31 09:34:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437685, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:34:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437685, 1209600, 500, 1, 0]) 2020-03-31 09:34:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:34:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:34:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:34:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:15 UTC)" executed successfully 2020-03-31 09:34:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:34:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:56.924497+00:00 (in 9.999111 seconds) 2020-03-31 09:34:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:56 UTC)" (scheduled at 2020-03-31 09:34:46.924497+00:00) 2020-03-31 09:34:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:34:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:56.924497+00:00 (in 9.999111 seconds) 2020-03-31 09:34:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:56 UTC)" (scheduled at 2020-03-31 09:34:46.924497+00:00) 2020-03-31 09:34:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 46, 926073), True, datetime.datetime(2020, 3, 31, 9, 34, 46, 926073), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:34:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 46, 926073), True, datetime.datetime(2020, 3, 31, 9, 34, 46, 926073), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:34:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:34:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:34:46,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:46,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:34:56 UTC)" executed successfully 2020-03-31 09:34:48,273 [525] [DEBUG] [app] Starting request: urn:request:da1ec22b-ad31-4f4f-9cb5-b2468f889de5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:48,273 [525] [DEBUG] [app] Starting request: urn:request:da1ec22b-ad31-4f4f-9cb5-b2468f889de5 (/health/instance) 2020-03-31 09:34:48,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:48,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:48,279 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:48,279 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:48,280 [538] [DEBUG] [app] Starting request: urn:request:d9cf5ee3-52dc-4622-bf01-487ae0c8d514 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:48,280 [538] [DEBUG] [app] Starting request: urn:request:d9cf5ee3-52dc-4622-bf01-487ae0c8d514 (/v1/_internal_ping) 2020-03-31 09:34:48,281 [538] [DEBUG] [app] Ending request: urn:request:d9cf5ee3-52dc-4622-bf01-487ae0c8d514 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:48,281 [538] [DEBUG] [app] Ending request: urn:request:d9cf5ee3-52dc-4622-bf01-487ae0c8d514 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:48,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:34:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:48,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:48,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:48,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:48,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:48,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:48,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:48,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:48,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:48,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:48,308 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:48,308 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:48,311 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:48,311 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:48,313 [518] [DEBUG] [app] Starting request: urn:request:37513217-f5b1-4317-b334-c58c806b908c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:48,313 [518] [DEBUG] [app] Starting request: urn:request:37513217-f5b1-4317-b334-c58c806b908c (/_internal_ping) 2020-03-31 09:34:48,314 [518] [DEBUG] [app] Ending request: urn:request:37513217-f5b1-4317-b334-c58c806b908c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:48,314 [518] [DEBUG] [app] Ending request: urn:request:37513217-f5b1-4317-b334-c58c806b908c (/_internal_ping) 2020-03-31 09:34:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:48,314 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:34:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:48,314 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:34:48,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:48,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:48,319 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:34:48,319 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:48,322 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:48,322 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:48,323 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:48,323 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:48,326 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:48,326 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:48,328 [541] [DEBUG] [app] Starting request: urn:request:ac36a794-9ce6-45fc-bb80-a6f3c45d5169 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:48,328 [541] [DEBUG] [app] Starting request: urn:request:ac36a794-9ce6-45fc-bb80-a6f3c45d5169 (/c1/_internal_ping) 2020-03-31 09:34:48,329 [541] [DEBUG] [app] Ending request: urn:request:ac36a794-9ce6-45fc-bb80-a6f3c45d5169 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:48,329 [541] [DEBUG] [app] Ending request: urn:request:ac36a794-9ce6-45fc-bb80-a6f3c45d5169 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:48,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:48,330 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:34:48,329 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:48,330 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:48,331 [525] [DEBUG] [app] Ending request: urn:request:da1ec22b-ad31-4f4f-9cb5-b2468f889de5 (/health/instance) 2020-03-31 09:34:48,331 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:48,331 [525] [DEBUG] [app] Ending request: urn:request:da1ec22b-ad31-4f4f-9cb5-b2468f889de5 (/health/instance) 2020-03-31 09:34:48,331 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:48,332 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:34:48,332 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:34:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:34:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:54.323352+00:00 (in 4.822945 seconds) 2020-03-31 09:34:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:49 UTC)" (scheduled at 2020-03-31 09:34:49.498966+00:00) notificationworker stdout | 2020-03-31 09:34:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:54.323352+00:00 (in 4.822945 seconds) 2020-03-31 09:34:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:49 UTC)" (scheduled at 2020-03-31 09:34:49.498966+00:00) 2020-03-31 09:34:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:34:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:49 UTC)" executed successfully 2020-03-31 09:34:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:34:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:53.556282+00:00 (in 2.443637 seconds) 2020-03-31 09:34:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:51 UTC)" (scheduled at 2020-03-31 09:34:51.110610+00:00) 2020-03-31 09:34:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:34:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:53.556282+00:00 (in 2.443637 seconds) 2020-03-31 09:34:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:51 UTC)" (scheduled at 2020-03-31 09:34:51.110610+00:00) 2020-03-31 09:34:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 51, 113513), True, datetime.datetime(2020, 3, 31, 9, 34, 51, 113513), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:34:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 51, 113513), True, datetime.datetime(2020, 3, 31, 9, 34, 51, 113513), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:34:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:34:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:34:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:51 UTC)" executed successfully 2020-03-31 09:34:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:34:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:01.440707+00:00 (in 9.999172 seconds) 2020-03-31 09:34:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:01 UTC)" (scheduled at 2020-03-31 09:34:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:34:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:01.440707+00:00 (in 9.999172 seconds) 2020-03-31 09:34:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:01 UTC)" (scheduled at 2020-03-31 09:34:51.440707+00:00) 2020-03-31 09:34:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:34:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:34:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:34:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:34:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:34:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:01 UTC)" executed successfully 2020-03-31 09:34:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:34:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:22.113221+00:00 (in 29.997869 seconds) 2020-03-31 09:34:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:22 UTC)" (scheduled at 2020-03-31 09:34:52.113221+00:00) securityworker stdout | 2020-03-31 09:34:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:22.113221+00:00 (in 29.997869 seconds) 2020-03-31 09:34:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:22 UTC)" (scheduled at 2020-03-31 09:34:52.113221+00:00) 2020-03-31 09:34:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:34:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:34:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:34:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:34:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:34:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:34:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:34:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:22 UTC)" executed successfully 2020-03-31 09:34:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:34:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:59.560326+00:00 (in 6.003307 seconds) 2020-03-31 09:34:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:53 UTC)" (scheduled at 2020-03-31 09:34:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:34:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:34:59.560326+00:00 (in 6.003307 seconds) 2020-03-31 09:34:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:53 UTC)" (scheduled at 2020-03-31 09:34:53.556282+00:00) 2020-03-31 09:34:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:34:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), u'namespacegc/%']) 2020-03-31 09:34:53,574 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), True, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:34:53,574 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), True, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), 0, u'namespacegc/%']) 2020-03-31 09:34:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), True, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:34:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), True, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 34, 53, 557822), u'namespacegc/%']) 2020-03-31 09:34:53,582 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:34:53,582 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:53,584 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:34:53,584 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:53 UTC)" executed successfully 2020-03-31 09:34:53,584 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:34:53,584 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:34:53,586 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:53,586 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:34:53,586 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:34:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:34:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:04.323352+00:00 (in 9.999213 seconds) 2020-03-31 09:34:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:04 UTC)" (scheduled at 2020-03-31 09:34:54.323352+00:00) 2020-03-31 09:34:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:34:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:04.323352+00:00 (in 9.999213 seconds) 2020-03-31 09:34:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:04 UTC)" (scheduled at 2020-03-31 09:34:54.323352+00:00) 2020-03-31 09:34:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 54, 325208), True, datetime.datetime(2020, 3, 31, 9, 34, 54, 325208), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:34:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 54, 325208), True, datetime.datetime(2020, 3, 31, 9, 34, 54, 325208), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:34:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:34:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:34:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:04 UTC)" executed successfully 2020-03-31 09:34:54,446 [518] [DEBUG] [app] Starting request: urn:request:b8d98953-2e09-4373-ac94-ffef1550ed45 (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:54,446 [518] [DEBUG] [app] Starting request: urn:request:b8d98953-2e09-4373-ac94-ffef1550ed45 (/health/instance) 2020-03-31 09:34:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:54,453 [542] [DEBUG] [app] Starting request: urn:request:a5819521-307a-4876-8973-e4df40449b7a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:54,453 [542] [DEBUG] [app] Starting request: urn:request:a5819521-307a-4876-8973-e4df40449b7a (/v1/_internal_ping) 2020-03-31 09:34:54,454 [542] [DEBUG] [app] Ending request: urn:request:a5819521-307a-4876-8973-e4df40449b7a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:54,454 [542] [DEBUG] [app] Ending request: urn:request:a5819521-307a-4876-8973-e4df40449b7a (/v1/_internal_ping) 2020-03-31 09:34:54,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:34:54,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:54,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:34:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:34:54,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:54,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:54,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:54,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:54,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:54,484 [518] [DEBUG] [app] Starting request: urn:request:469ba43d-b965-48a6-9cbb-1f653e74a36d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:54,484 [518] [DEBUG] [app] Starting request: urn:request:469ba43d-b965-48a6-9cbb-1f653e74a36d (/_internal_ping) 2020-03-31 09:34:54,485 [518] [DEBUG] [app] Ending request: urn:request:469ba43d-b965-48a6-9cbb-1f653e74a36d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:54,485 [518] [DEBUG] [app] Ending request: urn:request:469ba43d-b965-48a6-9cbb-1f653e74a36d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:34:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:34:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:34:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:34:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:54,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:34:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:54,498 [537] [DEBUG] [app] Starting request: urn:request:45239157-aff7-483e-a436-593502e60828 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:54,498 [537] [DEBUG] [app] Starting request: urn:request:45239157-aff7-483e-a436-593502e60828 (/c1/_internal_ping) 2020-03-31 09:34:54,499 [537] [DEBUG] [app] Ending request: urn:request:45239157-aff7-483e-a436-593502e60828 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:54,499 [537] [DEBUG] [app] Ending request: urn:request:45239157-aff7-483e-a436-593502e60828 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:54,499 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:34:54,499 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:54,501 [518] [DEBUG] [app] Ending request: urn:request:b8d98953-2e09-4373-ac94-ffef1550ed45 (/health/instance) 2020-03-31 09:34:54,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:34:54,501 [518] [DEBUG] [app] Ending request: urn:request:b8d98953-2e09-4373-ac94-ffef1550ed45 (/health/instance) 2020-03-31 09:34:54,501 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:54,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:34:54,501 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:34:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:06 UTC)" (scheduled at 2020-03-31 09:34:56.924497+00:00) 2020-03-31 09:34:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:34:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 56, 925802), True, datetime.datetime(2020, 3, 31, 9, 34, 56, 925802), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:34:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:06.924497+00:00 (in 9.999344 seconds) storagereplication stdout | 2020-03-31 09:34:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:34:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:06 UTC)" (scheduled at 2020-03-31 09:34:56.924497+00:00) 2020-03-31 09:34:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:34:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 34, 56, 925802), True, datetime.datetime(2020, 3, 31, 9, 34, 56, 925802), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:34:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:06.924497+00:00 (in 9.999344 seconds) 2020-03-31 09:34:56,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:34:56,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:34:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:34:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:06 UTC)" executed successfully 2020-03-31 09:34:58,273 [519] [DEBUG] [app] Starting request: urn:request:161cf810-db36-47f2-8169-18abb597937f (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:58,273 [519] [DEBUG] [app] Starting request: urn:request:161cf810-db36-47f2-8169-18abb597937f (/health/instance) 2020-03-31 09:34:58,275 [519] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:58,275 [519] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:58,279 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:58,279 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:58,280 [538] [DEBUG] [app] Starting request: urn:request:d2ef682e-a4ee-4db4-ba80-6016bedb2379 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:58,280 [538] [DEBUG] [app] Starting request: urn:request:d2ef682e-a4ee-4db4-ba80-6016bedb2379 (/v1/_internal_ping) 2020-03-31 09:34:58,281 [538] [DEBUG] [app] Ending request: urn:request:d2ef682e-a4ee-4db4-ba80-6016bedb2379 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:34:58,281 [538] [DEBUG] [app] Ending request: urn:request:d2ef682e-a4ee-4db4-ba80-6016bedb2379 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:34:58,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:58,282 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:34:58,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:58,283 [519] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:34:58,282 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:34:58,283 [519] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:34:58,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:34:58,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:34:58,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:34:58,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:34:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:34:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:34:58,294 [519] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:34:58,294 [519] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:34:58,310 [519] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:34:58,310 [519] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:34:58,314 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:58,314 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:58,315 [519] [DEBUG] [app] Starting request: urn:request:abbd1dd4-7521-4f0f-b166-20d01fb6d0cd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:58,315 [519] [DEBUG] [app] Starting request: urn:request:abbd1dd4-7521-4f0f-b166-20d01fb6d0cd (/_internal_ping) 2020-03-31 09:34:58,316 [519] [DEBUG] [app] Ending request: urn:request:abbd1dd4-7521-4f0f-b166-20d01fb6d0cd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:34:58,316 [519] [DEBUG] [app] Ending request: urn:request:abbd1dd4-7521-4f0f-b166-20d01fb6d0cd (/_internal_ping) 2020-03-31 09:34:58,317 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:34:58,317 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:58,317 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:34:58,317 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:34:58,321 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:58,321 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:58,321 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:34:58,321 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:34:58,324 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:58,324 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:34:58,325 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:34:58,325 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:34:58,328 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:34:58,328 [519] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:34:58,329 [543] [DEBUG] [app] Starting request: urn:request:da4b6ee2-7b28-4a65-88dc-12db983c4376 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:58,329 [543] [DEBUG] [app] Starting request: urn:request:da4b6ee2-7b28-4a65-88dc-12db983c4376 (/c1/_internal_ping) 2020-03-31 09:34:58,330 [543] [DEBUG] [app] Ending request: urn:request:da4b6ee2-7b28-4a65-88dc-12db983c4376 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:34:58,330 [543] [DEBUG] [app] Ending request: urn:request:da4b6ee2-7b28-4a65-88dc-12db983c4376 (/c1/_internal_ping) 2020-03-31 09:34:58,331 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:58,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:34:58,331 [519] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:34:58,332 [519] [DEBUG] [app] Ending request: urn:request:161cf810-db36-47f2-8169-18abb597937f (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:34:58,331 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:34:58,332 [519] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:58,333 [519] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:34:58,332 [519] [DEBUG] [app] Ending request: urn:request:161cf810-db36-47f2-8169-18abb597937f (/health/instance) gunicorn-web stdout | 2020-03-31 09:34:58,332 [519] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:34:58,333 [519] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:34:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:34:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:34:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:34:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:34:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:51.110610+00:00 (in 51.549283 seconds) 2020-03-31 09:34:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:59 UTC)" (scheduled at 2020-03-31 09:34:59.560326+00:00) 2020-03-31 09:34:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:34:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:51.110610+00:00 (in 51.549283 seconds) 2020-03-31 09:34:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:59 UTC)" (scheduled at 2020-03-31 09:34:59.560326+00:00) 2020-03-31 09:34:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:34:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:34:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:59 UTC)" executed successfully 2020-03-31 09:35:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:35:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:30.337699+00:00 (in 29.999111 seconds) 2020-03-31 09:35:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:30 UTC)" (scheduled at 2020-03-31 09:35:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:35:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:30.337699+00:00 (in 29.999111 seconds) 2020-03-31 09:35:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:30 UTC)" (scheduled at 2020-03-31 09:35:00.337699+00:00) 2020-03-31 09:35:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 35, 0, 339467), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:35:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 35, 0, 339467), False, 50, 1, 0]) 2020-03-31 09:35:00,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:35:00,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:35:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:35:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:30 UTC)" executed successfully 2020-03-31 09:35:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:35:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:11.440707+00:00 (in 9.998998 seconds) 2020-03-31 09:35:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:11 UTC)" (scheduled at 2020-03-31 09:35:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:35:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:11.440707+00:00 (in 9.998998 seconds) 2020-03-31 09:35:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:11 UTC)" (scheduled at 2020-03-31 09:35:01.440707+00:00) 2020-03-31 09:35:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:35:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:35:01,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:35:01,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:35:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:35:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:11 UTC)" executed successfully 2020-03-31 09:35:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:35:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:24.410927+00:00 (in 21.079931 seconds) 2020-03-31 09:35:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:33 UTC)" (scheduled at 2020-03-31 09:35:03.330112+00:00) 2020-03-31 09:35:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:35:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:24.410927+00:00 (in 21.079931 seconds) 2020-03-31 09:35:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:33 UTC)" (scheduled at 2020-03-31 09:35:03.330112+00:00) 2020-03-31 09:35:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 3, 331998), True, datetime.datetime(2020, 3, 31, 9, 35, 3, 331998), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:35:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 3, 331998), True, datetime.datetime(2020, 3, 31, 9, 35, 3, 331998), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:35:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:35:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:35:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:33 UTC)" executed successfully 2020-03-31 09:35:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:35:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:14.323352+00:00 (in 9.998929 seconds) 2020-03-31 09:35:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:14 UTC)" (scheduled at 2020-03-31 09:35:04.323352+00:00) 2020-03-31 09:35:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:35:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:14.323352+00:00 (in 9.998929 seconds) 2020-03-31 09:35:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:14 UTC)" (scheduled at 2020-03-31 09:35:04.323352+00:00) 2020-03-31 09:35:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 4, 325256), True, datetime.datetime(2020, 3, 31, 9, 35, 4, 325256), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:35:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 4, 325256), True, datetime.datetime(2020, 3, 31, 9, 35, 4, 325256), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:35:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:35:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:35:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:14 UTC)" executed successfully 2020-03-31 09:35:04,446 [518] [DEBUG] [app] Starting request: urn:request:dad4b739-2f3d-4caf-bddd-b43145aa2e4a (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:04,446 [518] [DEBUG] [app] Starting request: urn:request:dad4b739-2f3d-4caf-bddd-b43145aa2e4a (/health/instance) 2020-03-31 09:35:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:04,454 [553] [DEBUG] [app] Starting request: urn:request:e23530ee-febb-429a-ade2-e00c021f9ab6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:04,454 [553] [DEBUG] [app] Starting request: urn:request:e23530ee-febb-429a-ade2-e00c021f9ab6 (/v1/_internal_ping) 2020-03-31 09:35:04,455 [553] [DEBUG] [app] Ending request: urn:request:e23530ee-febb-429a-ade2-e00c021f9ab6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:04,455 [553] [DEBUG] [app] Ending request: urn:request:e23530ee-febb-429a-ade2-e00c021f9ab6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:04,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:35:04,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:35:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:35:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:04,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:04,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:04,490 [518] [DEBUG] [app] Starting request: urn:request:0ea6965b-d4ea-4ef8-a95c-19af3f63e620 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:04,490 [518] [DEBUG] [app] Starting request: urn:request:0ea6965b-d4ea-4ef8-a95c-19af3f63e620 (/_internal_ping) 2020-03-31 09:35:04,491 [518] [DEBUG] [app] Ending request: urn:request:0ea6965b-d4ea-4ef8-a95c-19af3f63e620 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:04,491 [518] [DEBUG] [app] Ending request: urn:request:0ea6965b-d4ea-4ef8-a95c-19af3f63e620 (/_internal_ping) 2020-03-31 09:35:04,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:35:04,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:35:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:35:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:35:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:04,505 [532] [DEBUG] [app] Starting request: urn:request:c4ddf0e5-a3e1-4549-a929-0fae88188369 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:04,505 [532] [DEBUG] [app] Starting request: urn:request:c4ddf0e5-a3e1-4549-a929-0fae88188369 (/c1/_internal_ping) 2020-03-31 09:35:04,505 [532] [DEBUG] [app] Ending request: urn:request:c4ddf0e5-a3e1-4549-a929-0fae88188369 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:04,505 [532] [DEBUG] [app] Ending request: urn:request:c4ddf0e5-a3e1-4549-a929-0fae88188369 (/c1/_internal_ping) 2020-03-31 09:35:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:35:04,506 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:35:04,506 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:04,508 [518] [DEBUG] [app] Ending request: urn:request:dad4b739-2f3d-4caf-bddd-b43145aa2e4a (/health/instance) 2020-03-31 09:35:04,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:04,508 [518] [DEBUG] [app] Ending request: urn:request:dad4b739-2f3d-4caf-bddd-b43145aa2e4a (/health/instance) 2020-03-31 09:35:04,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:35:04,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:35:06,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:35:06,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:06,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:16.924497+00:00 (in 9.995736 seconds) 2020-03-31 09:35:06,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:16 UTC)" (scheduled at 2020-03-31 09:35:06.924497+00:00) 2020-03-31 09:35:06,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:35:06,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:16.924497+00:00 (in 9.995736 seconds) 2020-03-31 09:35:06,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:16 UTC)" (scheduled at 2020-03-31 09:35:06.924497+00:00) 2020-03-31 09:35:06,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:06,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 6, 929506), True, datetime.datetime(2020, 3, 31, 9, 35, 6, 929506), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:35:06,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 6, 929506), True, datetime.datetime(2020, 3, 31, 9, 35, 6, 929506), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:35:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:35:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:35:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:16 UTC)" executed successfully 2020-03-31 09:35:08,273 [518] [DEBUG] [app] Starting request: urn:request:60b0dd9a-6e74-40ae-bc57-4423ed0c2dd4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:08,273 [518] [DEBUG] [app] Starting request: urn:request:60b0dd9a-6e74-40ae-bc57-4423ed0c2dd4 (/health/instance) 2020-03-31 09:35:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:08,278 [538] [DEBUG] [app] Starting request: urn:request:b12ff64f-e6d4-4327-8e1e-4a7d32c76f75 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:08,278 [538] [DEBUG] [app] Starting request: urn:request:b12ff64f-e6d4-4327-8e1e-4a7d32c76f75 (/v1/_internal_ping) 2020-03-31 09:35:08,279 [538] [DEBUG] [app] Ending request: urn:request:b12ff64f-e6d4-4327-8e1e-4a7d32c76f75 (/v1/_internal_ping) 2020-03-31 09:35:08,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:35:08,279 [538] [DEBUG] [app] Ending request: urn:request:b12ff64f-e6d4-4327-8e1e-4a7d32c76f75 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:08,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:35:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:35:08,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:08,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:08,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:08,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:08,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:08,314 [518] [DEBUG] [app] Starting request: urn:request:8cf62375-46da-4c89-990e-520686e59fde (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:08,314 [518] [DEBUG] [app] Starting request: urn:request:8cf62375-46da-4c89-990e-520686e59fde (/_internal_ping) 2020-03-31 09:35:08,315 [518] [DEBUG] [app] Ending request: urn:request:8cf62375-46da-4c89-990e-520686e59fde (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:08,315 [518] [DEBUG] [app] Ending request: urn:request:8cf62375-46da-4c89-990e-520686e59fde (/_internal_ping) 2020-03-31 09:35:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:35:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:35:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:35:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:35:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:35:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:08,328 [537] [DEBUG] [app] Starting request: urn:request:385533a7-b1c4-4321-9b16-06c5dd206394 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:08,328 [537] [DEBUG] [app] Starting request: urn:request:385533a7-b1c4-4321-9b16-06c5dd206394 (/c1/_internal_ping) 2020-03-31 09:35:08,329 [537] [DEBUG] [app] Ending request: urn:request:385533a7-b1c4-4321-9b16-06c5dd206394 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:08,329 [537] [DEBUG] [app] Ending request: urn:request:385533a7-b1c4-4321-9b16-06c5dd206394 (/c1/_internal_ping) 2020-03-31 09:35:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:35:08,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:35:08,331 [518] [DEBUG] [app] Ending request: urn:request:60b0dd9a-6e74-40ae-bc57-4423ed0c2dd4 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:35:08,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:35:08,331 [518] [DEBUG] [app] Ending request: urn:request:60b0dd9a-6e74-40ae-bc57-4423ed0c2dd4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:35:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:35:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:21.440707+00:00 (in 9.999016 seconds) 2020-03-31 09:35:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:21 UTC)" (scheduled at 2020-03-31 09:35:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:35:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:21.440707+00:00 (in 9.999016 seconds) 2020-03-31 09:35:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:21 UTC)" (scheduled at 2020-03-31 09:35:11.440707+00:00) 2020-03-31 09:35:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:35:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:35:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:35:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:35:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:35:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:21 UTC)" executed successfully 2020-03-31 09:35:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:35:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:24.323352+00:00 (in 9.999258 seconds) 2020-03-31 09:35:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:24 UTC)" (scheduled at 2020-03-31 09:35:14.323352+00:00) 2020-03-31 09:35:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:35:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:24.323352+00:00 (in 9.999258 seconds) 2020-03-31 09:35:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:24 UTC)" (scheduled at 2020-03-31 09:35:14.323352+00:00) 2020-03-31 09:35:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 14, 324978), True, datetime.datetime(2020, 3, 31, 9, 35, 14, 324978), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:35:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 14, 324978), True, datetime.datetime(2020, 3, 31, 9, 35, 14, 324978), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:35:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:35:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:35:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:24 UTC)" executed successfully 2020-03-31 09:35:14,447 [518] [DEBUG] [app] Starting request: urn:request:3d56d2c9-981b-4915-b677-d537427797b8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:14,447 [518] [DEBUG] [app] Starting request: urn:request:3d56d2c9-981b-4915-b677-d537427797b8 (/health/instance) 2020-03-31 09:35:14,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:14,450 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:14,455 [542] [DEBUG] [app] Starting request: urn:request:ea3ee7b3-a744-4172-aafa-feba52c0dcc2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:14,455 [542] [DEBUG] [app] Starting request: urn:request:ea3ee7b3-a744-4172-aafa-feba52c0dcc2 (/v1/_internal_ping) 2020-03-31 09:35:14,456 [542] [DEBUG] [app] Ending request: urn:request:ea3ee7b3-a744-4172-aafa-feba52c0dcc2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:14,456 [542] [DEBUG] [app] Ending request: urn:request:ea3ee7b3-a744-4172-aafa-feba52c0dcc2 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:35:14,457 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:35:14,457 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:14,459 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:14,459 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:14,461 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:14,461 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:14,462 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:14,462 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:14,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:14,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:14,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:14,485 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:14,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:14,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:14,490 [518] [DEBUG] [app] Starting request: urn:request:9221743a-05d6-466a-9db0-86e17ec9aaa5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:14,490 [518] [DEBUG] [app] Starting request: urn:request:9221743a-05d6-466a-9db0-86e17ec9aaa5 (/_internal_ping) 2020-03-31 09:35:14,491 [518] [DEBUG] [app] Ending request: urn:request:9221743a-05d6-466a-9db0-86e17ec9aaa5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:14,491 [518] [DEBUG] [app] Ending request: urn:request:9221743a-05d6-466a-9db0-86e17ec9aaa5 (/_internal_ping) 2020-03-31 09:35:14,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:35:14,491 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:35:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:35:14,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:35:14,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:14,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:14,504 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:14,506 [543] [DEBUG] [app] Starting request: urn:request:8d60425c-0c47-464c-ba24-a7ffecefca7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:14,506 [543] [DEBUG] [app] Starting request: urn:request:8d60425c-0c47-464c-ba24-a7ffecefca7c (/c1/_internal_ping) 2020-03-31 09:35:14,506 [543] [DEBUG] [app] Ending request: urn:request:8d60425c-0c47-464c-ba24-a7ffecefca7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:14,506 [543] [DEBUG] [app] Ending request: urn:request:8d60425c-0c47-464c-ba24-a7ffecefca7c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:14,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:35:14,507 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:14,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:14,509 [518] [DEBUG] [app] Ending request: urn:request:3d56d2c9-981b-4915-b677-d537427797b8 (/health/instance) 2020-03-31 09:35:14,509 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:14,509 [518] [DEBUG] [app] Ending request: urn:request:3d56d2c9-981b-4915-b677-d537427797b8 (/health/instance) 2020-03-31 09:35:14,509 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:14,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:35:14,510 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:35:14,539 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:35:14,539 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:33.366791+00:00 (in 18.826747 seconds) 2020-03-31 09:35:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:40:14 UTC)" (scheduled at 2020-03-31 09:35:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:35:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:33.366791+00:00 (in 18.826747 seconds) 2020-03-31 09:35:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:40:14 UTC)" (scheduled at 2020-03-31 09:35:14.539408+00:00) 2020-03-31 09:35:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:35:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), u'exportactionlogs/%']) 2020-03-31 09:35:14,555 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), True, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:35:14,555 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), True, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), 0, u'exportactionlogs/%']) 2020-03-31 09:35:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), True, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:35:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), True, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 35, 14, 540858), u'exportactionlogs/%']) 2020-03-31 09:35:14,562 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:35:14,562 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:14,563 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:40:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:35:14,563 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:40:14 UTC)" executed successfully 2020-03-31 09:35:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:35:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:14,567 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:14,567 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:35:14,567 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:35:14,567 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:35:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:35:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:45.064514+00:00 (in 29.999082 seconds) 2020-03-31 09:35:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:45 UTC)" (scheduled at 2020-03-31 09:35:15.064514+00:00) gcworker stdout | 2020-03-31 09:35:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:45.064514+00:00 (in 29.999082 seconds) 2020-03-31 09:35:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:45 UTC)" (scheduled at 2020-03-31 09:35:15.064514+00:00) 2020-03-31 09:35:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437715, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:35:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437715, 1209600, 500, 1, 0]) 2020-03-31 09:35:15,095 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:35:15,095 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:35:15,095 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:15,096 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:35:15,095 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:15,096 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:45 UTC)" executed successfully 2020-03-31 09:35:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:35:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:26.924497+00:00 (in 9.999321 seconds) 2020-03-31 09:35:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:26 UTC)" (scheduled at 2020-03-31 09:35:16.924497+00:00) 2020-03-31 09:35:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:35:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:26.924497+00:00 (in 9.999321 seconds) 2020-03-31 09:35:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:26 UTC)" (scheduled at 2020-03-31 09:35:16.924497+00:00) 2020-03-31 09:35:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:16,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 16, 926106), True, datetime.datetime(2020, 3, 31, 9, 35, 16, 926106), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:35:16,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 16, 926106), True, datetime.datetime(2020, 3, 31, 9, 35, 16, 926106), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:35:16,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:35:16,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:16,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:35:16,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:16,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:26 UTC)" executed successfully 2020-03-31 09:35:18,272 [527] [DEBUG] [app] Starting request: urn:request:6f4cf18d-bc12-4a76-b6fb-6c8972411540 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:18,272 [527] [DEBUG] [app] Starting request: urn:request:6f4cf18d-bc12-4a76-b6fb-6c8972411540 (/health/instance) 2020-03-31 09:35:18,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:18,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:18,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:18,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:18,280 [538] [DEBUG] [app] Starting request: urn:request:430e7e1c-c35e-4255-b2bc-c8103afd9757 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:18,280 [538] [DEBUG] [app] Starting request: urn:request:430e7e1c-c35e-4255-b2bc-c8103afd9757 (/v1/_internal_ping) 2020-03-31 09:35:18,281 [538] [DEBUG] [app] Ending request: urn:request:430e7e1c-c35e-4255-b2bc-c8103afd9757 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:18,281 [538] [DEBUG] [app] Ending request: urn:request:430e7e1c-c35e-4255-b2bc-c8103afd9757 (/v1/_internal_ping) 2020-03-31 09:35:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:35:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:35:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:18,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:18,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:18,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:18,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:18,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:18,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:18,313 [527] [DEBUG] [app] Starting request: urn:request:4e83e66d-a38e-4889-8f24-5ad45804b88b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:18,313 [527] [DEBUG] [app] Starting request: urn:request:4e83e66d-a38e-4889-8f24-5ad45804b88b (/_internal_ping) 2020-03-31 09:35:18,313 [527] [DEBUG] [app] Ending request: urn:request:4e83e66d-a38e-4889-8f24-5ad45804b88b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:18,313 [527] [DEBUG] [app] Ending request: urn:request:4e83e66d-a38e-4889-8f24-5ad45804b88b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:35:18,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:18,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:18,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:18,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:18,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:18,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:35:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:35:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:35:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:18,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:18,326 [536] [DEBUG] [app] Starting request: urn:request:2061878a-6ace-4673-8f7a-08f8807e97b7 (/c1/_internal_ping) 2020-03-31 09:35:18,327 [536] [DEBUG] [app] Ending request: urn:request:2061878a-6ace-4673-8f7a-08f8807e97b7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:35:18,326 [536] [DEBUG] [app] Starting request: urn:request:2061878a-6ace-4673-8f7a-08f8807e97b7 (/c1/_internal_ping) 2020-03-31 09:35:18,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:18,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:35:18,327 [536] [DEBUG] [app] Ending request: urn:request:2061878a-6ace-4673-8f7a-08f8807e97b7 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:35:18,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:18,329 [527] [DEBUG] [app] Ending request: urn:request:6f4cf18d-bc12-4a76-b6fb-6c8972411540 (/health/instance) 2020-03-31 09:35:18,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:35:18,328 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:18,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:35:18,329 [527] [DEBUG] [app] Ending request: urn:request:6f4cf18d-bc12-4a76-b6fb-6c8972411540 (/health/instance) 2020-03-31 09:35:18,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:18,330 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:35:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:35:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:31.440707+00:00 (in 9.998864 seconds) 2020-03-31 09:35:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:31 UTC)" (scheduled at 2020-03-31 09:35:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:35:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:31.440707+00:00 (in 9.998864 seconds) 2020-03-31 09:35:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:31 UTC)" (scheduled at 2020-03-31 09:35:21.440707+00:00) 2020-03-31 09:35:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:35:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:35:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:35:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:35:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:35:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:31 UTC)" executed successfully 2020-03-31 09:35:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:35:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:52.113221+00:00 (in 29.999273 seconds) 2020-03-31 09:35:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:52 UTC)" (scheduled at 2020-03-31 09:35:22.113221+00:00) securityworker stdout | 2020-03-31 09:35:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:52.113221+00:00 (in 29.999273 seconds) 2020-03-31 09:35:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:52 UTC)" (scheduled at 2020-03-31 09:35:22.113221+00:00) 2020-03-31 09:35:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:35:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:35:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:35:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:35:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:35:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:35:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:35:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:35:52 UTC)" executed successfully 2020-03-31 09:35:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:35:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:34.323352+00:00 (in 9.998984 seconds) 2020-03-31 09:35:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:34 UTC)" (scheduled at 2020-03-31 09:35:24.323352+00:00) 2020-03-31 09:35:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:35:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:34.323352+00:00 (in 9.998984 seconds) 2020-03-31 09:35:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:34 UTC)" (scheduled at 2020-03-31 09:35:24.323352+00:00) 2020-03-31 09:35:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 24, 325317), True, datetime.datetime(2020, 3, 31, 9, 35, 24, 325317), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:35:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 24, 325317), True, datetime.datetime(2020, 3, 31, 9, 35, 24, 325317), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:35:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:35:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:35:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:34 UTC)" executed successfully 2020-03-31 09:35:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:35:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:33.330112+00:00 (in 8.918102 seconds) 2020-03-31 09:35:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:24 UTC)" (scheduled at 2020-03-31 09:35:24.410927+00:00) 2020-03-31 09:35:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:35:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:33.330112+00:00 (in 8.918102 seconds) 2020-03-31 09:35:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:24 UTC)" (scheduled at 2020-03-31 09:35:24.410927+00:00) 2020-03-31 09:35:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:24 UTC)" executed successfully 2020-03-31 09:35:24,446 [518] [DEBUG] [app] Starting request: urn:request:6c22386b-81bf-4b34-85a7-a77d8c2f1614 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:24,446 [518] [DEBUG] [app] Starting request: urn:request:6c22386b-81bf-4b34-85a7-a77d8c2f1614 (/health/instance) 2020-03-31 09:35:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:24,455 [556] [DEBUG] [app] Starting request: urn:request:b8b81245-00ec-4423-a1b6-7ae71c74bc56 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:24,455 [556] [DEBUG] [app] Starting request: urn:request:b8b81245-00ec-4423-a1b6-7ae71c74bc56 (/v1/_internal_ping) 2020-03-31 09:35:24,456 [556] [DEBUG] [app] Ending request: urn:request:b8b81245-00ec-4423-a1b6-7ae71c74bc56 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:24,456 [556] [DEBUG] [app] Ending request: urn:request:b8b81245-00ec-4423-a1b6-7ae71c74bc56 (/v1/_internal_ping) 2020-03-31 09:35:24,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:35:24,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:24,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:24,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:24,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:24,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:24,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:24,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:24,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:24,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:24,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:24,490 [527] [DEBUG] [app] Starting request: urn:request:379fcf2f-8c2e-43e9-81ba-dfcd05de3e97 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:24,490 [527] [DEBUG] [app] Starting request: urn:request:379fcf2f-8c2e-43e9-81ba-dfcd05de3e97 (/_internal_ping) 2020-03-31 09:35:24,492 [527] [DEBUG] [app] Ending request: urn:request:379fcf2f-8c2e-43e9-81ba-dfcd05de3e97 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:24,492 [527] [DEBUG] [app] Ending request: urn:request:379fcf2f-8c2e-43e9-81ba-dfcd05de3e97 (/_internal_ping) 2020-03-31 09:35:24,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:35:24,492 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:35:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:35:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:24,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:24,505 [543] [DEBUG] [app] Starting request: urn:request:3ba37091-faf4-4a10-b284-9fb80ffcf364 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:24,505 [543] [DEBUG] [app] Starting request: urn:request:3ba37091-faf4-4a10-b284-9fb80ffcf364 (/c1/_internal_ping) 2020-03-31 09:35:24,505 [543] [DEBUG] [app] Ending request: urn:request:3ba37091-faf4-4a10-b284-9fb80ffcf364 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:24,505 [543] [DEBUG] [app] Ending request: urn:request:3ba37091-faf4-4a10-b284-9fb80ffcf364 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:24,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:35:24,506 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:24,508 [518] [DEBUG] [app] Ending request: urn:request:6c22386b-81bf-4b34-85a7-a77d8c2f1614 (/health/instance) 2020-03-31 09:35:24,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:24,508 [518] [DEBUG] [app] Ending request: urn:request:6c22386b-81bf-4b34-85a7-a77d8c2f1614 (/health/instance) 2020-03-31 09:35:24,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:24,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:35:24,509 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:35:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:35:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:36.924497+00:00 (in 9.999205 seconds) 2020-03-31 09:35:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:36 UTC)" (scheduled at 2020-03-31 09:35:26.924497+00:00) 2020-03-31 09:35:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:35:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:36.924497+00:00 (in 9.999205 seconds) 2020-03-31 09:35:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:36 UTC)" (scheduled at 2020-03-31 09:35:26.924497+00:00) 2020-03-31 09:35:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 26, 925996), True, datetime.datetime(2020, 3, 31, 9, 35, 26, 925996), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:35:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 26, 925996), True, datetime.datetime(2020, 3, 31, 9, 35, 26, 925996), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:35:26,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:35:26,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:26,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:26,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:35:26,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:26,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:36 UTC)" executed successfully 2020-03-31 09:35:28,273 [518] [DEBUG] [app] Starting request: urn:request:3539cea2-7dc9-40fc-af71-28f3c3716fab (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:28,273 [518] [DEBUG] [app] Starting request: urn:request:3539cea2-7dc9-40fc-af71-28f3c3716fab (/health/instance) 2020-03-31 09:35:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:28,278 [538] [DEBUG] [app] Starting request: urn:request:3b0b0890-ec90-4085-a6eb-15ecf5175b2e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:28,278 [538] [DEBUG] [app] Starting request: urn:request:3b0b0890-ec90-4085-a6eb-15ecf5175b2e (/v1/_internal_ping) 2020-03-31 09:35:28,279 [538] [DEBUG] [app] Ending request: urn:request:3b0b0890-ec90-4085-a6eb-15ecf5175b2e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:28,279 [538] [DEBUG] [app] Ending request: urn:request:3b0b0890-ec90-4085-a6eb-15ecf5175b2e (/v1/_internal_ping) 2020-03-31 09:35:28,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:35:28,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:28,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:35:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:35:28,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:28,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:28,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:28,311 [520] [DEBUG] [app] Starting request: urn:request:f5de4b4f-4ec0-4d00-a4bf-ba6ec5eac395 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:28,311 [520] [DEBUG] [app] Starting request: urn:request:f5de4b4f-4ec0-4d00-a4bf-ba6ec5eac395 (/_internal_ping) 2020-03-31 09:35:28,312 [520] [DEBUG] [app] Ending request: urn:request:f5de4b4f-4ec0-4d00-a4bf-ba6ec5eac395 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:28,312 [520] [DEBUG] [app] Ending request: urn:request:f5de4b4f-4ec0-4d00-a4bf-ba6ec5eac395 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:35:28,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:28,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:28,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:28,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:35:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:28,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:35:28,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:35:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:28,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:28,324 [543] [DEBUG] [app] Starting request: urn:request:ee0415e5-d863-4c28-806c-a71885533cb0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:28,324 [543] [DEBUG] [app] Starting request: urn:request:ee0415e5-d863-4c28-806c-a71885533cb0 (/c1/_internal_ping) 2020-03-31 09:35:28,325 [543] [DEBUG] [app] Ending request: urn:request:ee0415e5-d863-4c28-806c-a71885533cb0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:28,325 [543] [DEBUG] [app] Ending request: urn:request:ee0415e5-d863-4c28-806c-a71885533cb0 (/c1/_internal_ping) 2020-03-31 09:35:28,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:35:28,326 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:35:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:28,328 [518] [DEBUG] [app] Ending request: urn:request:3539cea2-7dc9-40fc-af71-28f3c3716fab (/health/instance) 2020-03-31 09:35:28,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:28,328 [518] [DEBUG] [app] Ending request: urn:request:3539cea2-7dc9-40fc-af71-28f3c3716fab (/health/instance) 2020-03-31 09:35:28,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:28,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:35:28,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:35:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:35:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:00.337699+00:00 (in 29.999159 seconds) 2020-03-31 09:35:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:00 UTC)" (scheduled at 2020-03-31 09:35:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:35:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:00.337699+00:00 (in 29.999159 seconds) 2020-03-31 09:35:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:00 UTC)" (scheduled at 2020-03-31 09:35:30.337699+00:00) 2020-03-31 09:35:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 35, 30, 339602), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:35:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 35, 30, 339602), False, 50, 1, 0]) 2020-03-31 09:35:30,440 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:35:30,440 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:35:30,440 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:30,441 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:35:30,440 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:30,441 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:00 UTC)" executed successfully 2020-03-31 09:35:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:35:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:41.440707+00:00 (in 9.999047 seconds) 2020-03-31 09:35:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:41 UTC)" (scheduled at 2020-03-31 09:35:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:35:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:41.440707+00:00 (in 9.999047 seconds) 2020-03-31 09:35:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:41 UTC)" (scheduled at 2020-03-31 09:35:31.440707+00:00) 2020-03-31 09:35:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:35:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:35:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:35:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:35:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:35:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:41 UTC)" executed successfully 2020-03-31 09:35:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:35:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:03.330112+00:00 (in 29.999013 seconds) 2020-03-31 09:35:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:03 UTC)" (scheduled at 2020-03-31 09:35:33.330112+00:00) 2020-03-31 09:35:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:35:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:03.330112+00:00 (in 29.999013 seconds) 2020-03-31 09:35:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:03 UTC)" (scheduled at 2020-03-31 09:35:33.330112+00:00) 2020-03-31 09:35:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 33, 332302), True, datetime.datetime(2020, 3, 31, 9, 35, 33, 332302), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:35:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 33, 332302), True, datetime.datetime(2020, 3, 31, 9, 35, 33, 332302), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:35:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:35:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:35:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:03 UTC)" executed successfully 2020-03-31 09:35:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:35:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:43.539999+00:00 (in 10.172505 seconds) 2020-03-31 09:35:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:33 UTC)" (scheduled at 2020-03-31 09:35:33.366791+00:00) 2020-03-31 09:35:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:35:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:43.539999+00:00 (in 10.172505 seconds) 2020-03-31 09:35:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:33 UTC)" (scheduled at 2020-03-31 09:35:33.366791+00:00) 2020-03-31 09:35:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 33, 368391), True, datetime.datetime(2020, 3, 31, 9, 35, 33, 368391), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:35:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 33, 368391), True, datetime.datetime(2020, 3, 31, 9, 35, 33, 368391), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:35:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:35:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:35:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:33 UTC)" executed successfully 2020-03-31 09:35:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:35:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:44.323352+00:00 (in 9.999103 seconds) 2020-03-31 09:35:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:44 UTC)" (scheduled at 2020-03-31 09:35:34.323352+00:00) 2020-03-31 09:35:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:35:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:44.323352+00:00 (in 9.999103 seconds) 2020-03-31 09:35:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:44 UTC)" (scheduled at 2020-03-31 09:35:34.323352+00:00) 2020-03-31 09:35:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 34, 325260), True, datetime.datetime(2020, 3, 31, 9, 35, 34, 325260), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:35:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 34, 325260), True, datetime.datetime(2020, 3, 31, 9, 35, 34, 325260), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:35:34,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:35:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:35:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:44 UTC)" executed successfully 2020-03-31 09:35:34,446 [518] [DEBUG] [app] Starting request: urn:request:c8e6cfa9-cef3-4ea3-8050-3c5840eb0396 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:34,446 [518] [DEBUG] [app] Starting request: urn:request:c8e6cfa9-cef3-4ea3-8050-3c5840eb0396 (/health/instance) 2020-03-31 09:35:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:34,453 [548] [DEBUG] [app] Starting request: urn:request:b6663ce1-bbd1-4a46-9645-33cc847b6ad5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:34,453 [548] [DEBUG] [app] Starting request: urn:request:b6663ce1-bbd1-4a46-9645-33cc847b6ad5 (/v1/_internal_ping) 2020-03-31 09:35:34,454 [548] [DEBUG] [app] Ending request: urn:request:b6663ce1-bbd1-4a46-9645-33cc847b6ad5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:34,454 [548] [DEBUG] [app] Ending request: urn:request:b6663ce1-bbd1-4a46-9645-33cc847b6ad5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:34,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:35:34,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:34,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:34,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:34,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:34,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:34,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:34,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:34,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:34,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:34,486 [518] [DEBUG] [app] Starting request: urn:request:644d4aed-8b5d-4954-9c50-466f0c67da7a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:34,486 [518] [DEBUG] [app] Starting request: urn:request:644d4aed-8b5d-4954-9c50-466f0c67da7a (/_internal_ping) 2020-03-31 09:35:34,487 [518] [DEBUG] [app] Ending request: urn:request:644d4aed-8b5d-4954-9c50-466f0c67da7a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:34,487 [518] [DEBUG] [app] Ending request: urn:request:644d4aed-8b5d-4954-9c50-466f0c67da7a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:35:34,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:34,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:34,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:34,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:34,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:35:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:35:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:34,501 [543] [DEBUG] [app] Starting request: urn:request:43b67be1-e9f6-4136-9c2b-e9271e499721 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:34,501 [543] [DEBUG] [app] Starting request: urn:request:43b67be1-e9f6-4136-9c2b-e9271e499721 (/c1/_internal_ping) 2020-03-31 09:35:34,502 [543] [DEBUG] [app] Ending request: urn:request:43b67be1-e9f6-4136-9c2b-e9271e499721 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:34,502 [543] [DEBUG] [app] Ending request: urn:request:43b67be1-e9f6-4136-9c2b-e9271e499721 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:34,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:35:34,503 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:34,504 [518] [DEBUG] [app] Ending request: urn:request:c8e6cfa9-cef3-4ea3-8050-3c5840eb0396 (/health/instance) 2020-03-31 09:35:34,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:34,504 [518] [DEBUG] [app] Ending request: urn:request:c8e6cfa9-cef3-4ea3-8050-3c5840eb0396 (/health/instance) 2020-03-31 09:35:34,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:34,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:35:34,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:35:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:35:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:38.010934+00:00 (in 1.085619 seconds) 2020-03-31 09:35:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:46 UTC)" (scheduled at 2020-03-31 09:35:36.924497+00:00) 2020-03-31 09:35:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:35:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:38.010934+00:00 (in 1.085619 seconds) 2020-03-31 09:35:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:46 UTC)" (scheduled at 2020-03-31 09:35:36.924497+00:00) 2020-03-31 09:35:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 36, 926167), True, datetime.datetime(2020, 3, 31, 9, 35, 36, 926167), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:35:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 36, 926167), True, datetime.datetime(2020, 3, 31, 9, 35, 36, 926167), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:35:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:36,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:35:36,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:36,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:36,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:46 UTC)" executed successfully 2020-03-31 09:35:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:35:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:46.924497+00:00 (in 8.912844 seconds) 2020-03-31 09:35:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:38 UTC)" (scheduled at 2020-03-31 09:35:38.010934+00:00) 2020-03-31 09:35:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:35:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:46.924497+00:00 (in 8.912844 seconds) 2020-03-31 09:35:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:38 UTC)" (scheduled at 2020-03-31 09:35:38.010934+00:00) 2020-03-31 09:35:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:38 UTC)" executed successfully 2020-03-31 09:35:38,272 [518] [DEBUG] [app] Starting request: urn:request:7cb09dca-3efb-43c3-8e22-338d92e3b84a (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:38,272 [518] [DEBUG] [app] Starting request: urn:request:7cb09dca-3efb-43c3-8e22-338d92e3b84a (/health/instance) 2020-03-31 09:35:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:38,278 [553] [DEBUG] [app] Starting request: urn:request:d2b59ee5-1c27-4243-b380-260b076a53b4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:38,278 [553] [DEBUG] [app] Starting request: urn:request:d2b59ee5-1c27-4243-b380-260b076a53b4 (/v1/_internal_ping) 2020-03-31 09:35:38,279 [553] [DEBUG] [app] Ending request: urn:request:d2b59ee5-1c27-4243-b380-260b076a53b4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:38,279 [553] [DEBUG] [app] Ending request: urn:request:d2b59ee5-1c27-4243-b380-260b076a53b4 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:38,279 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:38,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:35:38,279 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:38,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:38,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:38,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:38,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:38,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:38,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:38,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:38,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:38,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:38,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:38,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:38,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:38,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:38,310 [527] [DEBUG] [app] Starting request: urn:request:0186d9ce-5629-4333-9eff-b85841daab38 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:38,310 [527] [DEBUG] [app] Starting request: urn:request:0186d9ce-5629-4333-9eff-b85841daab38 (/_internal_ping) 2020-03-31 09:35:38,311 [527] [DEBUG] [app] Ending request: urn:request:0186d9ce-5629-4333-9eff-b85841daab38 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:38,311 [527] [DEBUG] [app] Ending request: urn:request:0186d9ce-5629-4333-9eff-b85841daab38 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:35:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:38,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:38,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:38,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:38,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:35:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:38,324 [536] [DEBUG] [app] Starting request: urn:request:7dd37359-9bfe-4d99-9230-feeea029f21c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:38,324 [536] [DEBUG] [app] Starting request: urn:request:7dd37359-9bfe-4d99-9230-feeea029f21c (/c1/_internal_ping) 2020-03-31 09:35:38,325 [536] [DEBUG] [app] Ending request: urn:request:7dd37359-9bfe-4d99-9230-feeea029f21c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:38,325 [536] [DEBUG] [app] Ending request: urn:request:7dd37359-9bfe-4d99-9230-feeea029f21c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:38,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:38,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:35:38,326 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:38,327 [518] [DEBUG] [app] Ending request: urn:request:7cb09dca-3efb-43c3-8e22-338d92e3b84a (/health/instance) 2020-03-31 09:35:38,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:38,327 [518] [DEBUG] [app] Ending request: urn:request:7cb09dca-3efb-43c3-8e22-338d92e3b84a (/health/instance) 2020-03-31 09:35:38,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:35:38,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:35:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:35:41,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:51.440707+00:00 (in 9.998097 seconds) 2020-03-31 09:35:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:51 UTC)" (scheduled at 2020-03-31 09:35:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:35:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:51.440707+00:00 (in 9.998097 seconds) 2020-03-31 09:35:41,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:51 UTC)" (scheduled at 2020-03-31 09:35:41.440707+00:00) 2020-03-31 09:35:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:35:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:35:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:35:41,463 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:35:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:35:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:35:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:51 UTC)" executed successfully 2020-03-31 09:35:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:35:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:43 UTC)" (scheduled at 2020-03-31 09:35:43.539999+00:00) 2020-03-31 09:35:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:33.366791+00:00 (in 49.826039 seconds) exportactionlogsworker stdout | 2020-03-31 09:35:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:35:43 UTC)" (scheduled at 2020-03-31 09:35:43.539999+00:00) 2020-03-31 09:35:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:33.366791+00:00 (in 49.826039 seconds) 2020-03-31 09:35:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:35:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:43 UTC)" executed successfully 2020-03-31 09:35:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:35:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:54 UTC)" (scheduled at 2020-03-31 09:35:44.323352+00:00) 2020-03-31 09:35:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:35:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:54 UTC)" (scheduled at 2020-03-31 09:35:44.323352+00:00) 2020-03-31 09:35:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:49.498966+00:00 (in 5.174552 seconds) notificationworker stdout | 2020-03-31 09:35:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:49.498966+00:00 (in 5.174552 seconds) 2020-03-31 09:35:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 44, 325320), True, datetime.datetime(2020, 3, 31, 9, 35, 44, 325320), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:35:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 44, 325320), True, datetime.datetime(2020, 3, 31, 9, 35, 44, 325320), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:35:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:35:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:35:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:54 UTC)" executed successfully 2020-03-31 09:35:44,446 [527] [DEBUG] [app] Starting request: urn:request:c9c07760-38aa-40a9-8422-4ac1f5658464 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:44,446 [527] [DEBUG] [app] Starting request: urn:request:c9c07760-38aa-40a9-8422-4ac1f5658464 (/health/instance) 2020-03-31 09:35:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:44,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:44,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:44,453 [544] [DEBUG] [app] Starting request: urn:request:153cb204-c40b-4019-9f4a-44c3c8d6ae99 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:44,453 [544] [DEBUG] [app] Starting request: urn:request:153cb204-c40b-4019-9f4a-44c3c8d6ae99 (/v1/_internal_ping) 2020-03-31 09:35:44,454 [544] [DEBUG] [app] Ending request: urn:request:153cb204-c40b-4019-9f4a-44c3c8d6ae99 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:44,454 [544] [DEBUG] [app] Ending request: urn:request:153cb204-c40b-4019-9f4a-44c3c8d6ae99 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:44,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:35:44,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:44,459 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:44,459 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:44,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:44,460 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:44,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:44,461 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:44,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:44,486 [527] [DEBUG] [app] Starting request: urn:request:3a7b40e5-2140-4edd-8f23-8a772f59500b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:44,486 [527] [DEBUG] [app] Starting request: urn:request:3a7b40e5-2140-4edd-8f23-8a772f59500b (/_internal_ping) 2020-03-31 09:35:44,487 [527] [DEBUG] [app] Ending request: urn:request:3a7b40e5-2140-4edd-8f23-8a772f59500b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:44,487 [527] [DEBUG] [app] Ending request: urn:request:3a7b40e5-2140-4edd-8f23-8a772f59500b (/_internal_ping) 2020-03-31 09:35:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:35:44,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:35:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:44,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:44,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:35:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:44,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:44,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:44,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:44,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:44,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:44,501 [537] [DEBUG] [app] Starting request: urn:request:80989d48-82c3-44e9-a1a0-604099ac561e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:44,501 [537] [DEBUG] [app] Starting request: urn:request:80989d48-82c3-44e9-a1a0-604099ac561e (/c1/_internal_ping) 2020-03-31 09:35:44,502 [537] [DEBUG] [app] Ending request: urn:request:80989d48-82c3-44e9-a1a0-604099ac561e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:44,502 [537] [DEBUG] [app] Ending request: urn:request:80989d48-82c3-44e9-a1a0-604099ac561e (/c1/_internal_ping) 2020-03-31 09:35:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:44,503 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:35:44,503 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:44,505 [527] [DEBUG] [app] Ending request: urn:request:c9c07760-38aa-40a9-8422-4ac1f5658464 (/health/instance) 2020-03-31 09:35:44,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:44,505 [527] [DEBUG] [app] Ending request: urn:request:c9c07760-38aa-40a9-8422-4ac1f5658464 (/health/instance) 2020-03-31 09:35:44,505 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:35:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:35:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:35:45,067 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:45,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:15.064514+00:00 (in 29.996822 seconds) 2020-03-31 09:35:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:15 UTC)" (scheduled at 2020-03-31 09:35:45.064514+00:00) gcworker stdout | 2020-03-31 09:35:45,067 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:15.064514+00:00 (in 29.996822 seconds) 2020-03-31 09:35:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:15 UTC)" (scheduled at 2020-03-31 09:35:45.064514+00:00) 2020-03-31 09:35:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437745, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:35:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437745, 1209600, 500, 1, 0]) 2020-03-31 09:35:45,087 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:35:45,087 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:35:45,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:35:45,087 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:15 UTC)" executed successfully 2020-03-31 09:35:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:35:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:56.924497+00:00 (in 9.999124 seconds) 2020-03-31 09:35:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:56 UTC)" (scheduled at 2020-03-31 09:35:46.924497+00:00) 2020-03-31 09:35:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:35:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:56.924497+00:00 (in 9.999124 seconds) 2020-03-31 09:35:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:56 UTC)" (scheduled at 2020-03-31 09:35:46.924497+00:00) 2020-03-31 09:35:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 46, 926072), True, datetime.datetime(2020, 3, 31, 9, 35, 46, 926072), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:35:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 46, 926072), True, datetime.datetime(2020, 3, 31, 9, 35, 46, 926072), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:35:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:35:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:35:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:46,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:35:56 UTC)" executed successfully 2020-03-31 09:35:48,273 [518] [DEBUG] [app] Starting request: urn:request:c0e36da4-1c49-4c43-826a-184d183590f5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:48,273 [518] [DEBUG] [app] Starting request: urn:request:c0e36da4-1c49-4c43-826a-184d183590f5 (/health/instance) 2020-03-31 09:35:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:48,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:48,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:48,282 [538] [DEBUG] [app] Starting request: urn:request:6ece004f-dd40-4ac7-9038-5944b8ac885b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:48,282 [538] [DEBUG] [app] Starting request: urn:request:6ece004f-dd40-4ac7-9038-5944b8ac885b (/v1/_internal_ping) 2020-03-31 09:35:48,284 [538] [DEBUG] [app] Ending request: urn:request:6ece004f-dd40-4ac7-9038-5944b8ac885b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:48,284 [538] [DEBUG] [app] Ending request: urn:request:6ece004f-dd40-4ac7-9038-5944b8ac885b (/v1/_internal_ping) 2020-03-31 09:35:48,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:35:48,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:48,286 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:35:48,287 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:48,286 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:48,287 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:48,289 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:48,289 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:48,291 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:48,291 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:48,292 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:48,292 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:48,301 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:48,301 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:48,315 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:48,315 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:48,320 [527] [DEBUG] [app] Starting request: urn:request:c4a00d73-921c-487b-b6c8-8a1b909c4676 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:48,320 [527] [DEBUG] [app] Starting request: urn:request:c4a00d73-921c-487b-b6c8-8a1b909c4676 (/_internal_ping) 2020-03-31 09:35:48,321 [527] [DEBUG] [app] Ending request: urn:request:c4a00d73-921c-487b-b6c8-8a1b909c4676 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:48,321 [527] [DEBUG] [app] Ending request: urn:request:c4a00d73-921c-487b-b6c8-8a1b909c4676 (/_internal_ping) 2020-03-31 09:35:48,322 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:35:48,322 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:35:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:48,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:35:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:48,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:48,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:48,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:35:48,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:48,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:48,334 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:48,336 [536] [DEBUG] [app] Starting request: urn:request:a7bfafc5-9ad5-45b9-b78b-667fbcdd34c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:48,336 [536] [DEBUG] [app] Starting request: urn:request:a7bfafc5-9ad5-45b9-b78b-667fbcdd34c9 (/c1/_internal_ping) 2020-03-31 09:35:48,337 [536] [DEBUG] [app] Ending request: urn:request:a7bfafc5-9ad5-45b9-b78b-667fbcdd34c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:48,337 [536] [DEBUG] [app] Ending request: urn:request:a7bfafc5-9ad5-45b9-b78b-667fbcdd34c9 (/c1/_internal_ping) 2020-03-31 09:35:48,337 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:35:48,337 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:48,337 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:35:48,337 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:48,339 [518] [DEBUG] [app] Ending request: urn:request:c0e36da4-1c49-4c43-826a-184d183590f5 (/health/instance) 2020-03-31 09:35:48,339 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:48,339 [518] [DEBUG] [app] Ending request: urn:request:c0e36da4-1c49-4c43-826a-184d183590f5 (/health/instance) 2020-03-31 09:35:48,339 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:48,340 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:35:48,340 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:35:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:35:49,502 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:54.323352+00:00 (in 4.820239 seconds) 2020-03-31 09:35:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:49 UTC)" (scheduled at 2020-03-31 09:35:49.498966+00:00) 2020-03-31 09:35:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:49,503 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:35:49,503 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:54.323352+00:00 (in 4.820239 seconds) 2020-03-31 09:35:49,503 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:49 UTC)" (scheduled at 2020-03-31 09:35:49.498966+00:00) 2020-03-31 09:35:49,503 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:49,503 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:49 UTC)" executed successfully 2020-03-31 09:35:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:35:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:59.560326+00:00 (in 8.448480 seconds) 2020-03-31 09:35:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:51 UTC)" (scheduled at 2020-03-31 09:35:51.110610+00:00) 2020-03-31 09:35:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:35:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:35:59.560326+00:00 (in 8.448480 seconds) 2020-03-31 09:35:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:51 UTC)" (scheduled at 2020-03-31 09:35:51.110610+00:00) 2020-03-31 09:35:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 51, 112776), True, datetime.datetime(2020, 3, 31, 9, 35, 51, 112776), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:35:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 51, 112776), True, datetime.datetime(2020, 3, 31, 9, 35, 51, 112776), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:35:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:35:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:35:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:51 UTC)" executed successfully 2020-03-31 09:35:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:35:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:01.440707+00:00 (in 9.998828 seconds) 2020-03-31 09:35:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:01 UTC)" (scheduled at 2020-03-31 09:35:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:35:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:01.440707+00:00 (in 9.998828 seconds) 2020-03-31 09:35:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:01 UTC)" (scheduled at 2020-03-31 09:35:51.440707+00:00) 2020-03-31 09:35:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:35:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:35:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:35:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:35:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:35:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:01 UTC)" executed successfully 2020-03-31 09:35:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:35:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:22.113221+00:00 (in 29.997906 seconds) 2020-03-31 09:35:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:22 UTC)" (scheduled at 2020-03-31 09:35:52.113221+00:00) securityworker stdout | 2020-03-31 09:35:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:22.113221+00:00 (in 29.997906 seconds) 2020-03-31 09:35:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:22 UTC)" (scheduled at 2020-03-31 09:35:52.113221+00:00) 2020-03-31 09:35:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:35:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:35:52,140 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:35:52,140 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:35:52,143 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:35:52,143 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:35:52,143 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:35:52,143 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:52,143 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:35:52,143 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:22 UTC)" executed successfully 2020-03-31 09:35:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:35:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:04.323352+00:00 (in 9.999254 seconds) 2020-03-31 09:35:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:04 UTC)" (scheduled at 2020-03-31 09:35:54.323352+00:00) 2020-03-31 09:35:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:35:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:04.323352+00:00 (in 9.999254 seconds) 2020-03-31 09:35:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:04 UTC)" (scheduled at 2020-03-31 09:35:54.323352+00:00) 2020-03-31 09:35:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 54, 324905), True, datetime.datetime(2020, 3, 31, 9, 35, 54, 324905), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:35:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 54, 324905), True, datetime.datetime(2020, 3, 31, 9, 35, 54, 324905), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:35:54,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:35:54,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:35:54,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:04 UTC)" executed successfully 2020-03-31 09:35:54,446 [518] [DEBUG] [app] Starting request: urn:request:d08c1478-ef25-42bd-ace9-e68e452b08d6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:54,446 [518] [DEBUG] [app] Starting request: urn:request:d08c1478-ef25-42bd-ace9-e68e452b08d6 (/health/instance) 2020-03-31 09:35:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:54,454 [553] [DEBUG] [app] Starting request: urn:request:38bae410-2ba8-4465-800b-3eabe9ac059e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:54,454 [553] [DEBUG] [app] Starting request: urn:request:38bae410-2ba8-4465-800b-3eabe9ac059e (/v1/_internal_ping) 2020-03-31 09:35:54,456 [553] [DEBUG] [app] Ending request: urn:request:38bae410-2ba8-4465-800b-3eabe9ac059e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:54,456 [553] [DEBUG] [app] Ending request: urn:request:38bae410-2ba8-4465-800b-3eabe9ac059e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:54,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:35:54,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:35:54,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:35:54,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:54,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:54,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:54,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:54,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:54,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:54,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:54,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:54,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:54,496 [518] [DEBUG] [app] Starting request: urn:request:e42449d4-8cd5-4057-94da-28f7fb555c9d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:54,496 [518] [DEBUG] [app] Starting request: urn:request:e42449d4-8cd5-4057-94da-28f7fb555c9d (/_internal_ping) 2020-03-31 09:35:54,497 [518] [DEBUG] [app] Ending request: urn:request:e42449d4-8cd5-4057-94da-28f7fb555c9d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:54,497 [518] [DEBUG] [app] Ending request: urn:request:e42449d4-8cd5-4057-94da-28f7fb555c9d (/_internal_ping) 2020-03-31 09:35:54,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:35:54,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:35:54,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:35:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:54,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:54,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:35:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:54,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:54,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:54,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:54,512 [539] [DEBUG] [app] Starting request: urn:request:887487ce-a4cc-41cf-8d39-bdc5da33dbc5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:54,512 [539] [DEBUG] [app] Starting request: urn:request:887487ce-a4cc-41cf-8d39-bdc5da33dbc5 (/c1/_internal_ping) 2020-03-31 09:35:54,513 [539] [DEBUG] [app] Ending request: urn:request:887487ce-a4cc-41cf-8d39-bdc5da33dbc5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:54,513 [539] [DEBUG] [app] Ending request: urn:request:887487ce-a4cc-41cf-8d39-bdc5da33dbc5 (/c1/_internal_ping) 2020-03-31 09:35:54,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:54,514 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:54,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:35:54,514 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:54,515 [518] [DEBUG] [app] Ending request: urn:request:d08c1478-ef25-42bd-ace9-e68e452b08d6 (/health/instance) 2020-03-31 09:35:54,515 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:54,515 [518] [DEBUG] [app] Ending request: urn:request:d08c1478-ef25-42bd-ace9-e68e452b08d6 (/health/instance) 2020-03-31 09:35:54,515 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:35:54,516 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:35:54,516 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:35:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:35:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:06.924497+00:00 (in 9.999231 seconds) 2020-03-31 09:35:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:06 UTC)" (scheduled at 2020-03-31 09:35:56.924497+00:00) 2020-03-31 09:35:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:35:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:06.924497+00:00 (in 9.999231 seconds) 2020-03-31 09:35:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:06 UTC)" (scheduled at 2020-03-31 09:35:56.924497+00:00) 2020-03-31 09:35:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:35:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 56, 925861), True, datetime.datetime(2020, 3, 31, 9, 35, 56, 925861), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:35:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 35, 56, 925861), True, datetime.datetime(2020, 3, 31, 9, 35, 56, 925861), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:35:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:35:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:35:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:35:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:35:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:06 UTC)" executed successfully 2020-03-31 09:35:58,273 [518] [DEBUG] [app] Starting request: urn:request:106f8ac8-2903-435e-9b3d-5127eb60fd36 (/health/instance) gunicorn-web stdout | 2020-03-31 09:35:58,273 [518] [DEBUG] [app] Starting request: urn:request:106f8ac8-2903-435e-9b3d-5127eb60fd36 (/health/instance) 2020-03-31 09:35:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:58,278 [538] [DEBUG] [app] Starting request: urn:request:94e94e4a-0598-44b7-b9fa-4ad60291af21 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:58,278 [538] [DEBUG] [app] Starting request: urn:request:94e94e4a-0598-44b7-b9fa-4ad60291af21 (/v1/_internal_ping) 2020-03-31 09:35:58,279 [538] [DEBUG] [app] Ending request: urn:request:94e94e4a-0598-44b7-b9fa-4ad60291af21 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:35:58,279 [538] [DEBUG] [app] Ending request: urn:request:94e94e4a-0598-44b7-b9fa-4ad60291af21 (/v1/_internal_ping) 2020-03-31 09:35:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:35:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:35:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:35:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:35:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:35:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:35:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:35:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:35:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:35:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:35:58,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:35:58,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:35:58,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:35:58,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:35:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:58,311 [518] [DEBUG] [app] Starting request: urn:request:d8d6e081-5d31-4dff-9ae4-db2a5a8bce9e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:58,311 [518] [DEBUG] [app] Starting request: urn:request:d8d6e081-5d31-4dff-9ae4-db2a5a8bce9e (/_internal_ping) 2020-03-31 09:35:58,311 [518] [DEBUG] [app] Ending request: urn:request:d8d6e081-5d31-4dff-9ae4-db2a5a8bce9e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:35:58,311 [518] [DEBUG] [app] Ending request: urn:request:d8d6e081-5d31-4dff-9ae4-db2a5a8bce9e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:35:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:35:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:35:58,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:58,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:35:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:35:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:35:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:35:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:35:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:35:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:35:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:35:58,325 [539] [DEBUG] [app] Starting request: urn:request:00e2e614-3060-49f6-be49-5a1f8de99c32 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:58,325 [539] [DEBUG] [app] Starting request: urn:request:00e2e614-3060-49f6-be49-5a1f8de99c32 (/c1/_internal_ping) 2020-03-31 09:35:58,326 [539] [DEBUG] [app] Ending request: urn:request:00e2e614-3060-49f6-be49-5a1f8de99c32 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:35:58,326 [539] [DEBUG] [app] Ending request: urn:request:00e2e614-3060-49f6-be49-5a1f8de99c32 (/c1/_internal_ping) 2020-03-31 09:35:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:35:58,327 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:35:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:35:58,327 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:35:58,328 [518] [DEBUG] [app] Ending request: urn:request:106f8ac8-2903-435e-9b3d-5127eb60fd36 (/health/instance) 2020-03-31 09:35:58,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:35:58,328 [518] [DEBUG] [app] Ending request: urn:request:106f8ac8-2903-435e-9b3d-5127eb60fd36 (/health/instance) 2020-03-31 09:35:58,328 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:35:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:35:58,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:35:58,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:35:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:35:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:35:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:35:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:59 UTC)" (scheduled at 2020-03-31 09:35:59.560326+00:00) 2020-03-31 09:35:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:51.110610+00:00 (in 51.549253 seconds) namespacegcworker stdout | 2020-03-31 09:35:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:59 UTC)" (scheduled at 2020-03-31 09:35:59.560326+00:00) 2020-03-31 09:35:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:35:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:51.110610+00:00 (in 51.549253 seconds) 2020-03-31 09:35:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:35:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:36:59 UTC)" executed successfully 2020-03-31 09:36:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:36:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:30.337699+00:00 (in 29.999238 seconds) 2020-03-31 09:36:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:30 UTC)" (scheduled at 2020-03-31 09:36:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:36:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:30.337699+00:00 (in 29.999238 seconds) 2020-03-31 09:36:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:30 UTC)" (scheduled at 2020-03-31 09:36:00.337699+00:00) 2020-03-31 09:36:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 36, 0, 339341), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:36:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 36, 0, 339341), False, 50, 1, 0]) 2020-03-31 09:36:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:36:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:36:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:36:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:30 UTC)" executed successfully 2020-03-31 09:36:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:36:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:11.440707+00:00 (in 9.999081 seconds) 2020-03-31 09:36:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:11 UTC)" (scheduled at 2020-03-31 09:36:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:36:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:11.440707+00:00 (in 9.999081 seconds) 2020-03-31 09:36:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:11 UTC)" (scheduled at 2020-03-31 09:36:01.440707+00:00) 2020-03-31 09:36:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:36:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:36:01,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:36:01,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:36:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:11 UTC)" executed successfully 2020-03-31 09:36:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:36:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:24.410927+00:00 (in 21.079820 seconds) 2020-03-31 09:36:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:33 UTC)" (scheduled at 2020-03-31 09:36:03.330112+00:00) 2020-03-31 09:36:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:36:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:24.410927+00:00 (in 21.079820 seconds) 2020-03-31 09:36:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:33 UTC)" (scheduled at 2020-03-31 09:36:03.330112+00:00) 2020-03-31 09:36:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 3, 331964), True, datetime.datetime(2020, 3, 31, 9, 36, 3, 331964), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:36:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 3, 331964), True, datetime.datetime(2020, 3, 31, 9, 36, 3, 331964), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:36:03,355 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:36:03,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:36:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:33 UTC)" executed successfully 2020-03-31 09:36:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:36:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:14.323352+00:00 (in 9.999303 seconds) 2020-03-31 09:36:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:14 UTC)" (scheduled at 2020-03-31 09:36:04.323352+00:00) 2020-03-31 09:36:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:36:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:14.323352+00:00 (in 9.999303 seconds) 2020-03-31 09:36:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:14 UTC)" (scheduled at 2020-03-31 09:36:04.323352+00:00) 2020-03-31 09:36:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 4, 324719), True, datetime.datetime(2020, 3, 31, 9, 36, 4, 324719), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:36:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 4, 324719), True, datetime.datetime(2020, 3, 31, 9, 36, 4, 324719), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:36:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:36:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:36:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:14 UTC)" executed successfully 2020-03-31 09:36:04,446 [527] [DEBUG] [app] Starting request: urn:request:971001dd-ffe2-4af5-9467-65d83e217fd1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:04,446 [527] [DEBUG] [app] Starting request: urn:request:971001dd-ffe2-4af5-9467-65d83e217fd1 (/health/instance) 2020-03-31 09:36:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:04,454 [552] [DEBUG] [app] Starting request: urn:request:747d8287-1a69-4a05-baf6-a056ab079380 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:04,454 [552] [DEBUG] [app] Starting request: urn:request:747d8287-1a69-4a05-baf6-a056ab079380 (/v1/_internal_ping) 2020-03-31 09:36:04,456 [552] [DEBUG] [app] Ending request: urn:request:747d8287-1a69-4a05-baf6-a056ab079380 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:04,456 [552] [DEBUG] [app] Ending request: urn:request:747d8287-1a69-4a05-baf6-a056ab079380 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:04,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:04,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:04,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:36:04,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:04,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:36:04,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:04,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:04,461 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:04,459 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:04,461 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:04,462 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:04,462 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:04,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:04,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:04,490 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:04,490 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:04,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:04,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:04,496 [527] [DEBUG] [app] Starting request: urn:request:bc246b2a-64ae-4418-b28d-01379a4d5264 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:04,496 [527] [DEBUG] [app] Starting request: urn:request:bc246b2a-64ae-4418-b28d-01379a4d5264 (/_internal_ping) 2020-03-31 09:36:04,496 [527] [DEBUG] [app] Ending request: urn:request:bc246b2a-64ae-4418-b28d-01379a4d5264 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:04,496 [527] [DEBUG] [app] Ending request: urn:request:bc246b2a-64ae-4418-b28d-01379a4d5264 (/_internal_ping) 2020-03-31 09:36:04,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:04,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:04,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:36:04,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:36:04,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:04,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:04,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:36:04,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:04,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:04,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:04,509 [537] [DEBUG] [app] Starting request: urn:request:a677d03f-3ffc-4ead-9d93-7450cb1e8ec0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:04,509 [537] [DEBUG] [app] Starting request: urn:request:a677d03f-3ffc-4ead-9d93-7450cb1e8ec0 (/c1/_internal_ping) 2020-03-31 09:36:04,510 [537] [DEBUG] [app] Ending request: urn:request:a677d03f-3ffc-4ead-9d93-7450cb1e8ec0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:04,510 [537] [DEBUG] [app] Ending request: urn:request:a677d03f-3ffc-4ead-9d93-7450cb1e8ec0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:04,511 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:04,511 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:36:04,511 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:04,511 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:04,512 [527] [DEBUG] [app] Ending request: urn:request:971001dd-ffe2-4af5-9467-65d83e217fd1 (/health/instance) 2020-03-31 09:36:04,512 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:04,512 [527] [DEBUG] [app] Ending request: urn:request:971001dd-ffe2-4af5-9467-65d83e217fd1 (/health/instance) 2020-03-31 09:36:04,512 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:04,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:36:04,513 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:36:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:36:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:16.924497+00:00 (in 9.999284 seconds) 2020-03-31 09:36:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:16 UTC)" (scheduled at 2020-03-31 09:36:06.924497+00:00) 2020-03-31 09:36:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:36:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:16.924497+00:00 (in 9.999284 seconds) 2020-03-31 09:36:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:16 UTC)" (scheduled at 2020-03-31 09:36:06.924497+00:00) 2020-03-31 09:36:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 6, 925988), True, datetime.datetime(2020, 3, 31, 9, 36, 6, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:36:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 6, 925988), True, datetime.datetime(2020, 3, 31, 9, 36, 6, 925988), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:36:06,951 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:36:06,951 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:06,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:06,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:36:06,951 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:06,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:16 UTC)" executed successfully 2020-03-31 09:36:08,273 [518] [DEBUG] [app] Starting request: urn:request:78c912d8-2296-461f-81b9-c0081f9ad46d (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:08,273 [518] [DEBUG] [app] Starting request: urn:request:78c912d8-2296-461f-81b9-c0081f9ad46d (/health/instance) 2020-03-31 09:36:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:08,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:08,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:08,282 [555] [DEBUG] [app] Starting request: urn:request:cb0b52ef-5736-47ef-9a24-61d7cdd5000f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:08,282 [555] [DEBUG] [app] Starting request: urn:request:cb0b52ef-5736-47ef-9a24-61d7cdd5000f (/v1/_internal_ping) 2020-03-31 09:36:08,282 [555] [DEBUG] [app] Ending request: urn:request:cb0b52ef-5736-47ef-9a24-61d7cdd5000f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:08,282 [555] [DEBUG] [app] Ending request: urn:request:cb0b52ef-5736-47ef-9a24-61d7cdd5000f (/v1/_internal_ping) 2020-03-31 09:36:08,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:36:08,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:08,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:08,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:08,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:08,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:08,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:08,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:08,297 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:08,297 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:08,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:08,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:08,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:08,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:08,318 [520] [DEBUG] [app] Starting request: urn:request:ab211117-399e-42ae-9dbc-8c00bfea4a7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:08,318 [520] [DEBUG] [app] Starting request: urn:request:ab211117-399e-42ae-9dbc-8c00bfea4a7b (/_internal_ping) 2020-03-31 09:36:08,319 [520] [DEBUG] [app] Ending request: urn:request:ab211117-399e-42ae-9dbc-8c00bfea4a7b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:08,319 [520] [DEBUG] [app] Ending request: urn:request:ab211117-399e-42ae-9dbc-8c00bfea4a7b (/_internal_ping) 2020-03-31 09:36:08,319 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:08,319 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:36:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:36:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:08,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:08,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:08,332 [535] [DEBUG] [app] Starting request: urn:request:df44cffa-ee55-42f0-a3b5-cd965ac24111 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:08,332 [535] [DEBUG] [app] Starting request: urn:request:df44cffa-ee55-42f0-a3b5-cd965ac24111 (/c1/_internal_ping) 2020-03-31 09:36:08,333 [535] [DEBUG] [app] Ending request: urn:request:df44cffa-ee55-42f0-a3b5-cd965ac24111 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:08,333 [535] [DEBUG] [app] Ending request: urn:request:df44cffa-ee55-42f0-a3b5-cd965ac24111 (/c1/_internal_ping) 2020-03-31 09:36:08,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:08,334 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:08,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:36:08,334 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:08,336 [518] [DEBUG] [app] Ending request: urn:request:78c912d8-2296-461f-81b9-c0081f9ad46d (/health/instance) 2020-03-31 09:36:08,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:08,336 [518] [DEBUG] [app] Ending request: urn:request:78c912d8-2296-461f-81b9-c0081f9ad46d (/health/instance) 2020-03-31 09:36:08,336 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:36:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:36:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:36:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:36:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:21.440707+00:00 (in 9.999247 seconds) 2020-03-31 09:36:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:21 UTC)" (scheduled at 2020-03-31 09:36:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:36:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:21.440707+00:00 (in 9.999247 seconds) 2020-03-31 09:36:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:21 UTC)" (scheduled at 2020-03-31 09:36:11.440707+00:00) 2020-03-31 09:36:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:36:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:36:11,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:36:11,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:36:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:21 UTC)" executed successfully 2020-03-31 09:36:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:36:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:24.323352+00:00 (in 9.999223 seconds) 2020-03-31 09:36:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:24 UTC)" (scheduled at 2020-03-31 09:36:14.323352+00:00) 2020-03-31 09:36:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:36:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:24.323352+00:00 (in 9.999223 seconds) 2020-03-31 09:36:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:24 UTC)" (scheduled at 2020-03-31 09:36:14.323352+00:00) 2020-03-31 09:36:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 14, 325082), True, datetime.datetime(2020, 3, 31, 9, 36, 14, 325082), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:36:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 14, 325082), True, datetime.datetime(2020, 3, 31, 9, 36, 14, 325082), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:36:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:36:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:36:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:24 UTC)" executed successfully 2020-03-31 09:36:14,446 [526] [DEBUG] [app] Starting request: urn:request:48d0e799-7da5-4a02-927b-8d8be3fc653a (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:14,446 [526] [DEBUG] [app] Starting request: urn:request:48d0e799-7da5-4a02-927b-8d8be3fc653a (/health/instance) 2020-03-31 09:36:14,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:14,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:14,453 [540] [DEBUG] [app] Starting request: urn:request:81347335-ac16-4d44-a7bf-dcb4267e93d6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:14,453 [540] [DEBUG] [app] Starting request: urn:request:81347335-ac16-4d44-a7bf-dcb4267e93d6 (/v1/_internal_ping) 2020-03-31 09:36:14,454 [540] [DEBUG] [app] Ending request: urn:request:81347335-ac16-4d44-a7bf-dcb4267e93d6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:14,454 [540] [DEBUG] [app] Ending request: urn:request:81347335-ac16-4d44-a7bf-dcb4267e93d6 (/v1/_internal_ping) 2020-03-31 09:36:14,454 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:36:14,454 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:14,454 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:14,455 [526] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:36:14,454 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:14,455 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:14,457 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:14,457 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:14,458 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:14,458 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:14,459 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:14,459 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:14,496 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:14,496 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:14,501 [526] [DEBUG] [app] Starting request: urn:request:5e1171f3-377f-4978-b9d6-7ebfbf53e36c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:14,501 [526] [DEBUG] [app] Starting request: urn:request:5e1171f3-377f-4978-b9d6-7ebfbf53e36c (/_internal_ping) 2020-03-31 09:36:14,502 [526] [DEBUG] [app] Ending request: urn:request:5e1171f3-377f-4978-b9d6-7ebfbf53e36c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:14,502 [526] [DEBUG] [app] Ending request: urn:request:5e1171f3-377f-4978-b9d6-7ebfbf53e36c (/_internal_ping) 2020-03-31 09:36:14,503 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:14,503 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:14,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:14,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:14,507 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:14,507 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:14,508 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:36:14,508 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:36:14,511 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:14,511 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:36:14,512 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:14,512 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:14,515 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:14,515 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:14,517 [537] [DEBUG] [app] Starting request: urn:request:a54b3795-e43a-4fad-92ca-a559c5b6e38a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:14,517 [537] [DEBUG] [app] Starting request: urn:request:a54b3795-e43a-4fad-92ca-a559c5b6e38a (/c1/_internal_ping) 2020-03-31 09:36:14,517 [537] [DEBUG] [app] Ending request: urn:request:a54b3795-e43a-4fad-92ca-a559c5b6e38a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:14,517 [537] [DEBUG] [app] Ending request: urn:request:a54b3795-e43a-4fad-92ca-a559c5b6e38a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:14,518 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:14,518 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:14,518 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:36:14,518 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:14,519 [526] [DEBUG] [app] Ending request: urn:request:48d0e799-7da5-4a02-927b-8d8be3fc653a (/health/instance) 2020-03-31 09:36:14,519 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:14,519 [526] [DEBUG] [app] Ending request: urn:request:48d0e799-7da5-4a02-927b-8d8be3fc653a (/health/instance) 2020-03-31 09:36:14,519 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:14,520 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.076 49 0.076) gunicorn-web stdout | 2020-03-31 09:36:14,520 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.076 49 0.076) 2020-03-31 09:36:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:36:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:45.064514+00:00 (in 29.999231 seconds) 2020-03-31 09:36:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:45 UTC)" (scheduled at 2020-03-31 09:36:15.064514+00:00) gcworker stdout | 2020-03-31 09:36:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:45.064514+00:00 (in 29.999231 seconds) 2020-03-31 09:36:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:45 UTC)" (scheduled at 2020-03-31 09:36:15.064514+00:00) 2020-03-31 09:36:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437775, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:36:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437775, 1209600, 500, 1, 0]) 2020-03-31 09:36:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:36:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:36:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:36:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:45 UTC)" executed successfully 2020-03-31 09:36:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:36:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:26.924497+00:00 (in 9.999248 seconds) 2020-03-31 09:36:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:26 UTC)" (scheduled at 2020-03-31 09:36:16.924497+00:00) 2020-03-31 09:36:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:36:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:26.924497+00:00 (in 9.999248 seconds) 2020-03-31 09:36:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:26 UTC)" (scheduled at 2020-03-31 09:36:16.924497+00:00) 2020-03-31 09:36:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 16, 925938), True, datetime.datetime(2020, 3, 31, 9, 36, 16, 925938), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:36:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 16, 925938), True, datetime.datetime(2020, 3, 31, 9, 36, 16, 925938), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:36:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:16,948 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:36:16,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:16,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:16,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:36:16,949 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:26 UTC)" executed successfully 2020-03-31 09:36:18,272 [527] [DEBUG] [app] Starting request: urn:request:d39c9e15-fce1-4d3e-bef7-b0c1aa87fd04 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:18,272 [527] [DEBUG] [app] Starting request: urn:request:d39c9e15-fce1-4d3e-bef7-b0c1aa87fd04 (/health/instance) 2020-03-31 09:36:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:18,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:18,281 [555] [DEBUG] [app] Starting request: urn:request:88c5b199-bc6f-4707-9f4f-de60d355b168 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:18,281 [555] [DEBUG] [app] Starting request: urn:request:88c5b199-bc6f-4707-9f4f-de60d355b168 (/v1/_internal_ping) 2020-03-31 09:36:18,282 [555] [DEBUG] [app] Ending request: urn:request:88c5b199-bc6f-4707-9f4f-de60d355b168 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:18,282 [555] [DEBUG] [app] Ending request: urn:request:88c5b199-bc6f-4707-9f4f-de60d355b168 (/v1/_internal_ping) 2020-03-31 09:36:18,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:36:18,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:18,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:18,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:36:18,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:36:18,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:18,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:18,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:18,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:18,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:18,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:18,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:18,312 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:18,312 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:18,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:18,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:18,316 [527] [DEBUG] [app] Starting request: urn:request:2d76fa35-7500-4494-abcf-e8a062f97dcb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:18,316 [527] [DEBUG] [app] Starting request: urn:request:2d76fa35-7500-4494-abcf-e8a062f97dcb (/_internal_ping) 2020-03-31 09:36:18,317 [527] [DEBUG] [app] Ending request: urn:request:2d76fa35-7500-4494-abcf-e8a062f97dcb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:18,317 [527] [DEBUG] [app] Ending request: urn:request:2d76fa35-7500-4494-abcf-e8a062f97dcb (/_internal_ping) 2020-03-31 09:36:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:18,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:18,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:18,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:18,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:18,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:18,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:18,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:18,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:18,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:18,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:18,330 [535] [DEBUG] [app] Starting request: urn:request:4d5fea10-b36f-4de4-a460-4e83a60d63b8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:18,330 [535] [DEBUG] [app] Starting request: urn:request:4d5fea10-b36f-4de4-a460-4e83a60d63b8 (/c1/_internal_ping) 2020-03-31 09:36:18,332 [535] [DEBUG] [app] Ending request: urn:request:4d5fea10-b36f-4de4-a460-4e83a60d63b8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:18,332 [535] [DEBUG] [app] Ending request: urn:request:4d5fea10-b36f-4de4-a460-4e83a60d63b8 (/c1/_internal_ping) 2020-03-31 09:36:18,332 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:36:18,332 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:36:18,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:18,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:18,334 [527] [DEBUG] [app] Ending request: urn:request:d39c9e15-fce1-4d3e-bef7-b0c1aa87fd04 (/health/instance) 2020-03-31 09:36:18,334 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:18,334 [527] [DEBUG] [app] Ending request: urn:request:d39c9e15-fce1-4d3e-bef7-b0c1aa87fd04 (/health/instance) 2020-03-31 09:36:18,334 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:18,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:36:18,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:36:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:36:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:31.440707+00:00 (in 9.999145 seconds) 2020-03-31 09:36:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:31 UTC)" (scheduled at 2020-03-31 09:36:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:36:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:31.440707+00:00 (in 9.999145 seconds) 2020-03-31 09:36:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:31 UTC)" (scheduled at 2020-03-31 09:36:21.440707+00:00) 2020-03-31 09:36:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:36:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:36:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:21,460 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:36:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:36:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:31 UTC)" executed successfully 2020-03-31 09:36:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:36:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:52.113221+00:00 (in 29.999208 seconds) 2020-03-31 09:36:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:52 UTC)" (scheduled at 2020-03-31 09:36:22.113221+00:00) securityworker stdout | 2020-03-31 09:36:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:52.113221+00:00 (in 29.999208 seconds) 2020-03-31 09:36:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:52 UTC)" (scheduled at 2020-03-31 09:36:22.113221+00:00) 2020-03-31 09:36:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:36:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:36:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:36:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:36:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:36:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:36:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:36:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:36:52 UTC)" executed successfully 2020-03-31 09:36:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:36:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:34.323352+00:00 (in 9.998957 seconds) 2020-03-31 09:36:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:34 UTC)" (scheduled at 2020-03-31 09:36:24.323352+00:00) 2020-03-31 09:36:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:36:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:34.323352+00:00 (in 9.998957 seconds) 2020-03-31 09:36:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:34 UTC)" (scheduled at 2020-03-31 09:36:24.323352+00:00) 2020-03-31 09:36:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 24, 325087), True, datetime.datetime(2020, 3, 31, 9, 36, 24, 325087), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:36:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 24, 325087), True, datetime.datetime(2020, 3, 31, 9, 36, 24, 325087), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:36:24,352 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:36:24,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:24,353 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:24,353 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:36:24,353 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:24,353 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:34 UTC)" executed successfully 2020-03-31 09:36:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:36:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:24 UTC)" (scheduled at 2020-03-31 09:36:24.410927+00:00) security_notification_worker stdout | 2020-03-31 09:36:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:24 UTC)" (scheduled at 2020-03-31 09:36:24.410927+00:00) 2020-03-31 09:36:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:36:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:33.330112+00:00 (in 8.918454 seconds) 2020-03-31 09:36:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:36:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:36:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:33.330112+00:00 (in 8.918454 seconds) 2020-03-31 09:36:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:24 UTC)" executed successfully 2020-03-31 09:36:24,446 [518] [DEBUG] [app] Starting request: urn:request:902aa6a8-020d-4e65-b8aa-365186f04e56 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:24,446 [518] [DEBUG] [app] Starting request: urn:request:902aa6a8-020d-4e65-b8aa-365186f04e56 (/health/instance) 2020-03-31 09:36:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:24,453 [538] [DEBUG] [app] Starting request: urn:request:714ea36c-ce85-4840-b47f-9a35261846d6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:24,453 [538] [DEBUG] [app] Starting request: urn:request:714ea36c-ce85-4840-b47f-9a35261846d6 (/v1/_internal_ping) 2020-03-31 09:36:24,454 [538] [DEBUG] [app] Ending request: urn:request:714ea36c-ce85-4840-b47f-9a35261846d6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:24,454 [538] [DEBUG] [app] Ending request: urn:request:714ea36c-ce85-4840-b47f-9a35261846d6 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:36:24,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:24,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:24,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:24,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:24,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:24,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:24,488 [527] [DEBUG] [app] Starting request: urn:request:f0e2a63b-5999-4e0f-b51b-0b223e7062f5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:24,488 [527] [DEBUG] [app] Starting request: urn:request:f0e2a63b-5999-4e0f-b51b-0b223e7062f5 (/_internal_ping) 2020-03-31 09:36:24,489 [527] [DEBUG] [app] Ending request: urn:request:f0e2a63b-5999-4e0f-b51b-0b223e7062f5 (/_internal_ping) 2020-03-31 09:36:24,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:24,489 [527] [DEBUG] [app] Ending request: urn:request:f0e2a63b-5999-4e0f-b51b-0b223e7062f5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:24,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:36:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:36:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:24,501 [535] [DEBUG] [app] Starting request: urn:request:75fd3ec7-e5ff-4656-b0b2-7b069cfd1b4c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:24,501 [535] [DEBUG] [app] Starting request: urn:request:75fd3ec7-e5ff-4656-b0b2-7b069cfd1b4c (/c1/_internal_ping) 2020-03-31 09:36:24,502 [535] [DEBUG] [app] Ending request: urn:request:75fd3ec7-e5ff-4656-b0b2-7b069cfd1b4c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:24,502 [535] [DEBUG] [app] Ending request: urn:request:75fd3ec7-e5ff-4656-b0b2-7b069cfd1b4c (/c1/_internal_ping) 2020-03-31 09:36:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:24,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:36:24,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:24,504 [518] [DEBUG] [app] Ending request: urn:request:902aa6a8-020d-4e65-b8aa-365186f04e56 (/health/instance) 2020-03-31 09:36:24,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:24,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:36:24,504 [518] [DEBUG] [app] Ending request: urn:request:902aa6a8-020d-4e65-b8aa-365186f04e56 (/health/instance) 2020-03-31 09:36:24,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:24,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:36:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:36:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:36.924497+00:00 (in 9.999316 seconds) 2020-03-31 09:36:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:36 UTC)" (scheduled at 2020-03-31 09:36:26.924497+00:00) 2020-03-31 09:36:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:36:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:36.924497+00:00 (in 9.999316 seconds) 2020-03-31 09:36:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:36 UTC)" (scheduled at 2020-03-31 09:36:26.924497+00:00) 2020-03-31 09:36:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 26, 925883), True, datetime.datetime(2020, 3, 31, 9, 36, 26, 925883), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:36:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 26, 925883), True, datetime.datetime(2020, 3, 31, 9, 36, 26, 925883), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:36:26,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:36:26,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:36:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:36 UTC)" executed successfully 2020-03-31 09:36:28,273 [526] [DEBUG] [app] Starting request: urn:request:cb3648a5-1d65-4300-8c2b-67dd33a717c3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:28,273 [526] [DEBUG] [app] Starting request: urn:request:cb3648a5-1d65-4300-8c2b-67dd33a717c3 (/health/instance) 2020-03-31 09:36:28,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:28,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:28,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:28,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:28,281 [547] [DEBUG] [app] Starting request: urn:request:3e17f1d8-1536-498c-9dcf-b8f1ef2cefd7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:28,281 [547] [DEBUG] [app] Starting request: urn:request:3e17f1d8-1536-498c-9dcf-b8f1ef2cefd7 (/v1/_internal_ping) 2020-03-31 09:36:28,282 [547] [DEBUG] [app] Ending request: urn:request:3e17f1d8-1536-498c-9dcf-b8f1ef2cefd7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:28,282 [547] [DEBUG] [app] Ending request: urn:request:3e17f1d8-1536-498c-9dcf-b8f1ef2cefd7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:28,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:28,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:36:28,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:28,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:28,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:28,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:28,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:28,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:28,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:28,286 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:28,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:28,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:28,303 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:28,303 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:28,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:28,315 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:28,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:28,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:28,321 [526] [DEBUG] [app] Starting request: urn:request:eab93a6c-5db4-41ad-8d4a-e38b38ccd8ff (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:28,321 [526] [DEBUG] [app] Starting request: urn:request:eab93a6c-5db4-41ad-8d4a-e38b38ccd8ff (/_internal_ping) 2020-03-31 09:36:28,321 [526] [DEBUG] [app] Ending request: urn:request:eab93a6c-5db4-41ad-8d4a-e38b38ccd8ff (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:28,321 [526] [DEBUG] [app] Ending request: urn:request:eab93a6c-5db4-41ad-8d4a-e38b38ccd8ff (/_internal_ping) 2020-03-31 09:36:28,322 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:28,322 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:28,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:28,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:28,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:28,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:28,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:28,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:28,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:28,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:28,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:28,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:28,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:28,333 [537] [DEBUG] [app] Starting request: urn:request:8483443a-c7b9-4c29-9853-80c284f4d54e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:28,333 [537] [DEBUG] [app] Starting request: urn:request:8483443a-c7b9-4c29-9853-80c284f4d54e (/c1/_internal_ping) 2020-03-31 09:36:28,334 [537] [DEBUG] [app] Ending request: urn:request:8483443a-c7b9-4c29-9853-80c284f4d54e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:28,334 [537] [DEBUG] [app] Ending request: urn:request:8483443a-c7b9-4c29-9853-80c284f4d54e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:28,335 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:28,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:28,335 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:36:28,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:28,336 [526] [DEBUG] [app] Ending request: urn:request:cb3648a5-1d65-4300-8c2b-67dd33a717c3 (/health/instance) 2020-03-31 09:36:28,337 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:28,336 [526] [DEBUG] [app] Ending request: urn:request:cb3648a5-1d65-4300-8c2b-67dd33a717c3 (/health/instance) 2020-03-31 09:36:28,337 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:28,337 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:36:28,337 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:36:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:36:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:00.337699+00:00 (in 29.999005 seconds) 2020-03-31 09:36:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:00 UTC)" (scheduled at 2020-03-31 09:36:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:36:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:00.337699+00:00 (in 29.999005 seconds) 2020-03-31 09:36:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:00 UTC)" (scheduled at 2020-03-31 09:36:30.337699+00:00) 2020-03-31 09:36:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 36, 30, 339554), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:36:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 36, 30, 339554), False, 50, 1, 0]) 2020-03-31 09:36:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:36:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:36:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:36:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:00 UTC)" executed successfully 2020-03-31 09:36:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:36:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:41 UTC)" (scheduled at 2020-03-31 09:36:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:36:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:41 UTC)" (scheduled at 2020-03-31 09:36:31.440707+00:00) 2020-03-31 09:36:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:36:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:36:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:41.440707+00:00 (in 9.999168 seconds) repositoryactioncounter stdout | 2020-03-31 09:36:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:41.440707+00:00 (in 9.999168 seconds) 2020-03-31 09:36:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:36:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:31,461 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:36:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:36:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:41 UTC)" executed successfully 2020-03-31 09:36:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:36:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:03.330112+00:00 (in 29.998583 seconds) 2020-03-31 09:36:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:03 UTC)" (scheduled at 2020-03-31 09:36:33.330112+00:00) 2020-03-31 09:36:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:36:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:03.330112+00:00 (in 29.998583 seconds) 2020-03-31 09:36:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:03 UTC)" (scheduled at 2020-03-31 09:36:33.330112+00:00) 2020-03-31 09:36:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 33, 332452), True, datetime.datetime(2020, 3, 31, 9, 36, 33, 332452), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:36:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 33, 332452), True, datetime.datetime(2020, 3, 31, 9, 36, 33, 332452), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:36:33,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:36:33,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:33,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:36:33,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:03 UTC)" executed successfully 2020-03-31 09:36:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:36:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:43.539999+00:00 (in 10.171616 seconds) 2020-03-31 09:36:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:33 UTC)" (scheduled at 2020-03-31 09:36:33.366791+00:00) 2020-03-31 09:36:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:36:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:43.539999+00:00 (in 10.171616 seconds) 2020-03-31 09:36:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:33 UTC)" (scheduled at 2020-03-31 09:36:33.366791+00:00) 2020-03-31 09:36:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 33, 369175), True, datetime.datetime(2020, 3, 31, 9, 36, 33, 369175), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:36:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 33, 369175), True, datetime.datetime(2020, 3, 31, 9, 36, 33, 369175), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:36:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:36:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:36:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:33 UTC)" executed successfully 2020-03-31 09:36:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:36:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:44.323352+00:00 (in 9.999313 seconds) 2020-03-31 09:36:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:44 UTC)" (scheduled at 2020-03-31 09:36:34.323352+00:00) 2020-03-31 09:36:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:36:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:44.323352+00:00 (in 9.999313 seconds) 2020-03-31 09:36:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:44 UTC)" (scheduled at 2020-03-31 09:36:34.323352+00:00) 2020-03-31 09:36:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 34, 325044), True, datetime.datetime(2020, 3, 31, 9, 36, 34, 325044), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:36:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 34, 325044), True, datetime.datetime(2020, 3, 31, 9, 36, 34, 325044), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:36:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:36:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:36:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:44 UTC)" executed successfully 2020-03-31 09:36:34,446 [518] [DEBUG] [app] Starting request: urn:request:485b9d08-bb6a-4b93-9be5-e86b2a6ddbd5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:34,446 [518] [DEBUG] [app] Starting request: urn:request:485b9d08-bb6a-4b93-9be5-e86b2a6ddbd5 (/health/instance) 2020-03-31 09:36:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:34,453 [538] [DEBUG] [app] Starting request: urn:request:d76bba3d-b800-472d-a1b1-a108d2c05f5f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:34,453 [538] [DEBUG] [app] Starting request: urn:request:d76bba3d-b800-472d-a1b1-a108d2c05f5f (/v1/_internal_ping) 2020-03-31 09:36:34,454 [538] [DEBUG] [app] Ending request: urn:request:d76bba3d-b800-472d-a1b1-a108d2c05f5f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:34,454 [538] [DEBUG] [app] Ending request: urn:request:d76bba3d-b800-472d-a1b1-a108d2c05f5f (/v1/_internal_ping) 2020-03-31 09:36:34,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:36:34,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:36:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:34,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:34,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:34,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:34,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:34,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:34,473 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:34,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:34,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:34,496 [518] [DEBUG] [app] Starting request: urn:request:947e9d4c-f10c-4518-b031-7f3c17cb2707 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:34,496 [518] [DEBUG] [app] Starting request: urn:request:947e9d4c-f10c-4518-b031-7f3c17cb2707 (/_internal_ping) 2020-03-31 09:36:34,497 [518] [DEBUG] [app] Ending request: urn:request:947e9d4c-f10c-4518-b031-7f3c17cb2707 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:34,497 [518] [DEBUG] [app] Ending request: urn:request:947e9d4c-f10c-4518-b031-7f3c17cb2707 (/_internal_ping) 2020-03-31 09:36:34,497 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:34,497 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:34,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:34,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:34,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:34,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:34,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:36:34,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:36:34,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:34,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:34,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:34,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:34,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:34,509 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:34,510 [535] [DEBUG] [app] Starting request: urn:request:e5139768-fc9d-4dcc-b0bf-0aed249de3d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:34,510 [535] [DEBUG] [app] Starting request: urn:request:e5139768-fc9d-4dcc-b0bf-0aed249de3d2 (/c1/_internal_ping) 2020-03-31 09:36:34,511 [535] [DEBUG] [app] Ending request: urn:request:e5139768-fc9d-4dcc-b0bf-0aed249de3d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:34,511 [535] [DEBUG] [app] Ending request: urn:request:e5139768-fc9d-4dcc-b0bf-0aed249de3d2 (/c1/_internal_ping) 2020-03-31 09:36:34,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:34,511 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:34,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:34,513 [518] [DEBUG] [app] Ending request: urn:request:485b9d08-bb6a-4b93-9be5-e86b2a6ddbd5 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:36:34,511 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:34,514 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:34,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:36:34,513 [518] [DEBUG] [app] Ending request: urn:request:485b9d08-bb6a-4b93-9be5-e86b2a6ddbd5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:34,514 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:34,514 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:36:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:36:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:38.010934+00:00 (in 1.085727 seconds) storagereplication stdout | 2020-03-31 09:36:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:38.010934+00:00 (in 1.085727 seconds) 2020-03-31 09:36:36,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:46 UTC)" (scheduled at 2020-03-31 09:36:36.924497+00:00) storagereplication stdout | 2020-03-31 09:36:36,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:46 UTC)" (scheduled at 2020-03-31 09:36:36.924497+00:00) 2020-03-31 09:36:36,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:36:36,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 36, 927351), True, datetime.datetime(2020, 3, 31, 9, 36, 36, 927351), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:36:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 36, 927351), True, datetime.datetime(2020, 3, 31, 9, 36, 36, 927351), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:36:36,943 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 56, 36, 927351), 1, u'6ea41156-db9f-48a8-a733-cfb40f31f565', 3, u'146d5d09-d37f-4627-8baf-e607709b29af']) storagereplication stdout | 2020-03-31 09:36:36,943 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 56, 36, 927351), 1, u'6ea41156-db9f-48a8-a733-cfb40f31f565', 3, u'146d5d09-d37f-4627-8baf-e607709b29af']) 2020-03-31 09:36:36,950 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} storagereplication stdout | 2020-03-31 09:36:36,950 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:36:36,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:36,950 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 storagereplication stdout | 2020-03-31 09:36:36,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:36,950 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 2020-03-31 09:36:36,953 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:36:36,953 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:36:36,967 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:36:36,967 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:36:36,971 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) storagereplication stdout | 2020-03-31 09:36:36,971 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) 2020-03-31 09:36:36,975 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:36:36,975 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:36:36,975 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1ddcc846-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:36 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:36,975 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1ddcc846-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1ddcc846-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gEqj1ag3/UfLiwpUxCyj3Z2DjuzwPczqgy6cp7R9M2g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:36 GMT'}. storagereplication stdout | 2020-03-31 09:36:36,975 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1ddcc846-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:36 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:36,975 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1ddcc846-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1ddcc846-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gEqj1ag3/UfLiwpUxCyj3Z2DjuzwPczqgy6cp7R9M2g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:36 GMT'}. 2020-03-31 09:36:36,977 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:36:36,977 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:36:37,100 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:36:37,100 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:36:37,101 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1ddcc846-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:36 GMT, Server-Request-ID=2fc76b72-001e-006b-683f-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1ddcc846-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gEqj1ag3/UfLiwpUxCyj3Z2DjuzwPczqgy6cp7R9M2g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:36 GMT'}. storagereplication stdout | 2020-03-31 09:36:37,101 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1ddcc846-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:36 GMT, Server-Request-ID=2fc76b72-001e-006b-683f-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1ddcc846-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gEqj1ag3/UfLiwpUxCyj3Z2DjuzwPczqgy6cp7R9M2g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:36 GMT'}. 2020-03-31 09:36:37,102 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:36:37,102 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:36:37,102 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:36:37,102 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:37,102 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:36:37,102 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:36:37,103 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:1df039ee-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=upm4TkiD0ZZC9BCvYtz26fjt02I2qeKPtHWsB6lSB1U%3D&sv=2017-07-29&se=2020-03-31T09%3A37%3A37Z x-ms-date:Tue, 31 Mar 2020 09:36:37 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:36:37,103 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df039ee-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=upm4TkiD0ZZC9BCvYtz26fjt02I2qeKPtHWsB6lSB1U%3D&sv=2017-07-29&se=2020-03-31T09%3A37%3A37Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:N7GTfU287b+VtM9sWBWRJVJyTyyHnP+41L1kAf4rNCk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1df039ee-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. storagereplication stdout | 2020-03-31 09:36:37,103 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:1df039ee-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=upm4TkiD0ZZC9BCvYtz26fjt02I2qeKPtHWsB6lSB1U%3D&sv=2017-07-29&se=2020-03-31T09%3A37%3A37Z x-ms-date:Tue, 31 Mar 2020 09:36:37 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:36:37,103 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df039ee-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=upm4TkiD0ZZC9BCvYtz26fjt02I2qeKPtHWsB6lSB1U%3D&sv=2017-07-29&se=2020-03-31T09%3A37%3A37Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:N7GTfU287b+VtM9sWBWRJVJyTyyHnP+41L1kAf4rNCk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1df039ee-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. 2020-03-31 09:36:37,143 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:36:37,143 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 2020-03-31 09:36:37,144 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df039ee-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:36 GMT, Server-Request-ID=2fc76b76-001e-006b-6a3f-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=upm4TkiD0ZZC9BCvYtz26fjt02I2qeKPtHWsB6lSB1U%3D&sv=2017-07-29&se=2020-03-31T09%3A37%3A37Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:N7GTfU287b+VtM9sWBWRJVJyTyyHnP+41L1kAf4rNCk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1df039ee-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. storagereplication stdout | 2020-03-31 09:36:37,144 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df039ee-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:36 GMT, Server-Request-ID=2fc76b76-001e-006b-6a3f-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=upm4TkiD0ZZC9BCvYtz26fjt02I2qeKPtHWsB6lSB1U%3D&sv=2017-07-29&se=2020-03-31T09%3A37%3A37Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:N7GTfU287b+VtM9sWBWRJVJyTyyHnP+41L1kAf4rNCk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1df039ee-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. 2020-03-31 09:36:37,144 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:36:37,145 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1df69c1c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:37 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:36:37,144 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:36:37,145 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1df69c1c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:37 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:37,145 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1df69c1c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AeXrQJCwS6G6nHgMSw9huNxvTq9pBz1FneGTA2+5KRA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. storagereplication stdout | 2020-03-31 09:36:37,145 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1df69c1c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AeXrQJCwS6G6nHgMSw9huNxvTq9pBz1FneGTA2+5KRA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. 2020-03-31 09:36:37,147 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:36:37,147 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:36:37,568 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:36:37,568 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:36:37,568 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:37 GMT, Server-Request-ID=03ea1c2d-f01e-0021-243f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1df69c1c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AeXrQJCwS6G6nHgMSw9huNxvTq9pBz1FneGTA2+5KRA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. 2020-03-31 09:36:37,569 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:37 GMT, Server-Request-ID=03ea1c2d-f01e-0021-243f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:37,569 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:37 GMT, Server-Request-ID=03ea1c2d-f01e-0021-243f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:37,569 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:36:37,568 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:37 GMT, Server-Request-ID=03ea1c2d-f01e-0021-243f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1df69c1c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AeXrQJCwS6G6nHgMSw9huNxvTq9pBz1FneGTA2+5KRA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:37 GMT'}. 2020-03-31 09:36:37,569 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:37 GMT, Server-Request-ID=03ea1c2d-f01e-0021-243f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:37,569 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=1df69c1c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:37 GMT, Server-Request-ID=03ea1c2d-f01e-0021-243f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:37,569 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:36:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:36:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:46.924497+00:00 (in 8.912916 seconds) 2020-03-31 09:36:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:38 UTC)" (scheduled at 2020-03-31 09:36:38.010934+00:00) 2020-03-31 09:36:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:36:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:46.924497+00:00 (in 8.912916 seconds) 2020-03-31 09:36:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:38 UTC)" (scheduled at 2020-03-31 09:36:38.010934+00:00) 2020-03-31 09:36:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:36:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:36:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:38 UTC)" executed successfully 2020-03-31 09:36:38,273 [525] [DEBUG] [app] Starting request: urn:request:f819bc84-5897-45e8-80e7-d157149909bb (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:38,273 [525] [DEBUG] [app] Starting request: urn:request:f819bc84-5897-45e8-80e7-d157149909bb (/health/instance) 2020-03-31 09:36:38,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:38,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:38,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:38,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:38,280 [549] [DEBUG] [app] Starting request: urn:request:937b4b86-42b5-4c90-b8b7-0abb5003e1ea (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:38,280 [549] [DEBUG] [app] Starting request: urn:request:937b4b86-42b5-4c90-b8b7-0abb5003e1ea (/v1/_internal_ping) 2020-03-31 09:36:38,281 [549] [DEBUG] [app] Ending request: urn:request:937b4b86-42b5-4c90-b8b7-0abb5003e1ea (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:38,281 [549] [DEBUG] [app] Ending request: urn:request:937b4b86-42b5-4c90-b8b7-0abb5003e1ea (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:38,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:38,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:36:38,282 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:38,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:38,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:38,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:38,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:38,284 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:38,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:38,286 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:38,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:38,286 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:38,292 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:38,292 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:38,307 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:38,307 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:38,310 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:38,310 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:38,312 [518] [DEBUG] [app] Starting request: urn:request:7158a93b-a76e-4506-bd43-8872989d2774 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:38,312 [518] [DEBUG] [app] Starting request: urn:request:7158a93b-a76e-4506-bd43-8872989d2774 (/_internal_ping) 2020-03-31 09:36:38,313 [518] [DEBUG] [app] Ending request: urn:request:7158a93b-a76e-4506-bd43-8872989d2774 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:38,313 [518] [DEBUG] [app] Ending request: urn:request:7158a93b-a76e-4506-bd43-8872989d2774 (/_internal_ping) 2020-03-31 09:36:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:38,314 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:38,314 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:36:38,317 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:38,317 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:38,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:36:38,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:36:38,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:38,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:38,322 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:36:38,322 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:38,325 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:38,325 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:38,326 [534] [DEBUG] [app] Starting request: urn:request:5b9ea4f9-956e-42fa-8000-598f07ea8c7e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:38,326 [534] [DEBUG] [app] Starting request: urn:request:5b9ea4f9-956e-42fa-8000-598f07ea8c7e (/c1/_internal_ping) 2020-03-31 09:36:38,327 [534] [DEBUG] [app] Ending request: urn:request:5b9ea4f9-956e-42fa-8000-598f07ea8c7e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:38,327 [534] [DEBUG] [app] Ending request: urn:request:5b9ea4f9-956e-42fa-8000-598f07ea8c7e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 10.131.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:36:38,327 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 10.131.0.1 () - - [31/Mar/2020:09:36:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:36:38,328 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:38,329 [525] [DEBUG] [app] Ending request: urn:request:f819bc84-5897-45e8-80e7-d157149909bb (/health/instance) 2020-03-31 09:36:38,329 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:38,330 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:36:38,327 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:38,328 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:38,329 [525] [DEBUG] [app] Ending request: urn:request:f819bc84-5897-45e8-80e7-d157149909bb (/health/instance) 2020-03-31 09:36:38,329 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:38,330 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:36:39,571 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1f68dec0-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:39 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:39,571 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f68dec0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AlnHL5GzyEBsiiQy73AfHMFh/TcjMEN+Mumzvcw89IU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:39 GMT'}. storagereplication stdout | 2020-03-31 09:36:39,571 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1f68dec0-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:39 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:39,571 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f68dec0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AlnHL5GzyEBsiiQy73AfHMFh/TcjMEN+Mumzvcw89IU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:39 GMT'}. 2020-03-31 09:36:39,651 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:36:39,651 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:36:39,652 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:39 GMT, Server-Request-ID=03ea251e-f01e-0021-643f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f68dec0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AlnHL5GzyEBsiiQy73AfHMFh/TcjMEN+Mumzvcw89IU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:39 GMT'}. 2020-03-31 09:36:39,652 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:39 GMT, Server-Request-ID=03ea251e-f01e-0021-643f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:39,652 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:39 GMT, Server-Request-ID=03ea251e-f01e-0021-643f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:39,652 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:36:39,652 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:39 GMT, Server-Request-ID=03ea251e-f01e-0021-643f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f68dec0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AlnHL5GzyEBsiiQy73AfHMFh/TcjMEN+Mumzvcw89IU=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:39 GMT'}. 2020-03-31 09:36:39,652 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:39 GMT, Server-Request-ID=03ea251e-f01e-0021-643f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:39,652 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=1f68dec0-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:39 GMT, Server-Request-ID=03ea251e-f01e-0021-643f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:39,652 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:36:41,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:36:41,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:41,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:51.440707+00:00 (in 9.996318 seconds) 2020-03-31 09:36:41,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:51 UTC)" (scheduled at 2020-03-31 09:36:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:36:41,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:51.440707+00:00 (in 9.996318 seconds) 2020-03-31 09:36:41,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:51 UTC)" (scheduled at 2020-03-31 09:36:41.440707+00:00) 2020-03-31 09:36:41,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:36:41,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:36:41,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:36:41,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:41,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:41,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:36:41,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:41,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:51 UTC)" executed successfully 2020-03-31 09:36:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:36:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:43 UTC)" (scheduled at 2020-03-31 09:36:43.539999+00:00) 2020-03-31 09:36:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:36:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:43 UTC)" (scheduled at 2020-03-31 09:36:43.539999+00:00) 2020-03-31 09:36:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:36:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:43 UTC)" executed successfully 2020-03-31 09:36:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:33.366791+00:00 (in 49.826016 seconds) exportactionlogsworker stdout | 2020-03-31 09:36:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:43 UTC)" executed successfully 2020-03-31 09:36:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:33.366791+00:00 (in 49.826016 seconds) 2020-03-31 09:36:43,655 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:21d7ff24-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:43 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:36:43,655 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:21d7ff24-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:43 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:43,655 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '21d7ff24-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zEB1j1u5z1One2TT1Ko82OGEve/8wKbXJ+Yrd2Duc3M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:43 GMT'}. storagereplication stdout | 2020-03-31 09:36:43,655 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '21d7ff24-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zEB1j1u5z1One2TT1Ko82OGEve/8wKbXJ+Yrd2Duc3M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:43 GMT'}. 2020-03-31 09:36:43,736 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:36:43,736 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:36:43,737 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:43 GMT, Server-Request-ID=03ea3588-f01e-0021-183f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '21d7ff24-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zEB1j1u5z1One2TT1Ko82OGEve/8wKbXJ+Yrd2Duc3M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:43 GMT'}. 2020-03-31 09:36:43,737 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:43 GMT, Server-Request-ID=03ea3588-f01e-0021-183f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:43,737 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:43 GMT, Server-Request-ID=03ea3588-f01e-0021-183f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:36:43,737 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:43 GMT, Server-Request-ID=03ea3588-f01e-0021-183f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '21d7ff24-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zEB1j1u5z1One2TT1Ko82OGEve/8wKbXJ+Yrd2Duc3M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:43 GMT'}. 2020-03-31 09:36:43,737 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:43 GMT, Server-Request-ID=03ea3588-f01e-0021-183f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:43,737 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=21d7ff24-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:43 GMT, Server-Request-ID=03ea3588-f01e-0021-183f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:43,737 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:36:43,737 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:36:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:36:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:49.498966+00:00 (in 5.174568 seconds) 2020-03-31 09:36:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:54 UTC)" (scheduled at 2020-03-31 09:36:44.323352+00:00) 2020-03-31 09:36:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:36:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:49.498966+00:00 (in 5.174568 seconds) 2020-03-31 09:36:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:54 UTC)" (scheduled at 2020-03-31 09:36:44.323352+00:00) 2020-03-31 09:36:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 44, 325282), True, datetime.datetime(2020, 3, 31, 9, 36, 44, 325282), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:36:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 44, 325282), True, datetime.datetime(2020, 3, 31, 9, 36, 44, 325282), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:36:44,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:36:44,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:36:44,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:44,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:54 UTC)" executed successfully 2020-03-31 09:36:44,446 [518] [DEBUG] [app] Starting request: urn:request:7f2fb453-7164-43c0-a8b0-8d4ed99f43d8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:44,446 [518] [DEBUG] [app] Starting request: urn:request:7f2fb453-7164-43c0-a8b0-8d4ed99f43d8 (/health/instance) 2020-03-31 09:36:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:44,453 [542] [DEBUG] [app] Starting request: urn:request:52ed1994-ff91-47b9-962b-6a78b3ff3e80 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:44,453 [542] [DEBUG] [app] Starting request: urn:request:52ed1994-ff91-47b9-962b-6a78b3ff3e80 (/v1/_internal_ping) 2020-03-31 09:36:44,454 [542] [DEBUG] [app] Ending request: urn:request:52ed1994-ff91-47b9-962b-6a78b3ff3e80 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:44,454 [542] [DEBUG] [app] Ending request: urn:request:52ed1994-ff91-47b9-962b-6a78b3ff3e80 (/v1/_internal_ping) 2020-03-31 09:36:44,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:36:44,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:44,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:36:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:36:44,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:44,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:44,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:44,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:44,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:44,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:44,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:44,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:44,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:44,484 [520] [DEBUG] [app] Starting request: urn:request:2235c9f1-6649-44d3-b4c2-a74fd48a6fa4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:44,484 [520] [DEBUG] [app] Starting request: urn:request:2235c9f1-6649-44d3-b4c2-a74fd48a6fa4 (/_internal_ping) 2020-03-31 09:36:44,485 [520] [DEBUG] [app] Ending request: urn:request:2235c9f1-6649-44d3-b4c2-a74fd48a6fa4 (/_internal_ping) 2020-03-31 09:36:44,486 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:44,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:44,485 [520] [DEBUG] [app] Ending request: urn:request:2235c9f1-6649-44d3-b4c2-a74fd48a6fa4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:44,486 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:44,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:36:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:44,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:36:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:44,500 [535] [DEBUG] [app] Starting request: urn:request:b162ba8a-ba9c-455d-8d65-e8ab4a692dc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:44,500 [535] [DEBUG] [app] Starting request: urn:request:b162ba8a-ba9c-455d-8d65-e8ab4a692dc2 (/c1/_internal_ping) 2020-03-31 09:36:44,500 [535] [DEBUG] [app] Ending request: urn:request:b162ba8a-ba9c-455d-8d65-e8ab4a692dc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:44,500 [535] [DEBUG] [app] Ending request: urn:request:b162ba8a-ba9c-455d-8d65-e8ab4a692dc2 (/c1/_internal_ping) 2020-03-31 09:36:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:44,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:44,503 [518] [DEBUG] [app] Ending request: urn:request:7f2fb453-7164-43c0-a8b0-8d4ed99f43d8 (/health/instance) 2020-03-31 09:36:44,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:36:44,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:36:44,503 [518] [DEBUG] [app] Ending request: urn:request:7f2fb453-7164-43c0-a8b0-8d4ed99f43d8 (/health/instance) 2020-03-31 09:36:44,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:36:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:36:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:15.064514+00:00 (in 29.998753 seconds) 2020-03-31 09:36:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:15 UTC)" (scheduled at 2020-03-31 09:36:45.064514+00:00) gcworker stdout | 2020-03-31 09:36:45,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:15.064514+00:00 (in 29.998753 seconds) 2020-03-31 09:36:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:15 UTC)" (scheduled at 2020-03-31 09:36:45.064514+00:00) 2020-03-31 09:36:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437805, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:36:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437805, 1209600, 500, 1, 0]) 2020-03-31 09:36:45,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:36:45,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:36:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:36:45,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:45,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:15 UTC)" executed successfully 2020-03-31 09:36:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:36:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:36:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:56.924497+00:00 (in 9.999181 seconds) storagereplication stdout | 2020-03-31 09:36:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:36:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:56.924497+00:00 (in 9.999181 seconds) 2020-03-31 09:36:48,274 [518] [DEBUG] [app] Starting request: urn:request:02885949-dec7-4d69-a599-fb6e131fc62f (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:48,274 [518] [DEBUG] [app] Starting request: urn:request:02885949-dec7-4d69-a599-fb6e131fc62f (/health/instance) 2020-03-31 09:36:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:48,279 [546] [DEBUG] [app] Starting request: urn:request:3113b48f-3c13-4551-9c9c-477e91d48441 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:48,279 [546] [DEBUG] [app] Starting request: urn:request:3113b48f-3c13-4551-9c9c-477e91d48441 (/v1/_internal_ping) 2020-03-31 09:36:48,280 [546] [DEBUG] [app] Ending request: urn:request:3113b48f-3c13-4551-9c9c-477e91d48441 (/v1/_internal_ping) 2020-03-31 09:36:48,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:36:48,280 [546] [DEBUG] [app] Ending request: urn:request:3113b48f-3c13-4551-9c9c-477e91d48441 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:48,280 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:48,282 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:36:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:48,284 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:48,282 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:48,285 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:48,284 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:48,285 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:48,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:48,310 [518] [DEBUG] [app] Starting request: urn:request:ea92d5b3-a40a-42b7-96fe-7e9e28468d04 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:48,310 [518] [DEBUG] [app] Starting request: urn:request:ea92d5b3-a40a-42b7-96fe-7e9e28468d04 (/_internal_ping) 2020-03-31 09:36:48,311 [518] [DEBUG] [app] Ending request: urn:request:ea92d5b3-a40a-42b7-96fe-7e9e28468d04 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:48,311 [518] [DEBUG] [app] Ending request: urn:request:ea92d5b3-a40a-42b7-96fe-7e9e28468d04 (/_internal_ping) 2020-03-31 09:36:48,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:48,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:36:48,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:48,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:36:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:48,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:48,326 [539] [DEBUG] [app] Starting request: urn:request:4e76495c-85dd-4f37-9a71-e513efb2e29b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:48,326 [539] [DEBUG] [app] Starting request: urn:request:4e76495c-85dd-4f37-9a71-e513efb2e29b (/c1/_internal_ping) 2020-03-31 09:36:48,327 [539] [DEBUG] [app] Ending request: urn:request:4e76495c-85dd-4f37-9a71-e513efb2e29b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:48,327 [539] [DEBUG] [app] Ending request: urn:request:4e76495c-85dd-4f37-9a71-e513efb2e29b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:48,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:36:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:48,330 [518] [DEBUG] [app] Ending request: urn:request:02885949-dec7-4d69-a599-fb6e131fc62f (/health/instance) 2020-03-31 09:36:48,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:48,330 [518] [DEBUG] [app] Ending request: urn:request:02885949-dec7-4d69-a599-fb6e131fc62f (/health/instance) 2020-03-31 09:36:48,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:36:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:36:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:36:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:54.323352+00:00 (in 4.822199 seconds) 2020-03-31 09:36:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:49 UTC)" (scheduled at 2020-03-31 09:36:49.498966+00:00) 2020-03-31 09:36:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:36:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:36:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:54.323352+00:00 (in 4.822199 seconds) 2020-03-31 09:36:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:49 UTC)" (scheduled at 2020-03-31 09:36:49.498966+00:00) 2020-03-31 09:36:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:36:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:49 UTC)" executed successfully 2020-03-31 09:36:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:36:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:59.560326+00:00 (in 8.448119 seconds) 2020-03-31 09:36:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:51 UTC)" (scheduled at 2020-03-31 09:36:51.110610+00:00) 2020-03-31 09:36:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:36:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:36:59.560326+00:00 (in 8.448119 seconds) 2020-03-31 09:36:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:51 UTC)" (scheduled at 2020-03-31 09:36:51.110610+00:00) 2020-03-31 09:36:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 51, 113090), True, datetime.datetime(2020, 3, 31, 9, 36, 51, 113090), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:36:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 51, 113090), True, datetime.datetime(2020, 3, 31, 9, 36, 51, 113090), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:36:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:36:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:36:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:51 UTC)" executed successfully 2020-03-31 09:36:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:36:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:01.440707+00:00 (in 9.999139 seconds) 2020-03-31 09:36:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:01 UTC)" (scheduled at 2020-03-31 09:36:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:36:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:01.440707+00:00 (in 9.999139 seconds) 2020-03-31 09:36:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:01 UTC)" (scheduled at 2020-03-31 09:36:51.440707+00:00) 2020-03-31 09:36:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:36:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:36:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:51,460 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:36:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:36:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:36:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:01 UTC)" executed successfully 2020-03-31 09:36:51,745 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:26aa6c6c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:51 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:51,745 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '26aa6c6c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0jTdK5wP/sAmBdjx6o0RBV8aoBOFN64vczuIS+rwT9Y=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:51 GMT'}. storagereplication stdout | 2020-03-31 09:36:51,745 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:26aa6c6c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:36:51 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:36:51,745 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '26aa6c6c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0jTdK5wP/sAmBdjx6o0RBV8aoBOFN64vczuIS+rwT9Y=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:51 GMT'}. 2020-03-31 09:36:51,825 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:36:51,825 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:36:51,826 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:51 GMT, Server-Request-ID=03ea5622-f01e-0021-803f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '26aa6c6c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0jTdK5wP/sAmBdjx6o0RBV8aoBOFN64vczuIS+rwT9Y=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:51 GMT'}. 2020-03-31 09:36:51,826 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:51 GMT, Server-Request-ID=03ea5622-f01e-0021-803f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:51,826 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:51 GMT, Server-Request-ID=03ea5622-f01e-0021-803f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:51,826 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:36:51,826 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:36:51 GMT, Server-Request-ID=03ea5622-f01e-0021-803f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '26aa6c6c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0jTdK5wP/sAmBdjx6o0RBV8aoBOFN64vczuIS+rwT9Y=', 'x-ms-date': 'Tue, 31 Mar 2020 09:36:51 GMT'}. 2020-03-31 09:36:51,826 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:36:51 GMT, Server-Request-ID=03ea5622-f01e-0021-803f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:51,826 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=26aa6c6c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:36:51 GMT, Server-Request-ID=03ea5622-f01e-0021-803f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:36:51,826 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:36:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:36:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:22.113221+00:00 (in 29.997482 seconds) 2020-03-31 09:36:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:22 UTC)" (scheduled at 2020-03-31 09:36:52.113221+00:00) securityworker stdout | 2020-03-31 09:36:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:22.113221+00:00 (in 29.997482 seconds) 2020-03-31 09:36:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:22 UTC)" (scheduled at 2020-03-31 09:36:52.113221+00:00) 2020-03-31 09:36:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:36:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:36:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:36:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:36:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:36:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:36:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:36:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:22 UTC)" executed successfully 2020-03-31 09:36:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:36:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:04.323352+00:00 (in 9.999135 seconds) 2020-03-31 09:36:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:04 UTC)" (scheduled at 2020-03-31 09:36:54.323352+00:00) 2020-03-31 09:36:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:36:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:04.323352+00:00 (in 9.999135 seconds) 2020-03-31 09:36:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:04 UTC)" (scheduled at 2020-03-31 09:36:54.323352+00:00) 2020-03-31 09:36:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:36:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 54, 325010), True, datetime.datetime(2020, 3, 31, 9, 36, 54, 325010), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:36:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 36, 54, 325010), True, datetime.datetime(2020, 3, 31, 9, 36, 54, 325010), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:36:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:36:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:36:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:36:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:04 UTC)" executed successfully 2020-03-31 09:36:54,445 [527] [DEBUG] [app] Starting request: urn:request:bc5d0a4c-1efd-467e-9f31-21e97b4a19d3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:54,445 [527] [DEBUG] [app] Starting request: urn:request:bc5d0a4c-1efd-467e-9f31-21e97b4a19d3 (/health/instance) 2020-03-31 09:36:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:54,453 [538] [DEBUG] [app] Starting request: urn:request:b2247fe7-eb3e-4bca-8e60-a7c87bf208b8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:54,453 [538] [DEBUG] [app] Starting request: urn:request:b2247fe7-eb3e-4bca-8e60-a7c87bf208b8 (/v1/_internal_ping) 2020-03-31 09:36:54,455 [538] [DEBUG] [app] Ending request: urn:request:b2247fe7-eb3e-4bca-8e60-a7c87bf208b8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:54,455 [538] [DEBUG] [app] Ending request: urn:request:b2247fe7-eb3e-4bca-8e60-a7c87bf208b8 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:54,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:36:54,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:54,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:54,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:54,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:54,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:54,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:54,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:54,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:54,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:54,488 [526] [DEBUG] [app] Starting request: urn:request:085c2897-8088-4588-bf46-61a99e39a1a6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:54,488 [526] [DEBUG] [app] Starting request: urn:request:085c2897-8088-4588-bf46-61a99e39a1a6 (/_internal_ping) 2020-03-31 09:36:54,489 [526] [DEBUG] [app] Ending request: urn:request:085c2897-8088-4588-bf46-61a99e39a1a6 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:54,489 [526] [DEBUG] [app] Ending request: urn:request:085c2897-8088-4588-bf46-61a99e39a1a6 (/_internal_ping) 2020-03-31 09:36:54,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:54,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:36:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:36:54,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:54,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:54,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:54,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:54,503 [539] [DEBUG] [app] Starting request: urn:request:9c97420a-07e4-48d2-a8c8-3ba48f3c1d25 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:54,503 [539] [DEBUG] [app] Starting request: urn:request:9c97420a-07e4-48d2-a8c8-3ba48f3c1d25 (/c1/_internal_ping) 2020-03-31 09:36:54,504 [539] [DEBUG] [app] Ending request: urn:request:9c97420a-07e4-48d2-a8c8-3ba48f3c1d25 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:54,504 [539] [DEBUG] [app] Ending request: urn:request:9c97420a-07e4-48d2-a8c8-3ba48f3c1d25 (/c1/_internal_ping) 2020-03-31 09:36:54,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:36:54,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:54,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:36:54,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:54,507 [527] [DEBUG] [app] Ending request: urn:request:bc5d0a4c-1efd-467e-9f31-21e97b4a19d3 (/health/instance) 2020-03-31 09:36:54,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:36:54,507 [527] [DEBUG] [app] Ending request: urn:request:bc5d0a4c-1efd-467e-9f31-21e97b4a19d3 (/health/instance) 2020-03-31 09:36:54,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:36:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:36:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:36:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:36:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:03.010318+00:00 (in 6.084968 seconds) storagereplication stdout | 2020-03-31 09:36:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:36:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:36:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:03.010318+00:00 (in 6.084968 seconds) 2020-03-31 09:36:58,272 [518] [DEBUG] [app] Starting request: urn:request:6b273775-0a01-45e1-99b0-e61997e1e572 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:58,272 [518] [DEBUG] [app] Starting request: urn:request:6b273775-0a01-45e1-99b0-e61997e1e572 (/health/instance) 2020-03-31 09:36:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:58,280 [551] [DEBUG] [app] Starting request: urn:request:c32e186c-bb4f-4e10-bc2d-4ab6f4b0dd30 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:58,280 [551] [DEBUG] [app] Starting request: urn:request:c32e186c-bb4f-4e10-bc2d-4ab6f4b0dd30 (/v1/_internal_ping) 2020-03-31 09:36:58,281 [551] [DEBUG] [app] Ending request: urn:request:c32e186c-bb4f-4e10-bc2d-4ab6f4b0dd30 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:36:58,281 [551] [DEBUG] [app] Ending request: urn:request:c32e186c-bb4f-4e10-bc2d-4ab6f4b0dd30 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:58,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:36:58,282 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:58,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:36:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:36:58,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:36:58,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:36:58,284 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:36:58,285 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:36:58,285 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:36:58,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:36:58,286 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:36:58,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:36:58,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:36:58,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:36:58,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:36:58,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:58,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:58,312 [527] [DEBUG] [app] Starting request: urn:request:161d87b6-a02a-4b5c-b355-48f21b753f35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:58,312 [527] [DEBUG] [app] Starting request: urn:request:161d87b6-a02a-4b5c-b355-48f21b753f35 (/_internal_ping) 2020-03-31 09:36:58,313 [527] [DEBUG] [app] Ending request: urn:request:161d87b6-a02a-4b5c-b355-48f21b753f35 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:36:58,313 [527] [DEBUG] [app] Ending request: urn:request:161d87b6-a02a-4b5c-b355-48f21b753f35 (/_internal_ping) 2020-03-31 09:36:58,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:36:58,313 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:36:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:36:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:36:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:36:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:36:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:36:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:36:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:36:58,326 [541] [DEBUG] [app] Starting request: urn:request:9af12ac2-7d6e-4654-a4df-488bebb2f8dd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:58,326 [541] [DEBUG] [app] Starting request: urn:request:9af12ac2-7d6e-4654-a4df-488bebb2f8dd (/c1/_internal_ping) 2020-03-31 09:36:58,327 [541] [DEBUG] [app] Ending request: urn:request:9af12ac2-7d6e-4654-a4df-488bebb2f8dd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:36:58,327 [541] [DEBUG] [app] Ending request: urn:request:9af12ac2-7d6e-4654-a4df-488bebb2f8dd (/c1/_internal_ping) 2020-03-31 09:36:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:36:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:36:58,330 [518] [DEBUG] [app] Ending request: urn:request:6b273775-0a01-45e1-99b0-e61997e1e572 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:36:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:36:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:36:58,330 [518] [DEBUG] [app] Ending request: urn:request:6b273775-0a01-45e1-99b0-e61997e1e572 (/health/instance) gunicorn-web stdout | 2020-03-31 09:36:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:36:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:36:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:36:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:36:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:36:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:36:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:51.110610+00:00 (in 51.549166 seconds) 2020-03-31 09:36:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:59 UTC)" (scheduled at 2020-03-31 09:36:59.560326+00:00) 2020-03-31 09:36:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:36:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:51.110610+00:00 (in 51.549166 seconds) 2020-03-31 09:36:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:59 UTC)" (scheduled at 2020-03-31 09:36:59.560326+00:00) 2020-03-31 09:36:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:36:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:36:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:59 UTC)" executed successfully 2020-03-31 09:37:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:37:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:30.337699+00:00 (in 29.999158 seconds) 2020-03-31 09:37:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:30 UTC)" (scheduled at 2020-03-31 09:37:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:37:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:30.337699+00:00 (in 29.999158 seconds) 2020-03-31 09:37:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:30 UTC)" (scheduled at 2020-03-31 09:37:00.337699+00:00) 2020-03-31 09:37:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 37, 0, 339367), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:37:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 37, 0, 339367), False, 50, 1, 0]) 2020-03-31 09:37:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:37:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:37:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:37:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:30 UTC)" executed successfully 2020-03-31 09:37:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:37:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:11.440707+00:00 (in 9.998885 seconds) 2020-03-31 09:37:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:11 UTC)" (scheduled at 2020-03-31 09:37:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:37:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:11.440707+00:00 (in 9.998885 seconds) 2020-03-31 09:37:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:11 UTC)" (scheduled at 2020-03-31 09:37:01.440707+00:00) 2020-03-31 09:37:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:37:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:37:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:37:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:37:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:37:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:11 UTC)" executed successfully 2020-03-31 09:37:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:03,010 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:06.924497+00:00 (in 3.913334 seconds) 2020-03-31 09:37:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:03 UTC)" (scheduled at 2020-03-31 09:37:03.010318+00:00) storagereplication stdout | 2020-03-31 09:37:03,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:06.924497+00:00 (in 3.913334 seconds) 2020-03-31 09:37:03,011 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:03 UTC)" (scheduled at 2020-03-31 09:37:03.010318+00:00) 2020-03-31 09:37:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:37:03,013 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), u'imagestoragereplication/%']) 2020-03-31 09:37:03,027 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), True, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:37:03,027 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), True, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), 0, u'imagestoragereplication/%']) 2020-03-31 09:37:03,031 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), True, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:37:03,031 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), True, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 37, 3, 12091), u'imagestoragereplication/%']) 2020-03-31 09:37:03,035 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:37:03,035 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:03,038 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:03 UTC)" executed successfully 2020-03-31 09:37:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:37:03,038 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:37:03,039 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:37:03,040 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:03,041 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 09:37:03,041 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:37:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:37:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:24.410927+00:00 (in 21.080029 seconds) 2020-03-31 09:37:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:33 UTC)" (scheduled at 2020-03-31 09:37:03.330112+00:00) 2020-03-31 09:37:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:37:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:24.410927+00:00 (in 21.080029 seconds) 2020-03-31 09:37:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:33 UTC)" (scheduled at 2020-03-31 09:37:03.330112+00:00) 2020-03-31 09:37:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 3, 331856), True, datetime.datetime(2020, 3, 31, 9, 37, 3, 331856), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:37:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 3, 331856), True, datetime.datetime(2020, 3, 31, 9, 37, 3, 331856), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:37:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:37:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:37:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:33 UTC)" executed successfully 2020-03-31 09:37:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:14.323352+00:00 (in 9.999152 seconds) 2020-03-31 09:37:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:14 UTC)" (scheduled at 2020-03-31 09:37:04.323352+00:00) notificationworker stdout | 2020-03-31 09:37:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:14.323352+00:00 (in 9.999152 seconds) 2020-03-31 09:37:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:14 UTC)" (scheduled at 2020-03-31 09:37:04.323352+00:00) 2020-03-31 09:37:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:37:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 4, 325262), True, datetime.datetime(2020, 3, 31, 9, 37, 4, 325262), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:37:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 4, 325262), True, datetime.datetime(2020, 3, 31, 9, 37, 4, 325262), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:37:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:37:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:14 UTC)" executed successfully 2020-03-31 09:37:04,446 [518] [DEBUG] [app] Starting request: urn:request:e2a5cd60-a0a6-4780-9a4d-0cdce0d5ed34 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:04,446 [518] [DEBUG] [app] Starting request: urn:request:e2a5cd60-a0a6-4780-9a4d-0cdce0d5ed34 (/health/instance) 2020-03-31 09:37:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:04,453 [553] [DEBUG] [app] Starting request: urn:request:069506cb-2280-466b-965a-48a370280193 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:04,453 [553] [DEBUG] [app] Starting request: urn:request:069506cb-2280-466b-965a-48a370280193 (/v1/_internal_ping) 2020-03-31 09:37:04,454 [553] [DEBUG] [app] Ending request: urn:request:069506cb-2280-466b-965a-48a370280193 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:04,454 [553] [DEBUG] [app] Ending request: urn:request:069506cb-2280-466b-965a-48a370280193 (/v1/_internal_ping) 2020-03-31 09:37:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:37:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:37:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:04,476 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:04,476 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:04,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:04,491 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:04,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:04,497 [518] [DEBUG] [app] Starting request: urn:request:a2be29f9-59b0-432a-adea-1b39e72aec24 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:04,497 [518] [DEBUG] [app] Starting request: urn:request:a2be29f9-59b0-432a-adea-1b39e72aec24 (/_internal_ping) 2020-03-31 09:37:04,497 [518] [DEBUG] [app] Ending request: urn:request:a2be29f9-59b0-432a-adea-1b39e72aec24 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:04,497 [518] [DEBUG] [app] Ending request: urn:request:a2be29f9-59b0-432a-adea-1b39e72aec24 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:37:04,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:37:04,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:04,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:04,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:04,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:04,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:37:04,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:04,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:04,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:04,512 [541] [DEBUG] [app] Starting request: urn:request:79479bb7-f839-413f-808f-db86ef23b013 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:04,512 [541] [DEBUG] [app] Starting request: urn:request:79479bb7-f839-413f-808f-db86ef23b013 (/c1/_internal_ping) 2020-03-31 09:37:04,512 [541] [DEBUG] [app] Ending request: urn:request:79479bb7-f839-413f-808f-db86ef23b013 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:04,512 [541] [DEBUG] [app] Ending request: urn:request:79479bb7-f839-413f-808f-db86ef23b013 (/c1/_internal_ping) 2020-03-31 09:37:04,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:04,513 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:04,513 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:04,514 [518] [DEBUG] [app] Ending request: urn:request:e2a5cd60-a0a6-4780-9a4d-0cdce0d5ed34 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:37:04,513 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:04,515 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:04,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:37:04,514 [518] [DEBUG] [app] Ending request: urn:request:e2a5cd60-a0a6-4780-9a4d-0cdce0d5ed34 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:04,515 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:04,515 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:37:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:16.924497+00:00 (in 9.998874 seconds) storagereplication stdout | 2020-03-31 09:37:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:16.924497+00:00 (in 9.998874 seconds) 2020-03-31 09:37:07,842 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:37:07,842 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy 2020-03-31 09:37:07,842 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:37:07,842 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:37:07,842 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:37:07,842 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:37:07,854 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) storagereplication stdout | 2020-03-31 09:37:07,854 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) 2020-03-31 09:37:07,858 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 42, 7, 853397), True, u'6a7f39c7-7513-442a-bec5-4d3b37ffac5c', 3]) storagereplication stdout | 2020-03-31 09:37:07,858 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 42, 7, 853397), True, u'6a7f39c7-7513-442a-bec5-4d3b37ffac5c', 3]) 2020-03-31 09:37:07,865 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 7, 864627), True, datetime.datetime(2020, 3, 31, 9, 37, 7, 864627), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:37:07,865 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 7, 864627), True, datetime.datetime(2020, 3, 31, 9, 37, 7, 864627), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:37:07,870 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 57, 7, 864627), 1, u'61b132c8-97bf-4247-9941-ed44a5687b76', 4, u'43b1dced-dda3-48fd-81f0-713652abf8e2']) storagereplication stdout | 2020-03-31 09:37:07,870 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 57, 7, 864627), 1, u'61b132c8-97bf-4247-9941-ed44a5687b76', 4, u'43b1dced-dda3-48fd-81f0-713652abf8e2']) 2020-03-31 09:37:07,875 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} storagereplication stdout | 2020-03-31 09:37:07,875 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:37:07,875 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:07,875 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 storagereplication stdout | 2020-03-31 09:37:07,875 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:07,875 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 2020-03-31 09:37:07,877 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:37:07,877 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:37:07,893 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:37:07,893 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:37:07,898 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) storagereplication stdout | 2020-03-31 09:37:07,898 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) 2020-03-31 09:37:07,902 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:37:07,902 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:37:07,902 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:304bddd2-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:07 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:37:07,902 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304bddd2-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '304bddd2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:IjxWQ4cGAkgObJxWiuJbGWGoC6XeseUqoFz1aMtXBO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. storagereplication stdout | 2020-03-31 09:37:07,902 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:304bddd2-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:07 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:37:07,902 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304bddd2-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '304bddd2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:IjxWQ4cGAkgObJxWiuJbGWGoC6XeseUqoFz1aMtXBO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. 2020-03-31 09:37:07,911 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:37:07,911 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:37:07,912 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304bddd2-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=2fc77869-001e-006b-713f-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '304bddd2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:IjxWQ4cGAkgObJxWiuJbGWGoC6XeseUqoFz1aMtXBO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. storagereplication stdout | 2020-03-31 09:37:07,912 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304bddd2-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=2fc77869-001e-006b-713f-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '304bddd2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:IjxWQ4cGAkgObJxWiuJbGWGoC6XeseUqoFz1aMtXBO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. 2020-03-31 09:37:07,913 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us storagereplication stdout | 2020-03-31 09:37:07,913 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us 2020-03-31 09:37:07,913 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:07,913 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:37:07,913 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:07,913 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:37:07,913 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:304d924e-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=D52YmxiK72Wex1ocymClqiTo2BySUrf0nVBgB9Lidf0%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A07Z x-ms-date:Tue, 31 Mar 2020 09:37:07 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:37:07,914 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304d924e-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=D52YmxiK72Wex1ocymClqiTo2BySUrf0nVBgB9Lidf0%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A07Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OsA4HxLKVx6K7yYkeitRitO6D+ERP8gE6K++efdd8QM=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '304d924e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. storagereplication stdout | 2020-03-31 09:37:07,913 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:304d924e-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=D52YmxiK72Wex1ocymClqiTo2BySUrf0nVBgB9Lidf0%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A07Z x-ms-date:Tue, 31 Mar 2020 09:37:07 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:37:07,914 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304d924e-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=D52YmxiK72Wex1ocymClqiTo2BySUrf0nVBgB9Lidf0%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A07Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OsA4HxLKVx6K7yYkeitRitO6D+ERP8gE6K++efdd8QM=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '304d924e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. 2020-03-31 09:37:07,931 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:37:07,931 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 2020-03-31 09:37:07,932 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304d924e-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=2fc7786d-001e-006b-733f-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=D52YmxiK72Wex1ocymClqiTo2BySUrf0nVBgB9Lidf0%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A07Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OsA4HxLKVx6K7yYkeitRitO6D+ERP8gE6K++efdd8QM=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '304d924e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. storagereplication stdout | 2020-03-31 09:37:07,932 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=304d924e-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=2fc7786d-001e-006b-733f-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=D52YmxiK72Wex1ocymClqiTo2BySUrf0nVBgB9Lidf0%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A07Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OsA4HxLKVx6K7yYkeitRitO6D+ERP8gE6K++efdd8QM=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '304d924e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. 2020-03-31 09:37:07,932 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:37:07,933 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:305082ce-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:07 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:37:07,932 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:37:07,933 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:305082ce-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:07 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:37:07,933 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '305082ce-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:2f9S6AptSL1ulJdg9L7QRr4SMW+25SlJ3G30iKvFSK0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. storagereplication stdout | 2020-03-31 09:37:07,933 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '305082ce-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:2f9S6AptSL1ulJdg9L7QRr4SMW+25SlJ3G30iKvFSK0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. 2020-03-31 09:37:08,018 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:08,018 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:08,018 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=03ea9263-f01e-0021-1d3f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '305082ce-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:2f9S6AptSL1ulJdg9L7QRr4SMW+25SlJ3G30iKvFSK0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. 2020-03-31 09:37:08,018 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=03ea9263-f01e-0021-1d3f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:08,018 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=03ea9263-f01e-0021-1d3f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:08,019 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:37:08,018 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=03ea9263-f01e-0021-1d3f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '305082ce-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:2f9S6AptSL1ulJdg9L7QRr4SMW+25SlJ3G30iKvFSK0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:07 GMT'}. 2020-03-31 09:37:08,018 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=03ea9263-f01e-0021-1d3f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:08,018 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=305082ce-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:07 GMT, Server-Request-ID=03ea9263-f01e-0021-1d3f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:08,019 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:37:08,273 [518] [DEBUG] [app] Starting request: urn:request:fc27ed54-0fbf-4114-ae48-992f5fd3f13f (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:08,273 [518] [DEBUG] [app] Starting request: urn:request:fc27ed54-0fbf-4114-ae48-992f5fd3f13f (/health/instance) 2020-03-31 09:37:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:08,278 [538] [DEBUG] [app] Starting request: urn:request:0d78be4a-fe80-48c7-98e1-6ee7f6aa518e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:08,278 [538] [DEBUG] [app] Starting request: urn:request:0d78be4a-fe80-48c7-98e1-6ee7f6aa518e (/v1/_internal_ping) 2020-03-31 09:37:08,280 [538] [DEBUG] [app] Ending request: urn:request:0d78be4a-fe80-48c7-98e1-6ee7f6aa518e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:08,280 [538] [DEBUG] [app] Ending request: urn:request:0d78be4a-fe80-48c7-98e1-6ee7f6aa518e (/v1/_internal_ping) 2020-03-31 09:37:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:37:08,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:37:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:08,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:08,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:08,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:08,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:08,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:08,314 [518] [DEBUG] [app] Starting request: urn:request:823c002a-db8f-4e48-a63a-2e78e75baba6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:08,314 [518] [DEBUG] [app] Starting request: urn:request:823c002a-db8f-4e48-a63a-2e78e75baba6 (/_internal_ping) 2020-03-31 09:37:08,315 [518] [DEBUG] [app] Ending request: urn:request:823c002a-db8f-4e48-a63a-2e78e75baba6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:08,315 [518] [DEBUG] [app] Ending request: urn:request:823c002a-db8f-4e48-a63a-2e78e75baba6 (/_internal_ping) 2020-03-31 09:37:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:37:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:37:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:37:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:37:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:08,330 [539] [DEBUG] [app] Starting request: urn:request:55c2b13d-af60-4706-be49-147ef21bfb34 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:08,330 [539] [DEBUG] [app] Starting request: urn:request:55c2b13d-af60-4706-be49-147ef21bfb34 (/c1/_internal_ping) 2020-03-31 09:37:08,330 [539] [DEBUG] [app] Ending request: urn:request:55c2b13d-af60-4706-be49-147ef21bfb34 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:08,330 [539] [DEBUG] [app] Ending request: urn:request:55c2b13d-af60-4706-be49-147ef21bfb34 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:08,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:37:08,331 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:08,332 [518] [DEBUG] [app] Ending request: urn:request:fc27ed54-0fbf-4114-ae48-992f5fd3f13f (/health/instance) 2020-03-31 09:37:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:37:08,332 [518] [DEBUG] [app] Ending request: urn:request:fc27ed54-0fbf-4114-ae48-992f5fd3f13f (/health/instance) 2020-03-31 09:37:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:37:10,021 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:318f2a28-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:10 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:37:10,021 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:318f2a28-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:10 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:37:10,021 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '318f2a28-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xN9/U5VWsnCQpqmc+SzvpQlt6kQv0pK36aE4RJQLA6o=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:10 GMT'}. storagereplication stdout | 2020-03-31 09:37:10,021 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '318f2a28-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xN9/U5VWsnCQpqmc+SzvpQlt6kQv0pK36aE4RJQLA6o=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:10 GMT'}. 2020-03-31 09:37:10,102 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:10,102 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:10,103 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:09 GMT, Server-Request-ID=03ea9aa0-f01e-0021-233f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '318f2a28-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xN9/U5VWsnCQpqmc+SzvpQlt6kQv0pK36aE4RJQLA6o=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:10 GMT'}. 2020-03-31 09:37:10,103 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:09 GMT, Server-Request-ID=03ea9aa0-f01e-0021-233f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:10,103 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:09 GMT, Server-Request-ID=03ea9aa0-f01e-0021-233f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:10,103 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:37:10,103 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:09 GMT, Server-Request-ID=03ea9aa0-f01e-0021-233f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '318f2a28-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xN9/U5VWsnCQpqmc+SzvpQlt6kQv0pK36aE4RJQLA6o=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:10 GMT'}. 2020-03-31 09:37:10,103 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:09 GMT, Server-Request-ID=03ea9aa0-f01e-0021-233f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:10,103 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=318f2a28-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:09 GMT, Server-Request-ID=03ea9aa0-f01e-0021-233f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:10,103 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:37:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:37:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:21.440707+00:00 (in 9.999026 seconds) 2020-03-31 09:37:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:21 UTC)" (scheduled at 2020-03-31 09:37:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:37:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:21.440707+00:00 (in 9.999026 seconds) 2020-03-31 09:37:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:21 UTC)" (scheduled at 2020-03-31 09:37:11.440707+00:00) 2020-03-31 09:37:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:37:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:37:11,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:37:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:37:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:37:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:21 UTC)" executed successfully 2020-03-31 09:37:14,107 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:33fea40a-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:14 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:37:14,107 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:33fea40a-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:14 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:37:14,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '33fea40a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:NUcKJFpgntf2I5goNHbmiwh6b6+Pi31LSvbUNT+2cIw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:14 GMT'}. storagereplication stdout | 2020-03-31 09:37:14,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '33fea40a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:NUcKJFpgntf2I5goNHbmiwh6b6+Pi31LSvbUNT+2cIw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:14 GMT'}. 2020-03-31 09:37:14,188 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:14,188 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:14,189 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:13 GMT, Server-Request-ID=03eaac38-f01e-0021-393f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '33fea40a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:NUcKJFpgntf2I5goNHbmiwh6b6+Pi31LSvbUNT+2cIw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:14 GMT'}. 2020-03-31 09:37:14,190 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:13 GMT, Server-Request-ID=03eaac38-f01e-0021-393f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:14,190 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:13 GMT, Server-Request-ID=03eaac38-f01e-0021-393f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:37:14,189 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:13 GMT, Server-Request-ID=03eaac38-f01e-0021-393f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '33fea40a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:NUcKJFpgntf2I5goNHbmiwh6b6+Pi31LSvbUNT+2cIw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:14 GMT'}. 2020-03-31 09:37:14,190 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:13 GMT, Server-Request-ID=03eaac38-f01e-0021-393f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:14,190 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=33fea40a-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:13 GMT, Server-Request-ID=03eaac38-f01e-0021-393f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:14,190 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:37:14,190 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:37:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:24.323352+00:00 (in 9.995952 seconds) 2020-03-31 09:37:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:24 UTC)" (scheduled at 2020-03-31 09:37:14.323352+00:00) 2020-03-31 09:37:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:37:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:24.323352+00:00 (in 9.995952 seconds) 2020-03-31 09:37:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:24 UTC)" (scheduled at 2020-03-31 09:37:14.323352+00:00) 2020-03-31 09:37:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:14,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 14, 328309), True, datetime.datetime(2020, 3, 31, 9, 37, 14, 328309), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:37:14,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 14, 328309), True, datetime.datetime(2020, 3, 31, 9, 37, 14, 328309), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:37:14,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:14,347 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:37:14,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:14,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:14,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:14,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:24 UTC)" executed successfully 2020-03-31 09:37:14,445 [526] [DEBUG] [app] Starting request: urn:request:2c18f84f-760d-4386-9ff6-f7312008b8cd (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:14,445 [526] [DEBUG] [app] Starting request: urn:request:2c18f84f-760d-4386-9ff6-f7312008b8cd (/health/instance) 2020-03-31 09:37:14,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:14,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:14,453 [553] [DEBUG] [app] Starting request: urn:request:a09650ae-6d2b-4a72-8430-af93fdf1c827 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:14,453 [553] [DEBUG] [app] Starting request: urn:request:a09650ae-6d2b-4a72-8430-af93fdf1c827 (/v1/_internal_ping) 2020-03-31 09:37:14,454 [553] [DEBUG] [app] Ending request: urn:request:a09650ae-6d2b-4a72-8430-af93fdf1c827 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:14,454 [553] [DEBUG] [app] Ending request: urn:request:a09650ae-6d2b-4a72-8430-af93fdf1c827 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:37:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:37:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:14,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:14,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:14,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:14,468 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:14,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:14,482 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:14,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:14,486 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:14,487 [526] [DEBUG] [app] Starting request: urn:request:80c847b0-1496-4fb7-9bc7-63f88d62872e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:14,487 [526] [DEBUG] [app] Starting request: urn:request:80c847b0-1496-4fb7-9bc7-63f88d62872e (/_internal_ping) 2020-03-31 09:37:14,488 [526] [DEBUG] [app] Ending request: urn:request:80c847b0-1496-4fb7-9bc7-63f88d62872e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:14,488 [526] [DEBUG] [app] Ending request: urn:request:80c847b0-1496-4fb7-9bc7-63f88d62872e (/_internal_ping) 2020-03-31 09:37:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:37:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:14,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:37:14,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:14,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:14,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:37:14,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:37:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:14,501 [539] [DEBUG] [app] Starting request: urn:request:76fb9636-3385-4fee-bf94-59a0daf966ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:14,501 [539] [DEBUG] [app] Starting request: urn:request:76fb9636-3385-4fee-bf94-59a0daf966ec (/c1/_internal_ping) 2020-03-31 09:37:14,502 [539] [DEBUG] [app] Ending request: urn:request:76fb9636-3385-4fee-bf94-59a0daf966ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:14,502 [539] [DEBUG] [app] Ending request: urn:request:76fb9636-3385-4fee-bf94-59a0daf966ec (/c1/_internal_ping) 2020-03-31 09:37:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:14,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:14,505 [526] [DEBUG] [app] Ending request: urn:request:2c18f84f-760d-4386-9ff6-f7312008b8cd (/health/instance) 2020-03-31 09:37:14,505 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:37:14,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:37:14,505 [526] [DEBUG] [app] Ending request: urn:request:2c18f84f-760d-4386-9ff6-f7312008b8cd (/health/instance) 2020-03-31 09:37:14,505 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:37:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:37:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:45 UTC)" (scheduled at 2020-03-31 09:37:15.064514+00:00) gcworker stdout | 2020-03-31 09:37:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:45 UTC)" (scheduled at 2020-03-31 09:37:15.064514+00:00) 2020-03-31 09:37:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:45.064514+00:00 (in 29.999243 seconds) gcworker stdout | 2020-03-31 09:37:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:45.064514+00:00 (in 29.999243 seconds) 2020-03-31 09:37:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437835, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:37:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437835, 1209600, 500, 1, 0]) 2020-03-31 09:37:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:37:15,084 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:37:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:37:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:37:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:45 UTC)" executed successfully 2020-03-31 09:37:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:16,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:16,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:26.924497+00:00 (in 9.998462 seconds) storagereplication stdout | 2020-03-31 09:37:16,926 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:16,926 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:26.924497+00:00 (in 9.998462 seconds) 2020-03-31 09:37:18,272 [518] [DEBUG] [app] Starting request: urn:request:86850b81-2e49-41fc-8515-1e623e120be2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:18,272 [518] [DEBUG] [app] Starting request: urn:request:86850b81-2e49-41fc-8515-1e623e120be2 (/health/instance) 2020-03-31 09:37:18,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:18,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:18,280 [546] [DEBUG] [app] Starting request: urn:request:45b98162-20a2-498a-9f1e-34e84e077d39 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:18,280 [546] [DEBUG] [app] Starting request: urn:request:45b98162-20a2-498a-9f1e-34e84e077d39 (/v1/_internal_ping) 2020-03-31 09:37:18,281 [546] [DEBUG] [app] Ending request: urn:request:45b98162-20a2-498a-9f1e-34e84e077d39 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:18,281 [546] [DEBUG] [app] Ending request: urn:request:45b98162-20a2-498a-9f1e-34e84e077d39 (/v1/_internal_ping) 2020-03-31 09:37:18,281 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:37:18,281 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:18,283 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:37:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:37:18,283 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:18,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:18,285 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:18,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:18,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:18,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:18,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:18,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:18,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:18,317 [523] [DEBUG] [app] Starting request: urn:request:c73c8c04-5be2-4a3f-a9fc-d057a557bb19 (/_internal_ping) 2020-03-31 09:37:18,317 [523] [DEBUG] [app] Ending request: urn:request:c73c8c04-5be2-4a3f-a9fc-d057a557bb19 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:18,317 [523] [DEBUG] [app] Starting request: urn:request:c73c8c04-5be2-4a3f-a9fc-d057a557bb19 (/_internal_ping) 2020-03-31 09:37:18,317 [523] [DEBUG] [app] Ending request: urn:request:c73c8c04-5be2-4a3f-a9fc-d057a557bb19 (/_internal_ping) 2020-03-31 09:37:18,318 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:37:18,318 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:37:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:37:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:18,333 [541] [DEBUG] [app] Starting request: urn:request:cf60d215-d1d2-41d3-828f-3ea047728e01 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:18,333 [541] [DEBUG] [app] Starting request: urn:request:cf60d215-d1d2-41d3-828f-3ea047728e01 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:18,333 [541] [DEBUG] [app] Ending request: urn:request:cf60d215-d1d2-41d3-828f-3ea047728e01 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:18,334 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:37:18,333 [541] [DEBUG] [app] Ending request: urn:request:cf60d215-d1d2-41d3-828f-3ea047728e01 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:18,335 [518] [DEBUG] [app] Ending request: urn:request:86850b81-2e49-41fc-8515-1e623e120be2 (/health/instance) 2020-03-31 09:37:18,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:37:18,334 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:37:18,335 [518] [DEBUG] [app] Ending request: urn:request:86850b81-2e49-41fc-8515-1e623e120be2 (/health/instance) 2020-03-31 09:37:18,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:37:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:37:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:31.440707+00:00 (in 9.999146 seconds) 2020-03-31 09:37:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:31 UTC)" (scheduled at 2020-03-31 09:37:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:37:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:31.440707+00:00 (in 9.999146 seconds) 2020-03-31 09:37:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:31 UTC)" (scheduled at 2020-03-31 09:37:21.440707+00:00) 2020-03-31 09:37:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:37:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:37:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:37:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:37:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:37:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:31 UTC)" executed successfully 2020-03-31 09:37:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:37:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:52.113221+00:00 (in 29.999199 seconds) 2020-03-31 09:37:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:52 UTC)" (scheduled at 2020-03-31 09:37:22.113221+00:00) securityworker stdout | 2020-03-31 09:37:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:52.113221+00:00 (in 29.999199 seconds) 2020-03-31 09:37:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:52 UTC)" (scheduled at 2020-03-31 09:37:22.113221+00:00) 2020-03-31 09:37:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:37:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:37:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:37:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:37:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:37:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:37:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:37:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:37:52 UTC)" executed successfully 2020-03-31 09:37:22,193 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:38d0774c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:22 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:37:22,193 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:38d0774c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:22 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:37:22,193 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '38d0774c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xjyRgYVExuglTdKhWnp1C8OaQpUzqaGRYbL5Gg+IfCM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:22 GMT'}. storagereplication stdout | 2020-03-31 09:37:22,193 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '38d0774c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xjyRgYVExuglTdKhWnp1C8OaQpUzqaGRYbL5Gg+IfCM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:22 GMT'}. 2020-03-31 09:37:22,274 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:22,274 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:22,275 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:22 GMT, Server-Request-ID=03eacbd8-f01e-0021-483f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '38d0774c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xjyRgYVExuglTdKhWnp1C8OaQpUzqaGRYbL5Gg+IfCM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:22 GMT'}. 2020-03-31 09:37:22,275 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:22 GMT, Server-Request-ID=03eacbd8-f01e-0021-483f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:37:22,275 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:22 GMT, Server-Request-ID=03eacbd8-f01e-0021-483f-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '38d0774c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xjyRgYVExuglTdKhWnp1C8OaQpUzqaGRYbL5Gg+IfCM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:22 GMT'}. 2020-03-31 09:37:22,275 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:22 GMT, Server-Request-ID=03eacbd8-f01e-0021-483f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:22,275 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:22 GMT, Server-Request-ID=03eacbd8-f01e-0021-483f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:22,276 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:37:22,275 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=38d0774c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:22 GMT, Server-Request-ID=03eacbd8-f01e-0021-483f-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:22,276 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:37:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:31.498397+00:00 (in 7.174117 seconds) 2020-03-31 09:37:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:34 UTC)" (scheduled at 2020-03-31 09:37:24.323352+00:00) 2020-03-31 09:37:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:37:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:31.498397+00:00 (in 7.174117 seconds) 2020-03-31 09:37:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:34 UTC)" (scheduled at 2020-03-31 09:37:24.323352+00:00) 2020-03-31 09:37:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 24, 325078), True, datetime.datetime(2020, 3, 31, 9, 37, 24, 325078), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:37:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 24, 325078), True, datetime.datetime(2020, 3, 31, 9, 37, 24, 325078), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:37:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:37:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:34 UTC)" executed successfully 2020-03-31 09:37:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:37:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:33.330112+00:00 (in 8.918315 seconds) 2020-03-31 09:37:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:24 UTC)" (scheduled at 2020-03-31 09:37:24.410927+00:00) 2020-03-31 09:37:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:37:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:33.330112+00:00 (in 8.918315 seconds) 2020-03-31 09:37:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:24 UTC)" (scheduled at 2020-03-31 09:37:24.410927+00:00) 2020-03-31 09:37:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:37:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:24 UTC)" executed successfully 2020-03-31 09:37:24,446 [527] [DEBUG] [app] Starting request: urn:request:0aa65c63-f206-4eb8-b3ad-50c2544ad001 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:24,446 [527] [DEBUG] [app] Starting request: urn:request:0aa65c63-f206-4eb8-b3ad-50c2544ad001 (/health/instance) 2020-03-31 09:37:24,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:24,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:24,454 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:24,454 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:24,456 [552] [DEBUG] [app] Starting request: urn:request:75f60646-f72d-4e3a-920b-dab408301a1b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:24,456 [552] [DEBUG] [app] Starting request: urn:request:75f60646-f72d-4e3a-920b-dab408301a1b (/v1/_internal_ping) 2020-03-31 09:37:24,457 [552] [DEBUG] [app] Ending request: urn:request:75f60646-f72d-4e3a-920b-dab408301a1b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:24,457 [552] [DEBUG] [app] Ending request: urn:request:75f60646-f72d-4e3a-920b-dab408301a1b (/v1/_internal_ping) 2020-03-31 09:37:24,458 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:37:24,458 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:24,458 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:24,459 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:37:24,458 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:24,460 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:37:24,459 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:37:24,460 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:24,462 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:24,463 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:24,462 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:24,463 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:24,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:24,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:24,489 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:24,489 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:24,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:24,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:24,495 [520] [DEBUG] [app] Starting request: urn:request:f3408f69-ced4-4c77-84a4-89d9c604dacc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:24,495 [520] [DEBUG] [app] Starting request: urn:request:f3408f69-ced4-4c77-84a4-89d9c604dacc (/_internal_ping) 2020-03-31 09:37:24,496 [520] [DEBUG] [app] Ending request: urn:request:f3408f69-ced4-4c77-84a4-89d9c604dacc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:24,496 [520] [DEBUG] [app] Ending request: urn:request:f3408f69-ced4-4c77-84a4-89d9c604dacc (/_internal_ping) 2020-03-31 09:37:24,497 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:24,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:37:24,497 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:24,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:37:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:24,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:24,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:24,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:24,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:37:24,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:24,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:24,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:24,511 [541] [DEBUG] [app] Starting request: urn:request:2880833a-4b30-4414-9edb-d6e9226fa9f2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:24,511 [541] [DEBUG] [app] Starting request: urn:request:2880833a-4b30-4414-9edb-d6e9226fa9f2 (/c1/_internal_ping) 2020-03-31 09:37:24,512 [541] [DEBUG] [app] Ending request: urn:request:2880833a-4b30-4414-9edb-d6e9226fa9f2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:24,513 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:37:24,512 [541] [DEBUG] [app] Ending request: urn:request:2880833a-4b30-4414-9edb-d6e9226fa9f2 (/c1/_internal_ping) 2020-03-31 09:37:24,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:37:24,513 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:24,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:24,514 [527] [DEBUG] [app] Ending request: urn:request:0aa65c63-f206-4eb8-b3ad-50c2544ad001 (/health/instance) 2020-03-31 09:37:24,515 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:24,514 [527] [DEBUG] [app] Ending request: urn:request:0aa65c63-f206-4eb8-b3ad-50c2544ad001 (/health/instance) 2020-03-31 09:37:24,515 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:24,515 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:37:24,515 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:37:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:36.924497+00:00 (in 9.999167 seconds) storagereplication stdout | 2020-03-31 09:37:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:36.924497+00:00 (in 9.999167 seconds) 2020-03-31 09:37:28,273 [518] [DEBUG] [app] Starting request: urn:request:1d5a58c3-6080-4bb6-bdb0-45aab0ba2092 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:28,273 [518] [DEBUG] [app] Starting request: urn:request:1d5a58c3-6080-4bb6-bdb0-45aab0ba2092 (/health/instance) 2020-03-31 09:37:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:28,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:28,281 [544] [DEBUG] [app] Starting request: urn:request:bb390793-6e44-4cd6-8caa-d4c3fab49c96 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:28,281 [544] [DEBUG] [app] Starting request: urn:request:bb390793-6e44-4cd6-8caa-d4c3fab49c96 (/v1/_internal_ping) 2020-03-31 09:37:28,282 [544] [DEBUG] [app] Ending request: urn:request:bb390793-6e44-4cd6-8caa-d4c3fab49c96 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:28,282 [544] [DEBUG] [app] Ending request: urn:request:bb390793-6e44-4cd6-8caa-d4c3fab49c96 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:28,283 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:37:28,283 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:28,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:28,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:37:28,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:28,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:28,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:28,287 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:28,287 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:28,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:28,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:28,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:28,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:28,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:28,311 [518] [DEBUG] [app] Starting request: urn:request:e31f6efb-897b-4682-8e7b-e3dc5e8b26a5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:28,311 [518] [DEBUG] [app] Starting request: urn:request:e31f6efb-897b-4682-8e7b-e3dc5e8b26a5 (/_internal_ping) 2020-03-31 09:37:28,311 [518] [DEBUG] [app] Ending request: urn:request:e31f6efb-897b-4682-8e7b-e3dc5e8b26a5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:28,311 [518] [DEBUG] [app] Ending request: urn:request:e31f6efb-897b-4682-8e7b-e3dc5e8b26a5 (/_internal_ping) 2020-03-31 09:37:28,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:37:28,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:37:28,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:28,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:28,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:37:28,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:37:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:28,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:28,323 [534] [DEBUG] [app] Starting request: urn:request:2781d3f8-8088-4361-85d1-1cc7a67fd5ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:28,323 [534] [DEBUG] [app] Starting request: urn:request:2781d3f8-8088-4361-85d1-1cc7a67fd5ec (/c1/_internal_ping) 2020-03-31 09:37:28,324 [534] [DEBUG] [app] Ending request: urn:request:2781d3f8-8088-4361-85d1-1cc7a67fd5ec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:28,324 [534] [DEBUG] [app] Ending request: urn:request:2781d3f8-8088-4361-85d1-1cc7a67fd5ec (/c1/_internal_ping) 2020-03-31 09:37:28,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:28,325 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:28,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:28,326 [518] [DEBUG] [app] Ending request: urn:request:1d5a58c3-6080-4bb6-bdb0-45aab0ba2092 (/health/instance) 2020-03-31 09:37:28,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:37:28,325 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:28,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:37:28,326 [518] [DEBUG] [app] Ending request: urn:request:1d5a58c3-6080-4bb6-bdb0-45aab0ba2092 (/health/instance) 2020-03-31 09:37:28,326 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:28,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:37:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:37:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:00.337699+00:00 (in 29.999255 seconds) buildlogsarchiver stdout | 2020-03-31 09:37:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:00.337699+00:00 (in 29.999255 seconds) 2020-03-31 09:37:30,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:00 UTC)" (scheduled at 2020-03-31 09:37:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:37:30,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:00 UTC)" (scheduled at 2020-03-31 09:37:30.337699+00:00) 2020-03-31 09:37:30,344 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 37, 30, 342401), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:37:30,344 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 37, 30, 342401), False, 50, 1, 0]) 2020-03-31 09:37:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:37:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:37:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:37:30,359 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:37:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:00 UTC)" executed successfully 2020-03-31 09:37:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:37:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:41.440707+00:00 (in 9.999138 seconds) 2020-03-31 09:37:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:41 UTC)" (scheduled at 2020-03-31 09:37:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:37:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:41.440707+00:00 (in 9.999138 seconds) 2020-03-31 09:37:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:41 UTC)" (scheduled at 2020-03-31 09:37:31.440707+00:00) 2020-03-31 09:37:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:37:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:37:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:37:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:37:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:37:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:41 UTC)" executed successfully 2020-03-31 09:37:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:34.323352+00:00 (in 2.824076 seconds) 2020-03-31 09:37:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:31 UTC)" (scheduled at 2020-03-31 09:37:31.498397+00:00) notificationworker stdout | 2020-03-31 09:37:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:34.323352+00:00 (in 2.824076 seconds) 2020-03-31 09:37:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:31 UTC)" (scheduled at 2020-03-31 09:37:31.498397+00:00) 2020-03-31 09:37:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), u'notification/%']) notificationworker stdout | 2020-03-31 09:37:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), u'notification/%']) 2020-03-31 09:37:31,516 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), True, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:37:31,516 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), True, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), 0, u'notification/%']) 2020-03-31 09:37:31,521 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), True, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), u'notification/%']) notificationworker stdout | 2020-03-31 09:37:31,521 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), True, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 37, 31, 500222), u'notification/%']) 2020-03-31 09:37:31,524 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:37:31,524 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:31,524 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:31,524 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:42:31 UTC)" executed successfully 2020-03-31 09:37:31,527 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:37:31,527 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:31,528 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:37:31,528 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:31,529 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics notificationworker stdout | 2020-03-31 09:37:31,529 [175] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:37:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:37:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:03.330112+00:00 (in 29.998786 seconds) 2020-03-31 09:37:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:03 UTC)" (scheduled at 2020-03-31 09:37:33.330112+00:00) 2020-03-31 09:37:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:37:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:03.330112+00:00 (in 29.998786 seconds) 2020-03-31 09:37:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:03 UTC)" (scheduled at 2020-03-31 09:37:33.330112+00:00) 2020-03-31 09:37:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 33, 332278), True, datetime.datetime(2020, 3, 31, 9, 37, 33, 332278), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:37:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 33, 332278), True, datetime.datetime(2020, 3, 31, 9, 37, 33, 332278), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:37:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:37:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:37:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:03 UTC)" executed successfully 2020-03-31 09:37:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:37:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:43.539999+00:00 (in 10.171622 seconds) 2020-03-31 09:37:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:33 UTC)" (scheduled at 2020-03-31 09:37:33.366791+00:00) 2020-03-31 09:37:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:37:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:43.539999+00:00 (in 10.171622 seconds) 2020-03-31 09:37:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:33 UTC)" (scheduled at 2020-03-31 09:37:33.366791+00:00) 2020-03-31 09:37:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 33, 369025), True, datetime.datetime(2020, 3, 31, 9, 37, 33, 369025), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:37:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 33, 369025), True, datetime.datetime(2020, 3, 31, 9, 37, 33, 369025), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:37:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:37:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:37:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:33,385 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:33 UTC)" executed successfully 2020-03-31 09:37:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:34,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:44.323352+00:00 (in 9.997053 seconds) 2020-03-31 09:37:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:44 UTC)" (scheduled at 2020-03-31 09:37:34.323352+00:00) 2020-03-31 09:37:34,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:37:34,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:44.323352+00:00 (in 9.997053 seconds) 2020-03-31 09:37:34,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:44 UTC)" (scheduled at 2020-03-31 09:37:34.323352+00:00) 2020-03-31 09:37:34,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 34, 327062), True, datetime.datetime(2020, 3, 31, 9, 37, 34, 327062), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:37:34,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 34, 327062), True, datetime.datetime(2020, 3, 31, 9, 37, 34, 327062), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:37:34,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:34,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:37:34,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:34,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:44 UTC)" executed successfully 2020-03-31 09:37:34,446 [526] [DEBUG] [app] Starting request: urn:request:f71e524b-f91e-4982-82f4-f629920ad7a7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:34,446 [526] [DEBUG] [app] Starting request: urn:request:f71e524b-f91e-4982-82f4-f629920ad7a7 (/health/instance) 2020-03-31 09:37:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:34,453 [554] [DEBUG] [app] Starting request: urn:request:f82162f2-ecca-4b37-bdee-4aeaff642e97 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:34,453 [554] [DEBUG] [app] Starting request: urn:request:f82162f2-ecca-4b37-bdee-4aeaff642e97 (/v1/_internal_ping) 2020-03-31 09:37:34,454 [554] [DEBUG] [app] Ending request: urn:request:f82162f2-ecca-4b37-bdee-4aeaff642e97 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:34,454 [554] [DEBUG] [app] Ending request: urn:request:f82162f2-ecca-4b37-bdee-4aeaff642e97 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:34,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:37:34,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:34,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:34,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:34,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:34,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:34,459 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:34,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:34,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:34,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:34,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:34,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:34,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:34,486 [523] [DEBUG] [app] Starting request: urn:request:58dce266-1af0-4c75-b35d-6dab46264be9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:34,486 [523] [DEBUG] [app] Starting request: urn:request:58dce266-1af0-4c75-b35d-6dab46264be9 (/_internal_ping) 2020-03-31 09:37:34,487 [523] [DEBUG] [app] Ending request: urn:request:58dce266-1af0-4c75-b35d-6dab46264be9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:34,487 [523] [DEBUG] [app] Ending request: urn:request:58dce266-1af0-4c75-b35d-6dab46264be9 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:37:34,487 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:34,487 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:34,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:37:34,488 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:34,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:34,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:34,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:37:34,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:37:34,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:34,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:34,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:37:34,495 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:37:34,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:34,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:34,499 [541] [DEBUG] [app] Starting request: urn:request:e4a9b616-4a72-44e4-84b8-0df5089e89b3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:34,499 [541] [DEBUG] [app] Starting request: urn:request:e4a9b616-4a72-44e4-84b8-0df5089e89b3 (/c1/_internal_ping) 2020-03-31 09:37:34,500 [541] [DEBUG] [app] Ending request: urn:request:e4a9b616-4a72-44e4-84b8-0df5089e89b3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:34,500 [541] [DEBUG] [app] Ending request: urn:request:e4a9b616-4a72-44e4-84b8-0df5089e89b3 (/c1/_internal_ping) 2020-03-31 09:37:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.001) 2020-03-31 09:37:34,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.001) 2020-03-31 09:37:34,503 [526] [DEBUG] [app] Ending request: urn:request:f71e524b-f91e-4982-82f4-f629920ad7a7 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:37:34,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:34,503 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:34,504 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:37:34,503 [526] [DEBUG] [app] Ending request: urn:request:f71e524b-f91e-4982-82f4-f629920ad7a7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:34,503 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:34,504 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:37:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:38.010934+00:00 (in 1.085659 seconds) storagereplication stdout | 2020-03-31 09:37:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:38.010934+00:00 (in 1.085659 seconds) 2020-03-31 09:37:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:46.924497+00:00 (in 8.912852 seconds) 2020-03-31 09:37:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:38 UTC)" (scheduled at 2020-03-31 09:37:38.010934+00:00) 2020-03-31 09:37:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:37:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:46.924497+00:00 (in 8.912852 seconds) 2020-03-31 09:37:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:38 UTC)" (scheduled at 2020-03-31 09:37:38.010934+00:00) 2020-03-31 09:37:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:38 UTC)" executed successfully 2020-03-31 09:37:38,272 [518] [DEBUG] [app] Starting request: urn:request:1f171907-64b6-4932-b5a1-57a259001c45 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:38,272 [518] [DEBUG] [app] Starting request: urn:request:1f171907-64b6-4932-b5a1-57a259001c45 (/health/instance) 2020-03-31 09:37:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:38,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:38,281 [538] [DEBUG] [app] Starting request: urn:request:439d89fa-7ae7-4b9f-bbc0-7b87faa5ce9f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:38,281 [538] [DEBUG] [app] Starting request: urn:request:439d89fa-7ae7-4b9f-bbc0-7b87faa5ce9f (/v1/_internal_ping) 2020-03-31 09:37:38,282 [538] [DEBUG] [app] Ending request: urn:request:439d89fa-7ae7-4b9f-bbc0-7b87faa5ce9f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:38,282 [538] [DEBUG] [app] Ending request: urn:request:439d89fa-7ae7-4b9f-bbc0-7b87faa5ce9f (/v1/_internal_ping) 2020-03-31 09:37:38,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:38,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:37:38,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:38,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:38,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:38,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:37:38,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:37:38,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:38,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:38,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:38,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:38,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:38,292 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:37:38,292 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy 2020-03-31 09:37:38,292 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:37:38,292 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:37:38,292 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:37:38,292 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:37:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:38,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:38,302 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) storagereplication stdout | 2020-03-31 09:37:38,302 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) 2020-03-31 09:37:38,308 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 42, 38, 301916), True, u'2239fc24-e783-4a19-a59b-554d003f4d3c', 4]) storagereplication stdout | 2020-03-31 09:37:38,308 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 42, 38, 301916), True, u'2239fc24-e783-4a19-a59b-554d003f4d3c', 4]) 2020-03-31 09:37:38,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:38,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:38,313 [518] [DEBUG] [app] Starting request: urn:request:444a11bd-ade1-4d42-a499-110e266e7e36 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:38,313 [518] [DEBUG] [app] Starting request: urn:request:444a11bd-ade1-4d42-a499-110e266e7e36 (/_internal_ping) 2020-03-31 09:37:38,314 [518] [DEBUG] [app] Ending request: urn:request:444a11bd-ade1-4d42-a499-110e266e7e36 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:38,314 [518] [DEBUG] [app] Ending request: urn:request:444a11bd-ade1-4d42-a499-110e266e7e36 (/_internal_ping) 2020-03-31 09:37:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:37:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:37:38,316 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 38, 314330), True, datetime.datetime(2020, 3, 31, 9, 37, 38, 314330), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:37:38,316 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 38, 314330), True, datetime.datetime(2020, 3, 31, 9, 37, 38, 314330), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:37:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:37:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:37:38,321 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 57, 38, 314330), 1, u'3e5da1dd-386e-4ef7-9382-3f5de9ee1324', 2, u'ac14061e-bfca-44ec-8495-8cc1007b8301']) storagereplication stdout | 2020-03-31 09:37:38,321 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 57, 38, 314330), 1, u'3e5da1dd-386e-4ef7-9382-3f5de9ee1324', 2, u'ac14061e-bfca-44ec-8495-8cc1007b8301']) 2020-03-31 09:37:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:37:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:37:38,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:38,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:38,327 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} storagereplication stdout | 2020-03-31 09:37:38,327 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:37:38,328 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:38,328 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 storagereplication stdout | 2020-03-31 09:37:38,328 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:38,328 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 2020-03-31 09:37:38,328 [541] [DEBUG] [app] Starting request: urn:request:3fff8ad2-b5c1-422a-a1b6-bda36e3e0045 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:38,328 [541] [DEBUG] [app] Starting request: urn:request:3fff8ad2-b5c1-422a-a1b6-bda36e3e0045 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:38,329 [541] [DEBUG] [app] Ending request: urn:request:3fff8ad2-b5c1-422a-a1b6-bda36e3e0045 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:38,330 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:37:38,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:37:38,329 [541] [DEBUG] [app] Ending request: urn:request:3fff8ad2-b5c1-422a-a1b6-bda36e3e0045 (/c1/_internal_ping) 2020-03-31 09:37:38,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 storagereplication stdout | 2020-03-31 09:37:38,330 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) gunicorn-verbs stdout | 2020-03-31 09:37:38,330 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:38,332 [518] [DEBUG] [app] Ending request: urn:request:1f171907-64b6-4932-b5a1-57a259001c45 (/health/instance) 2020-03-31 09:37:38,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:38,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:38,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:37:38,332 [518] [DEBUG] [app] Ending request: urn:request:1f171907-64b6-4932-b5a1-57a259001c45 (/health/instance) 2020-03-31 09:37:38,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:38,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:37:38,345 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:37:38,345 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:37:38,349 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) storagereplication stdout | 2020-03-31 09:37:38,349 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) 2020-03-31 09:37:38,354 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:37:38,354 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:37:38,355 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:42729578-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:38 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:37:38,355 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42729578-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '42729578-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OFjFMY6YBu1QTsCny+RKGtQa+8xwonN9zz7rNE36cbg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. storagereplication stdout | 2020-03-31 09:37:38,355 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:42729578-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:38 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:37:38,355 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42729578-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '42729578-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OFjFMY6YBu1QTsCny+RKGtQa+8xwonN9zz7rNE36cbg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. 2020-03-31 09:37:38,364 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:37:38,364 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:37:38,364 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42729578-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:37 GMT, Server-Request-ID=2fc7845b-001e-006b-8040-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '42729578-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OFjFMY6YBu1QTsCny+RKGtQa+8xwonN9zz7rNE36cbg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. storagereplication stdout | 2020-03-31 09:37:38,364 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42729578-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:37 GMT, Server-Request-ID=2fc7845b-001e-006b-8040-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '42729578-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:OFjFMY6YBu1QTsCny+RKGtQa+8xwonN9zz7rNE36cbg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. 2020-03-31 09:37:38,365 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us storagereplication stdout | 2020-03-31 09:37:38,365 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us 2020-03-31 09:37:38,365 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:38,366 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:37:38,365 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:38,366 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:37:38,366 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:42744a12-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=utWC3SPeKvhs1bubcGvPkFfSv/TXL6p1zzIhTrm5IdA%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A38Z x-ms-date:Tue, 31 Mar 2020 09:37:38 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:37:38,366 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42744a12-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=utWC3SPeKvhs1bubcGvPkFfSv/TXL6p1zzIhTrm5IdA%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A38Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:a52Hv3nNuVEjBNvoMnRsYeLTDAMjscAnSV/ueVoDTx8=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '42744a12-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. storagereplication stdout | 2020-03-31 09:37:38,366 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:42744a12-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=utWC3SPeKvhs1bubcGvPkFfSv/TXL6p1zzIhTrm5IdA%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A38Z x-ms-date:Tue, 31 Mar 2020 09:37:38 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:37:38,366 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42744a12-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=utWC3SPeKvhs1bubcGvPkFfSv/TXL6p1zzIhTrm5IdA%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A38Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:a52Hv3nNuVEjBNvoMnRsYeLTDAMjscAnSV/ueVoDTx8=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '42744a12-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. 2020-03-31 09:37:38,404 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:37:38,404 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 2020-03-31 09:37:38,405 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42744a12-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:37 GMT, Server-Request-ID=2fc7845e-001e-006b-0140-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=utWC3SPeKvhs1bubcGvPkFfSv/TXL6p1zzIhTrm5IdA%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A38Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:a52Hv3nNuVEjBNvoMnRsYeLTDAMjscAnSV/ueVoDTx8=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '42744a12-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. storagereplication stdout | 2020-03-31 09:37:38,405 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=42744a12-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:37 GMT, Server-Request-ID=2fc7845e-001e-006b-0140-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=utWC3SPeKvhs1bubcGvPkFfSv/TXL6p1zzIhTrm5IdA%3D&sv=2017-07-29&se=2020-03-31T09%3A38%3A38Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:a52Hv3nNuVEjBNvoMnRsYeLTDAMjscAnSV/ueVoDTx8=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '42744a12-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. 2020-03-31 09:37:38,405 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:37:38,405 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:37:38,406 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:427a5038-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:38 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:37:38,406 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '427a5038-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zkTwNZ48DkTohUO2Wy7iwXWK2jolSIHJ/dWWxkRj/ak=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. storagereplication stdout | 2020-03-31 09:37:38,406 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:427a5038-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:38 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:37:38,406 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '427a5038-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zkTwNZ48DkTohUO2Wy7iwXWK2jolSIHJ/dWWxkRj/ak=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. 2020-03-31 09:37:38,488 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:38,488 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:38,489 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:38 GMT, Server-Request-ID=03eb138b-f01e-0021-3640-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '427a5038-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zkTwNZ48DkTohUO2Wy7iwXWK2jolSIHJ/dWWxkRj/ak=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. 2020-03-31 09:37:38,490 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:38 GMT, Server-Request-ID=03eb138b-f01e-0021-3640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:38,490 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:38 GMT, Server-Request-ID=03eb138b-f01e-0021-3640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:37:38,489 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:38 GMT, Server-Request-ID=03eb138b-f01e-0021-3640-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '427a5038-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:zkTwNZ48DkTohUO2Wy7iwXWK2jolSIHJ/dWWxkRj/ak=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:38 GMT'}. 2020-03-31 09:37:38,490 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:38 GMT, Server-Request-ID=03eb138b-f01e-0021-3640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:38,490 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=427a5038-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:38 GMT, Server-Request-ID=03eb138b-f01e-0021-3640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:38,490 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:37:38,490 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:37:40,492 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:43b8b516-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:40 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:37:40,492 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:43b8b516-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:40 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:37:40,493 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '43b8b516-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:J4ypKkRtg/2qA3bdWm4D0rvX1iT/VVD6ZFUF6PQUB6U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:40 GMT'}. storagereplication stdout | 2020-03-31 09:37:40,493 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '43b8b516-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:J4ypKkRtg/2qA3bdWm4D0rvX1iT/VVD6ZFUF6PQUB6U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:40 GMT'}. 2020-03-31 09:37:40,573 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:40,573 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:40,574 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:40 GMT, Server-Request-ID=03eb1cec-f01e-0021-3e40-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '43b8b516-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:J4ypKkRtg/2qA3bdWm4D0rvX1iT/VVD6ZFUF6PQUB6U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:40 GMT'}. 2020-03-31 09:37:40,574 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:40 GMT, Server-Request-ID=03eb1cec-f01e-0021-3e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:40,574 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:40 GMT, Server-Request-ID=03eb1cec-f01e-0021-3e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:37:40,574 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:40 GMT, Server-Request-ID=03eb1cec-f01e-0021-3e40-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '43b8b516-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:J4ypKkRtg/2qA3bdWm4D0rvX1iT/VVD6ZFUF6PQUB6U=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:40 GMT'}. 2020-03-31 09:37:40,574 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:40 GMT, Server-Request-ID=03eb1cec-f01e-0021-3e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:40,574 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=43b8b516-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:40 GMT, Server-Request-ID=03eb1cec-f01e-0021-3e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:40,574 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:37:40,574 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:37:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:37:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:51.440707+00:00 (in 9.999256 seconds) 2020-03-31 09:37:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:51 UTC)" (scheduled at 2020-03-31 09:37:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:37:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:51.440707+00:00 (in 9.999256 seconds) 2020-03-31 09:37:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:51 UTC)" (scheduled at 2020-03-31 09:37:41.440707+00:00) 2020-03-31 09:37:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:37:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:37:41,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:37:41,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:37:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:37:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:41,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:51 UTC)" executed successfully 2020-03-31 09:37:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:37:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:33.366791+00:00 (in 49.826074 seconds) 2020-03-31 09:37:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:43 UTC)" (scheduled at 2020-03-31 09:37:43.539999+00:00) 2020-03-31 09:37:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:37:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:33.366791+00:00 (in 49.826074 seconds) 2020-03-31 09:37:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:43 UTC)" (scheduled at 2020-03-31 09:37:43.539999+00:00) 2020-03-31 09:37:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:43 UTC)" executed successfully 2020-03-31 09:37:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:49.498966+00:00 (in 5.174649 seconds) 2020-03-31 09:37:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:54 UTC)" (scheduled at 2020-03-31 09:37:44.323352+00:00) 2020-03-31 09:37:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:37:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:49.498966+00:00 (in 5.174649 seconds) 2020-03-31 09:37:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:54 UTC)" (scheduled at 2020-03-31 09:37:44.323352+00:00) 2020-03-31 09:37:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 44, 325282), True, datetime.datetime(2020, 3, 31, 9, 37, 44, 325282), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:37:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 44, 325282), True, datetime.datetime(2020, 3, 31, 9, 37, 44, 325282), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:37:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:37:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:54 UTC)" executed successfully 2020-03-31 09:37:44,446 [527] [DEBUG] [app] Starting request: urn:request:a1092e27-5178-487d-85d0-4faaaec7f8cc (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:44,446 [527] [DEBUG] [app] Starting request: urn:request:a1092e27-5178-487d-85d0-4faaaec7f8cc (/health/instance) 2020-03-31 09:37:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:44,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:44,454 [551] [DEBUG] [app] Starting request: urn:request:c43a998a-4ce0-4134-b1ea-e1e4ec728424 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:44,454 [551] [DEBUG] [app] Starting request: urn:request:c43a998a-4ce0-4134-b1ea-e1e4ec728424 (/v1/_internal_ping) 2020-03-31 09:37:44,455 [551] [DEBUG] [app] Ending request: urn:request:c43a998a-4ce0-4134-b1ea-e1e4ec728424 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:44,455 [551] [DEBUG] [app] Ending request: urn:request:c43a998a-4ce0-4134-b1ea-e1e4ec728424 (/v1/_internal_ping) 2020-03-31 09:37:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:37:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:37:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:44,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:44,459 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:44,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:44,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:44,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:44,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:44,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:44,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:44,487 [518] [DEBUG] [app] Starting request: urn:request:c91f894d-1952-4de7-9876-489ca6ffd34a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:44,487 [518] [DEBUG] [app] Starting request: urn:request:c91f894d-1952-4de7-9876-489ca6ffd34a (/_internal_ping) 2020-03-31 09:37:44,488 [518] [DEBUG] [app] Ending request: urn:request:c91f894d-1952-4de7-9876-489ca6ffd34a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:44,488 [518] [DEBUG] [app] Ending request: urn:request:c91f894d-1952-4de7-9876-489ca6ffd34a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:37:44,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:44,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:37:44,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:44,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:37:44,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:37:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:44,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:37:44,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:37:44,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:44,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:44,504 [541] [DEBUG] [app] Starting request: urn:request:4ae0ee3b-0a28-4fad-8e32-8f409f4b2c4a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:44,504 [541] [DEBUG] [app] Starting request: urn:request:4ae0ee3b-0a28-4fad-8e32-8f409f4b2c4a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:44,505 [541] [DEBUG] [app] Ending request: urn:request:4ae0ee3b-0a28-4fad-8e32-8f409f4b2c4a (/c1/_internal_ping) 2020-03-31 09:37:44,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:44,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:37:44,505 [541] [DEBUG] [app] Ending request: urn:request:4ae0ee3b-0a28-4fad-8e32-8f409f4b2c4a (/c1/_internal_ping) 2020-03-31 09:37:44,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:44,506 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:44,507 [527] [DEBUG] [app] Ending request: urn:request:a1092e27-5178-487d-85d0-4faaaec7f8cc (/health/instance) 2020-03-31 09:37:44,508 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:44,507 [527] [DEBUG] [app] Ending request: urn:request:a1092e27-5178-487d-85d0-4faaaec7f8cc (/health/instance) 2020-03-31 09:37:44,508 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:44,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:37:44,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:37:44,575 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:4627a03c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:44 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:37:44,575 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:4627a03c-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:44 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:37:44,575 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4627a03c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4eNhBr3fxPWO2d+W9bgv40zxkmpPrkaLVfEbbQY4Nh0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:44 GMT'}. storagereplication stdout | 2020-03-31 09:37:44,575 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4627a03c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4eNhBr3fxPWO2d+W9bgv40zxkmpPrkaLVfEbbQY4Nh0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:44 GMT'}. 2020-03-31 09:37:44,655 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:44,655 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:44,656 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:44 GMT, Server-Request-ID=03eb2cf5-f01e-0021-4e40-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4627a03c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4eNhBr3fxPWO2d+W9bgv40zxkmpPrkaLVfEbbQY4Nh0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:44 GMT'}. 2020-03-31 09:37:44,656 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:44 GMT, Server-Request-ID=03eb2cf5-f01e-0021-4e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:44,656 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:44 GMT, Server-Request-ID=03eb2cf5-f01e-0021-4e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:44,656 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:37:44,656 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:44 GMT, Server-Request-ID=03eb2cf5-f01e-0021-4e40-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4627a03c-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4eNhBr3fxPWO2d+W9bgv40zxkmpPrkaLVfEbbQY4Nh0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:44 GMT'}. 2020-03-31 09:37:44,656 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:44 GMT, Server-Request-ID=03eb2cf5-f01e-0021-4e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:44,656 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=4627a03c-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:44 GMT, Server-Request-ID=03eb2cf5-f01e-0021-4e40-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:44,656 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:37:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:37:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:15.064514+00:00 (in 29.996090 seconds) 2020-03-31 09:37:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:15 UTC)" (scheduled at 2020-03-31 09:37:45.064514+00:00) gcworker stdout | 2020-03-31 09:37:45,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:15.064514+00:00 (in 29.996090 seconds) 2020-03-31 09:37:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:15 UTC)" (scheduled at 2020-03-31 09:37:45.064514+00:00) 2020-03-31 09:37:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437865, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:37:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437865, 1209600, 500, 1, 0]) 2020-03-31 09:37:45,099 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:37:45,099 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:37:45,099 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:45,099 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:37:45,099 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:45,099 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:15 UTC)" executed successfully 2020-03-31 09:37:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:56.924497+00:00 (in 9.999196 seconds) storagereplication stdout | 2020-03-31 09:37:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:56.924497+00:00 (in 9.999196 seconds) 2020-03-31 09:37:48,274 [526] [DEBUG] [app] Starting request: urn:request:67b93774-0d93-4c57-8e14-6a0ea196513b (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:48,274 [526] [DEBUG] [app] Starting request: urn:request:67b93774-0d93-4c57-8e14-6a0ea196513b (/health/instance) 2020-03-31 09:37:48,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:48,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:48,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:48,280 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:48,281 [556] [DEBUG] [app] Starting request: urn:request:be2a2039-16c9-4cce-bed8-9b5e4987b8e4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:48,281 [556] [DEBUG] [app] Starting request: urn:request:be2a2039-16c9-4cce-bed8-9b5e4987b8e4 (/v1/_internal_ping) 2020-03-31 09:37:48,282 [556] [DEBUG] [app] Ending request: urn:request:be2a2039-16c9-4cce-bed8-9b5e4987b8e4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:48,282 [556] [DEBUG] [app] Ending request: urn:request:be2a2039-16c9-4cce-bed8-9b5e4987b8e4 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:37:48,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:37:48,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:48,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:48,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:48,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:48,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:48,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:48,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:48,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:48,296 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:48,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:48,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:48,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:48,313 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:48,314 [526] [DEBUG] [app] Starting request: urn:request:c787daf0-ab8b-4e06-829d-214de424b39e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:48,314 [526] [DEBUG] [app] Starting request: urn:request:c787daf0-ab8b-4e06-829d-214de424b39e (/_internal_ping) 2020-03-31 09:37:48,315 [526] [DEBUG] [app] Ending request: urn:request:c787daf0-ab8b-4e06-829d-214de424b39e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:48,315 [526] [DEBUG] [app] Ending request: urn:request:c787daf0-ab8b-4e06-829d-214de424b39e (/_internal_ping) 2020-03-31 09:37:48,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:37:48,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:48,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:37:48,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:48,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:48,319 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:48,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:37:48,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:37:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:37:48,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:48,326 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:48,327 [537] [DEBUG] [app] Starting request: urn:request:665a0b57-db16-47dc-8e4a-86ebc7e9b7ed (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:48,327 [537] [DEBUG] [app] Starting request: urn:request:665a0b57-db16-47dc-8e4a-86ebc7e9b7ed (/c1/_internal_ping) 2020-03-31 09:37:48,328 [537] [DEBUG] [app] Ending request: urn:request:665a0b57-db16-47dc-8e4a-86ebc7e9b7ed (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:48,328 [537] [DEBUG] [app] Ending request: urn:request:665a0b57-db16-47dc-8e4a-86ebc7e9b7ed (/c1/_internal_ping) 2020-03-31 09:37:48,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:48,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:48,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:37:48,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:48,330 [526] [DEBUG] [app] Ending request: urn:request:67b93774-0d93-4c57-8e14-6a0ea196513b (/health/instance) 2020-03-31 09:37:48,330 [526] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:37:48,330 [526] [DEBUG] [app] Ending request: urn:request:67b93774-0d93-4c57-8e14-6a0ea196513b (/health/instance) 2020-03-31 09:37:48,330 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:48,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:37:48,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:37:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:54.323352+00:00 (in 4.823550 seconds) 2020-03-31 09:37:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:49 UTC)" (scheduled at 2020-03-31 09:37:49.498966+00:00) 2020-03-31 09:37:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:54.323352+00:00 (in 4.823550 seconds) 2020-03-31 09:37:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:49 UTC)" (scheduled at 2020-03-31 09:37:49.498966+00:00) 2020-03-31 09:37:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:49 UTC)" executed successfully 2020-03-31 09:37:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:37:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:59.560326+00:00 (in 8.448404 seconds) 2020-03-31 09:37:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:51 UTC)" (scheduled at 2020-03-31 09:37:51.110610+00:00) 2020-03-31 09:37:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:37:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:37:59.560326+00:00 (in 8.448404 seconds) 2020-03-31 09:37:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:51 UTC)" (scheduled at 2020-03-31 09:37:51.110610+00:00) 2020-03-31 09:37:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 51, 112825), True, datetime.datetime(2020, 3, 31, 9, 37, 51, 112825), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:37:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 51, 112825), True, datetime.datetime(2020, 3, 31, 9, 37, 51, 112825), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:37:51,132 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:37:51,132 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:37:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:51 UTC)" executed successfully 2020-03-31 09:37:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:37:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:01.440707+00:00 (in 9.999119 seconds) 2020-03-31 09:37:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:01 UTC)" (scheduled at 2020-03-31 09:37:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:37:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:01.440707+00:00 (in 9.999119 seconds) 2020-03-31 09:37:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:01 UTC)" (scheduled at 2020-03-31 09:37:51.440707+00:00) 2020-03-31 09:37:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:37:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:37:51,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:37:51,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:37:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:37:51,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:01 UTC)" executed successfully 2020-03-31 09:37:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:37:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:22.113221+00:00 (in 29.998995 seconds) 2020-03-31 09:37:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:22 UTC)" (scheduled at 2020-03-31 09:37:52.113221+00:00) securityworker stdout | 2020-03-31 09:37:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:22.113221+00:00 (in 29.998995 seconds) 2020-03-31 09:37:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:22 UTC)" (scheduled at 2020-03-31 09:37:52.113221+00:00) 2020-03-31 09:37:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:37:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:37:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:37:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:37:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:37:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:37:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:37:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:22 UTC)" executed successfully 2020-03-31 09:37:52,663 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:4af9cba8-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:52 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:37:52,663 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:4af9cba8-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:37:52 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:37:52,663 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4af9cba8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ux/gyL/KBnqpYxKOgHt/bbfBH66IjEzRu74LtIukx3Q=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:52 GMT'}. storagereplication stdout | 2020-03-31 09:37:52,663 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4af9cba8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ux/gyL/KBnqpYxKOgHt/bbfBH66IjEzRu74LtIukx3Q=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:52 GMT'}. 2020-03-31 09:37:52,753 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:37:52,753 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:37:52,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:52 GMT, Server-Request-ID=03eb54a4-f01e-0021-4640-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4af9cba8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ux/gyL/KBnqpYxKOgHt/bbfBH66IjEzRu74LtIukx3Q=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:52 GMT'}. 2020-03-31 09:37:52,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:52 GMT, Server-Request-ID=03eb54a4-f01e-0021-4640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:52,754 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:52 GMT, Server-Request-ID=03eb54a4-f01e-0021-4640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:52,754 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:37:52,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:37:52 GMT, Server-Request-ID=03eb54a4-f01e-0021-4640-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '4af9cba8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ux/gyL/KBnqpYxKOgHt/bbfBH66IjEzRu74LtIukx3Q=', 'x-ms-date': 'Tue, 31 Mar 2020 09:37:52 GMT'}. 2020-03-31 09:37:52,754 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:37:52 GMT, Server-Request-ID=03eb54a4-f01e-0021-4640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:52,754 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=4af9cba8-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:37:52 GMT, Server-Request-ID=03eb54a4-f01e-0021-4640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:37:52,754 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:37:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:37:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:04 UTC)" (scheduled at 2020-03-31 09:37:54.323352+00:00) 2020-03-31 09:37:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:37:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:04 UTC)" (scheduled at 2020-03-31 09:37:54.323352+00:00) 2020-03-31 09:37:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:37:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:04.323352+00:00 (in 9.999189 seconds) notificationworker stdout | 2020-03-31 09:37:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:04.323352+00:00 (in 9.999189 seconds) 2020-03-31 09:37:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 54, 325112), True, datetime.datetime(2020, 3, 31, 9, 37, 54, 325112), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:37:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 37, 54, 325112), True, datetime.datetime(2020, 3, 31, 9, 37, 54, 325112), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:37:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:37:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:37:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:37:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:04 UTC)" executed successfully 2020-03-31 09:37:54,446 [527] [DEBUG] [app] Starting request: urn:request:fa9091f0-cd77-4990-aac7-d8bc7373b8ba (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:54,446 [527] [DEBUG] [app] Starting request: urn:request:fa9091f0-cd77-4990-aac7-d8bc7373b8ba (/health/instance) 2020-03-31 09:37:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:54,453 [551] [DEBUG] [app] Starting request: urn:request:0ddadea4-7710-4a3b-9fc6-b61ff56730e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:54,453 [551] [DEBUG] [app] Starting request: urn:request:0ddadea4-7710-4a3b-9fc6-b61ff56730e9 (/v1/_internal_ping) 2020-03-31 09:37:54,454 [551] [DEBUG] [app] Ending request: urn:request:0ddadea4-7710-4a3b-9fc6-b61ff56730e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:54,454 [551] [DEBUG] [app] Ending request: urn:request:0ddadea4-7710-4a3b-9fc6-b61ff56730e9 (/v1/_internal_ping) 2020-03-31 09:37:54,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:37:54,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:37:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:54,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:54,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:54,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:54,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:54,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:54,460 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:54,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:54,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:54,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:54,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:54,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:54,484 [518] [DEBUG] [app] Starting request: urn:request:1b79cbf2-084e-4a68-9ebd-6861763a2d19 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:54,484 [518] [DEBUG] [app] Starting request: urn:request:1b79cbf2-084e-4a68-9ebd-6861763a2d19 (/_internal_ping) 2020-03-31 09:37:54,485 [518] [DEBUG] [app] Ending request: urn:request:1b79cbf2-084e-4a68-9ebd-6861763a2d19 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:54,485 [518] [DEBUG] [app] Ending request: urn:request:1b79cbf2-084e-4a68-9ebd-6861763a2d19 (/_internal_ping) 2020-03-31 09:37:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:54,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:37:54,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:54,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:37:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:54,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:37:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:37:54,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:54,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:37:54,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:37:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:54,498 [541] [DEBUG] [app] Starting request: urn:request:f50fb0e2-42d4-4c05-aba0-1ec0fdb280b0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:54,498 [541] [DEBUG] [app] Starting request: urn:request:f50fb0e2-42d4-4c05-aba0-1ec0fdb280b0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:54,498 [541] [DEBUG] [app] Ending request: urn:request:f50fb0e2-42d4-4c05-aba0-1ec0fdb280b0 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:54,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:54,499 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:37:54,498 [541] [DEBUG] [app] Ending request: urn:request:f50fb0e2-42d4-4c05-aba0-1ec0fdb280b0 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:54,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:54,500 [527] [DEBUG] [app] Ending request: urn:request:fa9091f0-cd77-4990-aac7-d8bc7373b8ba (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:37:54,499 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:54,500 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:54,501 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:37:54,500 [527] [DEBUG] [app] Ending request: urn:request:fa9091f0-cd77-4990-aac7-d8bc7373b8ba (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:54,500 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:37:54,501 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:37:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:37:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:06.924497+00:00 (in 9.999091 seconds) storagereplication stdout | 2020-03-31 09:37:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:37:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:37:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:06.924497+00:00 (in 9.999091 seconds) 2020-03-31 09:37:58,273 [518] [DEBUG] [app] Starting request: urn:request:b10bfb40-44e2-4a7b-8850-f6f689392a32 (/health/instance) gunicorn-web stdout | 2020-03-31 09:37:58,273 [518] [DEBUG] [app] Starting request: urn:request:b10bfb40-44e2-4a7b-8850-f6f689392a32 (/health/instance) 2020-03-31 09:37:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:58,280 [554] [DEBUG] [app] Starting request: urn:request:eb4cc8c2-d579-4de4-8bd3-1377097f478a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:58,280 [554] [DEBUG] [app] Starting request: urn:request:eb4cc8c2-d579-4de4-8bd3-1377097f478a (/v1/_internal_ping) 2020-03-31 09:37:58,281 [554] [DEBUG] [app] Ending request: urn:request:eb4cc8c2-d579-4de4-8bd3-1377097f478a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:37:58,281 [554] [DEBUG] [app] Ending request: urn:request:eb4cc8c2-d579-4de4-8bd3-1377097f478a (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:58,282 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:37:58,282 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:58,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:58,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:37:58,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:37:58,284 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:37:58,284 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:37:58,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:37:58,286 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:37:58,286 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:37:58,286 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:37:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:37:58,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:37:58,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:37:58,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:37:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:58,314 [518] [DEBUG] [app] Starting request: urn:request:adcef017-f412-4596-a2d6-98f56df23f0e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:58,314 [518] [DEBUG] [app] Starting request: urn:request:adcef017-f412-4596-a2d6-98f56df23f0e (/_internal_ping) 2020-03-31 09:37:58,315 [518] [DEBUG] [app] Ending request: urn:request:adcef017-f412-4596-a2d6-98f56df23f0e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:37:58,315 [518] [DEBUG] [app] Ending request: urn:request:adcef017-f412-4596-a2d6-98f56df23f0e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:37:58,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:37:58,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:37:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:37:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:37:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:37:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:37:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:37:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:37:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:37:58,328 [537] [DEBUG] [app] Starting request: urn:request:6630729c-a1e2-4538-b162-3ba2667fe34b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:58,328 [537] [DEBUG] [app] Starting request: urn:request:6630729c-a1e2-4538-b162-3ba2667fe34b (/c1/_internal_ping) 2020-03-31 09:37:58,329 [537] [DEBUG] [app] Ending request: urn:request:6630729c-a1e2-4538-b162-3ba2667fe34b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:37:58,329 [537] [DEBUG] [app] Ending request: urn:request:6630729c-a1e2-4538-b162-3ba2667fe34b (/c1/_internal_ping) 2020-03-31 09:37:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:58,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:37:58,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:37:58,331 [518] [DEBUG] [app] Ending request: urn:request:b10bfb40-44e2-4a7b-8850-f6f689392a32 (/health/instance) 2020-03-31 09:37:58,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:37:58,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:37:58,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:37:58,331 [518] [DEBUG] [app] Ending request: urn:request:b10bfb40-44e2-4a7b-8850-f6f689392a32 (/health/instance) 2020-03-31 09:37:58,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:37:58,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:37:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:37:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:37:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:37:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:37:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:59 UTC)" (scheduled at 2020-03-31 09:37:59.560326+00:00) 2020-03-31 09:37:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:37:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:37:59 UTC)" (scheduled at 2020-03-31 09:37:59.560326+00:00) 2020-03-31 09:37:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:37:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:59 UTC)" executed successfully 2020-03-31 09:37:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:51.110610+00:00 (in 51.549366 seconds) namespacegcworker stdout | 2020-03-31 09:37:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:38:59 UTC)" executed successfully 2020-03-31 09:37:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:51.110610+00:00 (in 51.549366 seconds) 2020-03-31 09:38:00,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:38:00,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:00,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:30 UTC)" (scheduled at 2020-03-31 09:38:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:38:00,342 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:30 UTC)" (scheduled at 2020-03-31 09:38:00.337699+00:00) 2020-03-31 09:38:00,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:30.337699+00:00 (in 29.995856 seconds) buildlogsarchiver stdout | 2020-03-31 09:38:00,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:30.337699+00:00 (in 29.995856 seconds) 2020-03-31 09:38:00,344 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 38, 0, 342576), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:38:00,344 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 38, 0, 342576), False, 50, 1, 0]) 2020-03-31 09:38:00,361 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:38:00,361 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:38:00,362 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:00,362 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:38:00,362 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:00,362 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:30 UTC)" executed successfully 2020-03-31 09:38:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:11.440707+00:00 (in 9.999240 seconds) 2020-03-31 09:38:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:11 UTC)" (scheduled at 2020-03-31 09:38:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:38:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:38:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:11.440707+00:00 (in 9.999240 seconds) 2020-03-31 09:38:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:11 UTC)" (scheduled at 2020-03-31 09:38:01.440707+00:00) 2020-03-31 09:38:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:38:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:38:01,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:38:01,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:38:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:38:01,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:01,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:11 UTC)" executed successfully 2020-03-31 09:38:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:38:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:24.410927+00:00 (in 21.080047 seconds) 2020-03-31 09:38:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:33 UTC)" (scheduled at 2020-03-31 09:38:03.330112+00:00) 2020-03-31 09:38:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:38:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:24.410927+00:00 (in 21.080047 seconds) 2020-03-31 09:38:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:33 UTC)" (scheduled at 2020-03-31 09:38:03.330112+00:00) 2020-03-31 09:38:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 3, 331789), True, datetime.datetime(2020, 3, 31, 9, 38, 3, 331789), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:38:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 3, 331789), True, datetime.datetime(2020, 3, 31, 9, 38, 3, 331789), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:38:03,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:38:03,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:33 UTC)" executed successfully 2020-03-31 09:38:04,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:38:04,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:04,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:14.323352+00:00 (in 9.997641 seconds) 2020-03-31 09:38:04,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:14 UTC)" (scheduled at 2020-03-31 09:38:04.323352+00:00) 2020-03-31 09:38:04,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:38:04,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:14.323352+00:00 (in 9.997641 seconds) 2020-03-31 09:38:04,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:14 UTC)" (scheduled at 2020-03-31 09:38:04.323352+00:00) 2020-03-31 09:38:04,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 4, 326439), True, datetime.datetime(2020, 3, 31, 9, 38, 4, 326439), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:38:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 4, 326439), True, datetime.datetime(2020, 3, 31, 9, 38, 4, 326439), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:38:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:04,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:38:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:38:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:14 UTC)" executed successfully 2020-03-31 09:38:04,446 [518] [DEBUG] [app] Starting request: urn:request:c35faa53-2399-4756-9da3-14efd939db3d (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:04,446 [518] [DEBUG] [app] Starting request: urn:request:c35faa53-2399-4756-9da3-14efd939db3d (/health/instance) 2020-03-31 09:38:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:04,454 [555] [DEBUG] [app] Starting request: urn:request:f6516201-defe-46ea-ab10-7e7eaaab9023 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:04,454 [555] [DEBUG] [app] Starting request: urn:request:f6516201-defe-46ea-ab10-7e7eaaab9023 (/v1/_internal_ping) 2020-03-31 09:38:04,455 [555] [DEBUG] [app] Ending request: urn:request:f6516201-defe-46ea-ab10-7e7eaaab9023 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:04,455 [555] [DEBUG] [app] Ending request: urn:request:f6516201-defe-46ea-ab10-7e7eaaab9023 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:04,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:38:04,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:04,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:04,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:04,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:04,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:04,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:04,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:04,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:04,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:04,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:04,478 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:04,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:04,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:04,483 [518] [DEBUG] [app] Starting request: urn:request:97e677b7-1a81-432e-8d08-afc9d1886374 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:04,483 [518] [DEBUG] [app] Starting request: urn:request:97e677b7-1a81-432e-8d08-afc9d1886374 (/_internal_ping) 2020-03-31 09:38:04,484 [518] [DEBUG] [app] Ending request: urn:request:97e677b7-1a81-432e-8d08-afc9d1886374 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:04,484 [518] [DEBUG] [app] Ending request: urn:request:97e677b7-1a81-432e-8d08-afc9d1886374 (/_internal_ping) 2020-03-31 09:38:04,484 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:04,484 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:04,484 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:04,484 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:04,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:04,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:38:04,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:38:04,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:04,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:04,496 [541] [DEBUG] [app] Starting request: urn:request:82e13262-138c-408c-b438-9de95f2948c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:04,496 [541] [DEBUG] [app] Starting request: urn:request:82e13262-138c-408c-b438-9de95f2948c2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:04,496 [541] [DEBUG] [app] Ending request: urn:request:82e13262-138c-408c-b438-9de95f2948c2 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:04,497 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:38:04,496 [541] [DEBUG] [app] Ending request: urn:request:82e13262-138c-408c-b438-9de95f2948c2 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:04,498 [518] [DEBUG] [app] Ending request: urn:request:c35faa53-2399-4756-9da3-14efd939db3d (/health/instance) 2020-03-31 09:38:04,498 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:38:04,497 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:04,499 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:38:04,498 [518] [DEBUG] [app] Ending request: urn:request:c35faa53-2399-4756-9da3-14efd939db3d (/health/instance) 2020-03-31 09:38:04,498 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:04,499 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:38:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:38:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:16.924497+00:00 (in 9.999200 seconds) storagereplication stdout | 2020-03-31 09:38:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:16.924497+00:00 (in 9.999200 seconds) 2020-03-31 09:38:08,273 [518] [DEBUG] [app] Starting request: urn:request:18dcc9a4-03ba-47f2-8ce3-74b8c1eedc21 (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:08,273 [518] [DEBUG] [app] Starting request: urn:request:18dcc9a4-03ba-47f2-8ce3-74b8c1eedc21 (/health/instance) 2020-03-31 09:38:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:08,279 [538] [DEBUG] [app] Starting request: urn:request:de8869d0-dd4f-45a5-aa77-ed14ef946cee (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:08,279 [538] [DEBUG] [app] Starting request: urn:request:de8869d0-dd4f-45a5-aa77-ed14ef946cee (/v1/_internal_ping) 2020-03-31 09:38:08,280 [538] [DEBUG] [app] Ending request: urn:request:de8869d0-dd4f-45a5-aa77-ed14ef946cee (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:08,280 [538] [DEBUG] [app] Ending request: urn:request:de8869d0-dd4f-45a5-aa77-ed14ef946cee (/v1/_internal_ping) 2020-03-31 09:38:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:38:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:08,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:38:08,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:08,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:08,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:08,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:08,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:08,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:08,313 [518] [DEBUG] [app] Starting request: urn:request:eebcd8ea-ee30-43d5-9f63-eb1e3bf96002 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:08,313 [518] [DEBUG] [app] Starting request: urn:request:eebcd8ea-ee30-43d5-9f63-eb1e3bf96002 (/_internal_ping) 2020-03-31 09:38:08,314 [518] [DEBUG] [app] Ending request: urn:request:eebcd8ea-ee30-43d5-9f63-eb1e3bf96002 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:08,314 [518] [DEBUG] [app] Ending request: urn:request:eebcd8ea-ee30-43d5-9f63-eb1e3bf96002 (/_internal_ping) 2020-03-31 09:38:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:38:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:08,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:38:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:38:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:38:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:08,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:08,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:08,331 [541] [DEBUG] [app] Starting request: urn:request:8b2b0c92-2491-408f-8ca6-65498770c7ea (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:08,331 [541] [DEBUG] [app] Starting request: urn:request:8b2b0c92-2491-408f-8ca6-65498770c7ea (/c1/_internal_ping) 2020-03-31 09:38:08,332 [541] [DEBUG] [app] Ending request: urn:request:8b2b0c92-2491-408f-8ca6-65498770c7ea (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:08,332 [541] [DEBUG] [app] Ending request: urn:request:8b2b0c92-2491-408f-8ca6-65498770c7ea (/c1/_internal_ping) 2020-03-31 09:38:08,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:08,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:08,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:38:08,332 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:08,334 [518] [DEBUG] [app] Ending request: urn:request:18dcc9a4-03ba-47f2-8ce3-74b8c1eedc21 (/health/instance) 2020-03-31 09:38:08,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:08,334 [518] [DEBUG] [app] Ending request: urn:request:18dcc9a4-03ba-47f2-8ce3-74b8c1eedc21 (/health/instance) 2020-03-31 09:38:08,334 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:38:08,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:38:08,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:38:08,767 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:38:08,767 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy 2020-03-31 09:38:08,767 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:38:08,767 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:38:08,767 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:38:08,767 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:38:08,777 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) storagereplication stdout | 2020-03-31 09:38:08,777 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) 2020-03-31 09:38:08,780 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 43, 8, 776475), True, u'bae61519-03c4-42e9-b0e3-e9d3ef876e69', 2]) storagereplication stdout | 2020-03-31 09:38:08,780 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 43, 8, 776475), True, u'bae61519-03c4-42e9-b0e3-e9d3ef876e69', 2]) 2020-03-31 09:38:08,786 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 8, 785337), True, datetime.datetime(2020, 3, 31, 9, 38, 8, 785337), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:38:08,786 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 8, 785337), True, datetime.datetime(2020, 3, 31, 9, 38, 8, 785337), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:38:08,790 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 58, 8, 785337), 1, u'c592bd1f-1b55-473f-8060-b31543bf7fcc', 1, u'bc7bff38-5bf4-4070-aec4-28ee90f350b5']) storagereplication stdout | 2020-03-31 09:38:08,790 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 9, 58, 8, 785337), 1, u'c592bd1f-1b55-473f-8060-b31543bf7fcc', 1, u'bc7bff38-5bf4-4070-aec4-28ee90f350b5']) 2020-03-31 09:38:08,796 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} storagereplication stdout | 2020-03-31 09:38:08,796 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:38:08,796 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:08,796 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 storagereplication stdout | 2020-03-31 09:38:08,796 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:08,796 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 2020-03-31 09:38:08,798 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:38:08,798 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:38:08,811 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:38:08,811 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:38:08,815 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) storagereplication stdout | 2020-03-31 09:38:08,815 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:38:08,819 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:38:08,819 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:38:08,819 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:549b1284-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:08 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:38:08,819 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549b1284-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '549b1284-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cG3bPdFznYsiqw3ZBr04eqGT9hWdAeyf33Wdr94jU0k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. storagereplication stdout | 2020-03-31 09:38:08,819 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:549b1284-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:08 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:38:08,819 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549b1284-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '549b1284-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cG3bPdFznYsiqw3ZBr04eqGT9hWdAeyf33Wdr94jU0k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. 2020-03-31 09:38:08,828 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:38:08,828 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:38:08,829 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549b1284-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=2fc79030-001e-006b-7e40-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '549b1284-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cG3bPdFznYsiqw3ZBr04eqGT9hWdAeyf33Wdr94jU0k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. storagereplication stdout | 2020-03-31 09:38:08,829 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549b1284-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=2fc79030-001e-006b-7e40-07589b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '549b1284-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cG3bPdFznYsiqw3ZBr04eqGT9hWdAeyf33Wdr94jU0k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. 2020-03-31 09:38:08,830 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:38:08,830 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:38:08,830 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:38:08,830 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:08,830 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:38:08,830 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:38:08,830 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:549cc318-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=gN6uRqzxxdd4iRcizoHfhUkN%2BZT7mMnz7y7VGmvrTL0%3D&sv=2017-07-29&se=2020-03-31T09%3A39%3A08Z x-ms-date:Tue, 31 Mar 2020 09:38:08 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:38:08,830 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549cc318-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=gN6uRqzxxdd4iRcizoHfhUkN%2BZT7mMnz7y7VGmvrTL0%3D&sv=2017-07-29&se=2020-03-31T09%3A39%3A08Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:VYtCTsGpO4EOeo4CkmQXL9DCjc0mzLKrIpbHfAC8k4o=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '549cc318-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. storagereplication stdout | 2020-03-31 09:38:08,830 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:549cc318-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=gN6uRqzxxdd4iRcizoHfhUkN%2BZT7mMnz7y7VGmvrTL0%3D&sv=2017-07-29&se=2020-03-31T09%3A39%3A08Z x-ms-date:Tue, 31 Mar 2020 09:38:08 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:38:08,830 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549cc318-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=gN6uRqzxxdd4iRcizoHfhUkN%2BZT7mMnz7y7VGmvrTL0%3D&sv=2017-07-29&se=2020-03-31T09%3A39%3A08Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:VYtCTsGpO4EOeo4CkmQXL9DCjc0mzLKrIpbHfAC8k4o=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '549cc318-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. 2020-03-31 09:38:08,850 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:38:08,850 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 2020-03-31 09:38:08,851 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549cc318-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=2fc79033-001e-006b-7f40-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=gN6uRqzxxdd4iRcizoHfhUkN%2BZT7mMnz7y7VGmvrTL0%3D&sv=2017-07-29&se=2020-03-31T09%3A39%3A08Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:VYtCTsGpO4EOeo4CkmQXL9DCjc0mzLKrIpbHfAC8k4o=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '549cc318-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. storagereplication stdout | 2020-03-31 09:38:08,851 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=549cc318-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=2fc79033-001e-006b-7f40-07589b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=gN6uRqzxxdd4iRcizoHfhUkN%2BZT7mMnz7y7VGmvrTL0%3D&sv=2017-07-29&se=2020-03-31T09%3A39%3A08Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:VYtCTsGpO4EOeo4CkmQXL9DCjc0mzLKrIpbHfAC8k4o=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '549cc318-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. 2020-03-31 09:38:08,851 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:38:08,851 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:38:08,852 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:54a0055a-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:08 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:38:08,852 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54a0055a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0fcf5g8a2U6oRQuMVdtvONFYfovFqfy4AGSBZrVIDR4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. storagereplication stdout | 2020-03-31 09:38:08,852 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:54a0055a-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:08 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:38:08,852 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54a0055a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0fcf5g8a2U6oRQuMVdtvONFYfovFqfy4AGSBZrVIDR4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. 2020-03-31 09:38:08,937 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:38:08,937 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:38:08,938 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=03eb9b19-f01e-0021-0740-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54a0055a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0fcf5g8a2U6oRQuMVdtvONFYfovFqfy4AGSBZrVIDR4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. 2020-03-31 09:38:08,938 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=03eb9b19-f01e-0021-0740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:08,938 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=03eb9b19-f01e-0021-0740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:08,938 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:38:08,938 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=03eb9b19-f01e-0021-0740-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '54a0055a-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:0fcf5g8a2U6oRQuMVdtvONFYfovFqfy4AGSBZrVIDR4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:08 GMT'}. 2020-03-31 09:38:08,938 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=03eb9b19-f01e-0021-0740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:08,938 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=54a0055a-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:08 GMT, Server-Request-ID=03eb9b19-f01e-0021-0740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:08,938 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:38:10,941 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:55debf7e-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:10 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:38:10,941 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:55debf7e-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:10 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:38:10,941 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '55debf7e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:rmuGjmzAjqtf5dD2QQVXY7RKnX3BfoeNB+IrS30mjVw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:10 GMT'}. storagereplication stdout | 2020-03-31 09:38:10,941 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '55debf7e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:rmuGjmzAjqtf5dD2QQVXY7RKnX3BfoeNB+IrS30mjVw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:10 GMT'}. 2020-03-31 09:38:11,021 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:38:11,021 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:38:11,022 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:10 GMT, Server-Request-ID=03eba4e9-f01e-0021-2440-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '55debf7e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:rmuGjmzAjqtf5dD2QQVXY7RKnX3BfoeNB+IrS30mjVw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:10 GMT'}. 2020-03-31 09:38:11,022 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:10 GMT, Server-Request-ID=03eba4e9-f01e-0021-2440-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:38:11,022 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:10 GMT, Server-Request-ID=03eba4e9-f01e-0021-2440-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '55debf7e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:rmuGjmzAjqtf5dD2QQVXY7RKnX3BfoeNB+IrS30mjVw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:10 GMT'}. 2020-03-31 09:38:11,022 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:10 GMT, Server-Request-ID=03eba4e9-f01e-0021-2440-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:11,022 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:10 GMT, Server-Request-ID=03eba4e9-f01e-0021-2440-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:11,022 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:38:11,022 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=55debf7e-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:10 GMT, Server-Request-ID=03eba4e9-f01e-0021-2440-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:11,022 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:38:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:38:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:21.440707+00:00 (in 9.999223 seconds) 2020-03-31 09:38:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:21 UTC)" (scheduled at 2020-03-31 09:38:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:38:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:21.440707+00:00 (in 9.999223 seconds) 2020-03-31 09:38:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:21 UTC)" (scheduled at 2020-03-31 09:38:11.440707+00:00) 2020-03-31 09:38:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:38:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:38:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:38:11,460 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:38:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:38:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:38:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:21 UTC)" executed successfully 2020-03-31 09:38:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:38:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:24.323352+00:00 (in 9.999218 seconds) 2020-03-31 09:38:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:24 UTC)" (scheduled at 2020-03-31 09:38:14.323352+00:00) 2020-03-31 09:38:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:38:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:24.323352+00:00 (in 9.999218 seconds) 2020-03-31 09:38:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:24 UTC)" (scheduled at 2020-03-31 09:38:14.323352+00:00) 2020-03-31 09:38:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 14, 325123), True, datetime.datetime(2020, 3, 31, 9, 38, 14, 325123), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:38:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 14, 325123), True, datetime.datetime(2020, 3, 31, 9, 38, 14, 325123), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:38:14,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:38:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:38:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:24 UTC)" executed successfully 2020-03-31 09:38:14,446 [527] [DEBUG] [app] Starting request: urn:request:c0f7d708-d5a1-42d5-93ac-d9df3404467b (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:14,446 [527] [DEBUG] [app] Starting request: urn:request:c0f7d708-d5a1-42d5-93ac-d9df3404467b (/health/instance) 2020-03-31 09:38:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:14,454 [556] [DEBUG] [app] Starting request: urn:request:e3706691-3fe8-4520-acc9-84b287c18f9e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:14,454 [556] [DEBUG] [app] Starting request: urn:request:e3706691-3fe8-4520-acc9-84b287c18f9e (/v1/_internal_ping) 2020-03-31 09:38:14,456 [556] [DEBUG] [app] Ending request: urn:request:e3706691-3fe8-4520-acc9-84b287c18f9e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:14,456 [556] [DEBUG] [app] Ending request: urn:request:e3706691-3fe8-4520-acc9-84b287c18f9e (/v1/_internal_ping) 2020-03-31 09:38:14,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:14,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:38:14,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:14,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:14,457 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:38:14,458 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:14,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:14,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:14,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:14,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:14,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:14,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:14,470 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:14,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:14,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:14,489 [518] [DEBUG] [app] Starting request: urn:request:0ebdfefb-10bb-43d0-86a8-bbd14a0b4b43 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:14,489 [518] [DEBUG] [app] Starting request: urn:request:0ebdfefb-10bb-43d0-86a8-bbd14a0b4b43 (/_internal_ping) 2020-03-31 09:38:14,490 [518] [DEBUG] [app] Ending request: urn:request:0ebdfefb-10bb-43d0-86a8-bbd14a0b4b43 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:14,490 [518] [DEBUG] [app] Ending request: urn:request:0ebdfefb-10bb-43d0-86a8-bbd14a0b4b43 (/_internal_ping) 2020-03-31 09:38:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:14,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:38:14,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:14,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:38:14,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:14,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:14,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:14,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:14,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:38:14,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:14,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:14,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:14,504 [541] [DEBUG] [app] Starting request: urn:request:f32b830e-87a4-44fb-af88-8fa038d1a862 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:14,504 [541] [DEBUG] [app] Starting request: urn:request:f32b830e-87a4-44fb-af88-8fa038d1a862 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:14,505 [541] [DEBUG] [app] Ending request: urn:request:f32b830e-87a4-44fb-af88-8fa038d1a862 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:14,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:38:14,505 [541] [DEBUG] [app] Ending request: urn:request:f32b830e-87a4-44fb-af88-8fa038d1a862 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:14,507 [527] [DEBUG] [app] Ending request: urn:request:c0f7d708-d5a1-42d5-93ac-d9df3404467b (/health/instance) 2020-03-31 09:38:14,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:38:14,505 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:14,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:38:14,507 [527] [DEBUG] [app] Ending request: urn:request:c0f7d708-d5a1-42d5-93ac-d9df3404467b (/health/instance) 2020-03-31 09:38:14,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:14,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:38:15,025 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:584df1c6-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:15 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:38:15,025 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:584df1c6-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:15 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:38:15,025 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '584df1c6-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:69/SXbV+tVzHxwoLwe8pnp6XT8P67So0BOJ3nYfmUAo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:15 GMT'}. storagereplication stdout | 2020-03-31 09:38:15,025 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '584df1c6-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:69/SXbV+tVzHxwoLwe8pnp6XT8P67So0BOJ3nYfmUAo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:15 GMT'}. 2020-03-31 09:38:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:38:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:45.064514+00:00 (in 29.999112 seconds) 2020-03-31 09:38:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:45 UTC)" (scheduled at 2020-03-31 09:38:15.064514+00:00) gcworker stdout | 2020-03-31 09:38:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:45.064514+00:00 (in 29.999112 seconds) 2020-03-31 09:38:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:45 UTC)" (scheduled at 2020-03-31 09:38:15.064514+00:00) 2020-03-31 09:38:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437895, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:38:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437895, 1209600, 500, 1, 0]) 2020-03-31 09:38:15,090 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:38:15,090 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:38:15,090 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:15,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:38:15,090 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:15,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:45 UTC)" executed successfully 2020-03-31 09:38:15,106 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:38:15,106 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:38:15,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:14 GMT, Server-Request-ID=03ebb8da-f01e-0021-2740-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '584df1c6-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:69/SXbV+tVzHxwoLwe8pnp6XT8P67So0BOJ3nYfmUAo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:15 GMT'}. 2020-03-31 09:38:15,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:14 GMT, Server-Request-ID=03ebb8da-f01e-0021-2740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:15,107 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:14 GMT, Server-Request-ID=03ebb8da-f01e-0021-2740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:15,108 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:38:15,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:14 GMT, Server-Request-ID=03ebb8da-f01e-0021-2740-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '584df1c6-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:69/SXbV+tVzHxwoLwe8pnp6XT8P67So0BOJ3nYfmUAo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:15 GMT'}. 2020-03-31 09:38:15,107 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:14 GMT, Server-Request-ID=03ebb8da-f01e-0021-2740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:15,107 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=584df1c6-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:14 GMT, Server-Request-ID=03ebb8da-f01e-0021-2740-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:15,108 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:38:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:38:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:26.924497+00:00 (in 9.999257 seconds) storagereplication stdout | 2020-03-31 09:38:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:26.924497+00:00 (in 9.999257 seconds) 2020-03-31 09:38:18,273 [518] [DEBUG] [app] Starting request: urn:request:92f208e0-dcdc-431f-9545-5174453f0ef1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:18,273 [518] [DEBUG] [app] Starting request: urn:request:92f208e0-dcdc-431f-9545-5174453f0ef1 (/health/instance) 2020-03-31 09:38:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:18,281 [545] [DEBUG] [app] Starting request: urn:request:70fade18-0adc-4a04-b792-05c06b76370b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:18,281 [545] [DEBUG] [app] Starting request: urn:request:70fade18-0adc-4a04-b792-05c06b76370b (/v1/_internal_ping) 2020-03-31 09:38:18,282 [545] [DEBUG] [app] Ending request: urn:request:70fade18-0adc-4a04-b792-05c06b76370b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:18,282 [545] [DEBUG] [app] Ending request: urn:request:70fade18-0adc-4a04-b792-05c06b76370b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:18,283 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:38:18,283 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:18,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:18,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:18,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:18,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:18,288 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:18,288 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:18,313 [518] [DEBUG] [app] Starting request: urn:request:c7b3c1e7-7414-4a6e-aa25-8dd7b2f54cb9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:18,313 [518] [DEBUG] [app] Starting request: urn:request:c7b3c1e7-7414-4a6e-aa25-8dd7b2f54cb9 (/_internal_ping) 2020-03-31 09:38:18,313 [518] [DEBUG] [app] Ending request: urn:request:c7b3c1e7-7414-4a6e-aa25-8dd7b2f54cb9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:18,313 [518] [DEBUG] [app] Ending request: urn:request:c7b3c1e7-7414-4a6e-aa25-8dd7b2f54cb9 (/_internal_ping) 2020-03-31 09:38:18,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:18,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:38:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:18,326 [541] [DEBUG] [app] Starting request: urn:request:c2cba0cf-d6ce-4fe9-af45-2135d9c21e3e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:18,326 [541] [DEBUG] [app] Starting request: urn:request:c2cba0cf-d6ce-4fe9-af45-2135d9c21e3e (/c1/_internal_ping) 2020-03-31 09:38:18,327 [541] [DEBUG] [app] Ending request: urn:request:c2cba0cf-d6ce-4fe9-af45-2135d9c21e3e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:18,327 [541] [DEBUG] [app] Ending request: urn:request:c2cba0cf-d6ce-4fe9-af45-2135d9c21e3e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:18,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:38:18,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:18,329 [518] [DEBUG] [app] Ending request: urn:request:92f208e0-dcdc-431f-9545-5174453f0ef1 (/health/instance) 2020-03-31 09:38:18,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:18,329 [518] [DEBUG] [app] Ending request: urn:request:92f208e0-dcdc-431f-9545-5174453f0ef1 (/health/instance) 2020-03-31 09:38:18,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:18,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:38:18,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:38:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:38:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:31.440707+00:00 (in 9.999266 seconds) 2020-03-31 09:38:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:31 UTC)" (scheduled at 2020-03-31 09:38:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:38:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:31.440707+00:00 (in 9.999266 seconds) 2020-03-31 09:38:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:31 UTC)" (scheduled at 2020-03-31 09:38:21.440707+00:00) 2020-03-31 09:38:21,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:38:21,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:38:21,470 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:38:21,470 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:38:21,470 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:21,470 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:38:21,470 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:21,470 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:31 UTC)" executed successfully 2020-03-31 09:38:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:38:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:52.113221+00:00 (in 29.997679 seconds) 2020-03-31 09:38:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:52 UTC)" (scheduled at 2020-03-31 09:38:22.113221+00:00) securityworker stdout | 2020-03-31 09:38:22,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:52.113221+00:00 (in 29.997679 seconds) 2020-03-31 09:38:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:52 UTC)" (scheduled at 2020-03-31 09:38:22.113221+00:00) 2020-03-31 09:38:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:38:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:38:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:38:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:38:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:38:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:38:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:38:22,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:22,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:38:52 UTC)" executed successfully 2020-03-31 09:38:23,116 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:5d209438-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:23 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:38:23,116 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:5d209438-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:38:23 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:38:23,116 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5d209438-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RYYjRY4AOpZdY1QJyFXIffL37hFs7PAC1khpXLQgP4w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:23 GMT'}. storagereplication stdout | 2020-03-31 09:38:23,116 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5d209438-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RYYjRY4AOpZdY1QJyFXIffL37hFs7PAC1khpXLQgP4w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:23 GMT'}. 2020-03-31 09:38:23,197 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:38:23,197 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:38:23,197 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:22 GMT, Server-Request-ID=03ebdd10-f01e-0021-5640-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5d209438-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RYYjRY4AOpZdY1QJyFXIffL37hFs7PAC1khpXLQgP4w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:23 GMT'}. 2020-03-31 09:38:23,198 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:22 GMT, Server-Request-ID=03ebdd10-f01e-0021-5640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:23,198 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:22 GMT, Server-Request-ID=03ebdd10-f01e-0021-5640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:23,198 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:38:23,197 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:38:22 GMT, Server-Request-ID=03ebdd10-f01e-0021-5640-07c574000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '5d209438-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RYYjRY4AOpZdY1QJyFXIffL37hFs7PAC1khpXLQgP4w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:38:23 GMT'}. 2020-03-31 09:38:23,198 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:38:22 GMT, Server-Request-ID=03ebdd10-f01e-0021-5640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:23,198 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=5d209438-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:38:22 GMT, Server-Request-ID=03ebdd10-f01e-0021-5640-07c574000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:38:23,198 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:38:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:38:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:34 UTC)" (scheduled at 2020-03-31 09:38:24.323352+00:00) 2020-03-31 09:38:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:38:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:34 UTC)" (scheduled at 2020-03-31 09:38:24.323352+00:00) 2020-03-31 09:38:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 24, 325044), True, datetime.datetime(2020, 3, 31, 9, 38, 24, 325044), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:38:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 24, 325044), True, datetime.datetime(2020, 3, 31, 9, 38, 24, 325044), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:38:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:34.323352+00:00 (in 9.999104 seconds) notificationworker stdout | 2020-03-31 09:38:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:34.323352+00:00 (in 9.999104 seconds) 2020-03-31 09:38:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:38:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:38:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:34 UTC)" executed successfully 2020-03-31 09:38:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:38:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:33.330112+00:00 (in 8.918121 seconds) 2020-03-31 09:38:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:24 UTC)" (scheduled at 2020-03-31 09:38:24.410927+00:00) security_notification_worker stdout | 2020-03-31 09:38:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:33.330112+00:00 (in 8.918121 seconds) 2020-03-31 09:38:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:24 UTC)" (scheduled at 2020-03-31 09:38:24.410927+00:00) 2020-03-31 09:38:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:38:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:24 UTC)" executed successfully 2020-03-31 09:38:24,446 [527] [DEBUG] [app] Starting request: urn:request:0fa2582f-c992-4269-b806-b299f7498404 (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:24,446 [527] [DEBUG] [app] Starting request: urn:request:0fa2582f-c992-4269-b806-b299f7498404 (/health/instance) 2020-03-31 09:38:24,457 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:24,457 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:24,466 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:24,466 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:24,468 [538] [DEBUG] [app] Starting request: urn:request:2596b835-a07e-4564-af64-831db47a991d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:24,468 [538] [DEBUG] [app] Starting request: urn:request:2596b835-a07e-4564-af64-831db47a991d (/v1/_internal_ping) 2020-03-31 09:38:24,469 [538] [DEBUG] [app] Ending request: urn:request:2596b835-a07e-4564-af64-831db47a991d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:24,469 [538] [DEBUG] [app] Ending request: urn:request:2596b835-a07e-4564-af64-831db47a991d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:24,469 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:24,469 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:38:24,469 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:24,469 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:24,471 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:24,471 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:24,473 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:24,473 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:24,474 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:24,474 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:24,475 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:24,475 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:24,482 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:24,482 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:24,497 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:24,497 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:24,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:24,503 [525] [DEBUG] [app] Starting request: urn:request:69366d4c-95f2-4c72-a500-55fa01ea7545 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:24,503 [525] [DEBUG] [app] Starting request: urn:request:69366d4c-95f2-4c72-a500-55fa01ea7545 (/_internal_ping) 2020-03-31 09:38:24,504 [525] [DEBUG] [app] Ending request: urn:request:69366d4c-95f2-4c72-a500-55fa01ea7545 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:24,504 [525] [DEBUG] [app] Ending request: urn:request:69366d4c-95f2-4c72-a500-55fa01ea7545 (/_internal_ping) 2020-03-31 09:38:24,505 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:24,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:38:24,505 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:24,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:38:24,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:24,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:24,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:24,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:24,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:24,512 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:24,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:38:24,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:24,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:24,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:24,517 [541] [DEBUG] [app] Starting request: urn:request:9cc80f34-b082-4b40-8d5b-a319dfd96f20 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:24,517 [541] [DEBUG] [app] Starting request: urn:request:9cc80f34-b082-4b40-8d5b-a319dfd96f20 (/c1/_internal_ping) 2020-03-31 09:38:24,518 [541] [DEBUG] [app] Ending request: urn:request:9cc80f34-b082-4b40-8d5b-a319dfd96f20 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:24,518 [541] [DEBUG] [app] Ending request: urn:request:9cc80f34-b082-4b40-8d5b-a319dfd96f20 (/c1/_internal_ping) 2020-03-31 09:38:24,519 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:24,519 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:24,519 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:24,520 [527] [DEBUG] [app] Ending request: urn:request:0fa2582f-c992-4269-b806-b299f7498404 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:38:24,519 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:24,520 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) gunicorn-web stdout | 2020-03-31 09:38:24,520 [527] [DEBUG] [app] Ending request: urn:request:0fa2582f-c992-4269-b806-b299f7498404 (/health/instance) 2020-03-31 09:38:24,521 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:38:24,520 [527] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.077 49 0.077) gunicorn-web stdout | 2020-03-31 09:38:24,521 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:38:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:38:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:36.924497+00:00 (in 9.999199 seconds) storagereplication stdout | 2020-03-31 09:38:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:36.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:38:28,273 [518] [DEBUG] [app] Starting request: urn:request:2c1db619-4f11-4ed7-979c-e995f0847d6a (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:28,273 [518] [DEBUG] [app] Starting request: urn:request:2c1db619-4f11-4ed7-979c-e995f0847d6a (/health/instance) 2020-03-31 09:38:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:28,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:28,280 [551] [DEBUG] [app] Starting request: urn:request:ff15ef79-c88a-4814-9a5e-bc19ab27afdb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:28,280 [551] [DEBUG] [app] Starting request: urn:request:ff15ef79-c88a-4814-9a5e-bc19ab27afdb (/v1/_internal_ping) 2020-03-31 09:38:28,281 [551] [DEBUG] [app] Ending request: urn:request:ff15ef79-c88a-4814-9a5e-bc19ab27afdb (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:28,281 [551] [DEBUG] [app] Ending request: urn:request:ff15ef79-c88a-4814-9a5e-bc19ab27afdb (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:28,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:28,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:28,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:38:28,283 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:28,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:28,285 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:28,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:28,286 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:28,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:28,287 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:28,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:28,302 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:28,316 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:28,316 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:28,321 [518] [DEBUG] [app] Starting request: urn:request:a6085b72-f3ed-4835-9c65-b273e0a4ca1a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:28,321 [518] [DEBUG] [app] Starting request: urn:request:a6085b72-f3ed-4835-9c65-b273e0a4ca1a (/_internal_ping) 2020-03-31 09:38:28,322 [518] [DEBUG] [app] Ending request: urn:request:a6085b72-f3ed-4835-9c65-b273e0a4ca1a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:28,322 [518] [DEBUG] [app] Ending request: urn:request:a6085b72-f3ed-4835-9c65-b273e0a4ca1a (/_internal_ping) 2020-03-31 09:38:28,322 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:28,322 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:28,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:38:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:28,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:28,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:28,335 [537] [DEBUG] [app] Starting request: urn:request:c21342ee-d22b-463f-b4a8-dbf9186657c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:28,335 [537] [DEBUG] [app] Starting request: urn:request:c21342ee-d22b-463f-b4a8-dbf9186657c7 (/c1/_internal_ping) 2020-03-31 09:38:28,336 [537] [DEBUG] [app] Ending request: urn:request:c21342ee-d22b-463f-b4a8-dbf9186657c7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:28,336 [537] [DEBUG] [app] Ending request: urn:request:c21342ee-d22b-463f-b4a8-dbf9186657c7 (/c1/_internal_ping) 2020-03-31 09:38:28,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:28,336 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:28,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:38:28,336 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:28,338 [518] [DEBUG] [app] Ending request: urn:request:2c1db619-4f11-4ed7-979c-e995f0847d6a (/health/instance) 2020-03-31 09:38:28,338 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:28,338 [518] [DEBUG] [app] Ending request: urn:request:2c1db619-4f11-4ed7-979c-e995f0847d6a (/health/instance) 2020-03-31 09:38:28,338 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:28,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:38:28,339 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:38:30,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:38:30,339 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:00.337699+00:00 (in 29.998396 seconds) 2020-03-31 09:38:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:00 UTC)" (scheduled at 2020-03-31 09:38:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:38:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:00.337699+00:00 (in 29.998396 seconds) 2020-03-31 09:38:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:00 UTC)" (scheduled at 2020-03-31 09:38:30.337699+00:00) 2020-03-31 09:38:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 38, 30, 339881), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:38:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 38, 30, 339881), False, 50, 1, 0]) 2020-03-31 09:38:30,566 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:38:30,566 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:38:30,566 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:38:30,566 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:30,567 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:38:30,567 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:00 UTC)" executed successfully 2020-03-31 09:38:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:38:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:41.440707+00:00 (in 9.998847 seconds) 2020-03-31 09:38:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:41 UTC)" (scheduled at 2020-03-31 09:38:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:38:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:41.440707+00:00 (in 9.998847 seconds) 2020-03-31 09:38:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:41 UTC)" (scheduled at 2020-03-31 09:38:31.440707+00:00) 2020-03-31 09:38:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:38:31,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:38:31,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:38:31,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:38:31,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:38:31,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:31,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:41 UTC)" executed successfully 2020-03-31 09:38:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:38:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:03.330112+00:00 (in 29.998786 seconds) 2020-03-31 09:38:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:03 UTC)" (scheduled at 2020-03-31 09:38:33.330112+00:00) 2020-03-31 09:38:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:38:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:03.330112+00:00 (in 29.998786 seconds) 2020-03-31 09:38:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:03 UTC)" (scheduled at 2020-03-31 09:38:33.330112+00:00) 2020-03-31 09:38:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 33, 332265), True, datetime.datetime(2020, 3, 31, 9, 38, 33, 332265), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:38:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 33, 332265), True, datetime.datetime(2020, 3, 31, 9, 38, 33, 332265), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:38:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:38:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:38:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:03 UTC)" executed successfully 2020-03-31 09:38:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:38:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:43.539999+00:00 (in 10.172430 seconds) 2020-03-31 09:38:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:33 UTC)" (scheduled at 2020-03-31 09:38:33.366791+00:00) 2020-03-31 09:38:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:38:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:43.539999+00:00 (in 10.172430 seconds) 2020-03-31 09:38:33,367 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:33 UTC)" (scheduled at 2020-03-31 09:38:33.366791+00:00) 2020-03-31 09:38:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 33, 368114), True, datetime.datetime(2020, 3, 31, 9, 38, 33, 368114), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:38:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 33, 368114), True, datetime.datetime(2020, 3, 31, 9, 38, 33, 368114), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:38:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:38:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:38:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:33 UTC)" executed successfully 2020-03-31 09:38:34,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:38:34,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:34,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:44.323352+00:00 (in 9.995221 seconds) 2020-03-31 09:38:34,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:44 UTC)" (scheduled at 2020-03-31 09:38:34.323352+00:00) notificationworker stdout | 2020-03-31 09:38:34,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:44.323352+00:00 (in 9.995221 seconds) 2020-03-31 09:38:34,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:44 UTC)" (scheduled at 2020-03-31 09:38:34.323352+00:00) 2020-03-31 09:38:34,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:38:34,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:34,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 34, 329178), True, datetime.datetime(2020, 3, 31, 9, 38, 34, 329178), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:38:34,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 34, 329178), True, datetime.datetime(2020, 3, 31, 9, 38, 34, 329178), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:38:34,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:34,346 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:38:34,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:34,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:38:34,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:44 UTC)" executed successfully 2020-03-31 09:38:34,446 [518] [DEBUG] [app] Starting request: urn:request:b9fdc47c-341a-4cca-9811-f78e17b8ec39 (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:34,446 [518] [DEBUG] [app] Starting request: urn:request:b9fdc47c-341a-4cca-9811-f78e17b8ec39 (/health/instance) 2020-03-31 09:38:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:34,453 [544] [DEBUG] [app] Starting request: urn:request:b3c9abc7-962c-477f-b1e7-671269c9aa89 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:34,453 [544] [DEBUG] [app] Starting request: urn:request:b3c9abc7-962c-477f-b1e7-671269c9aa89 (/v1/_internal_ping) 2020-03-31 09:38:34,454 [544] [DEBUG] [app] Ending request: urn:request:b3c9abc7-962c-477f-b1e7-671269c9aa89 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:34,454 [544] [DEBUG] [app] Ending request: urn:request:b3c9abc7-962c-477f-b1e7-671269c9aa89 (/v1/_internal_ping) 2020-03-31 09:38:34,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:38:34,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:34,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:38:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:34,457 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:34,457 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:34,458 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:34,458 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:34,459 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:34,459 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:34,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:34,484 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:34,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:34,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:34,489 [527] [DEBUG] [app] Starting request: urn:request:3fb72f46-ef8b-45f2-beb6-0f41509cd427 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:34,489 [527] [DEBUG] [app] Starting request: urn:request:3fb72f46-ef8b-45f2-beb6-0f41509cd427 (/_internal_ping) 2020-03-31 09:38:34,490 [527] [DEBUG] [app] Ending request: urn:request:3fb72f46-ef8b-45f2-beb6-0f41509cd427 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:34,490 [527] [DEBUG] [app] Ending request: urn:request:3fb72f46-ef8b-45f2-beb6-0f41509cd427 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:38:34,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:34,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:34,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:34,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:34,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:34,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:38:34,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:34,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:34,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:34,503 [537] [DEBUG] [app] Starting request: urn:request:a0641aa3-0cb2-49fb-a0ec-16ea7c2be6cb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:34,503 [537] [DEBUG] [app] Starting request: urn:request:a0641aa3-0cb2-49fb-a0ec-16ea7c2be6cb (/c1/_internal_ping) 2020-03-31 09:38:34,504 [537] [DEBUG] [app] Ending request: urn:request:a0641aa3-0cb2-49fb-a0ec-16ea7c2be6cb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:34,504 [537] [DEBUG] [app] Ending request: urn:request:a0641aa3-0cb2-49fb-a0ec-16ea7c2be6cb (/c1/_internal_ping) 2020-03-31 09:38:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:34,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:34,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:38:34,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:34,506 [518] [DEBUG] [app] Ending request: urn:request:b9fdc47c-341a-4cca-9811-f78e17b8ec39 (/health/instance) 2020-03-31 09:38:34,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:34,506 [518] [DEBUG] [app] Ending request: urn:request:b9fdc47c-341a-4cca-9811-f78e17b8ec39 (/health/instance) 2020-03-31 09:38:34,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:38:34,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:38:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:38:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:38.010934+00:00 (in 1.085463 seconds) storagereplication stdout | 2020-03-31 09:38:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:38:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:38.010934+00:00 (in 1.085463 seconds) 2020-03-31 09:38:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:38:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:46.924497+00:00 (in 8.912807 seconds) 2020-03-31 09:38:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:38 UTC)" (scheduled at 2020-03-31 09:38:38.010934+00:00) 2020-03-31 09:38:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:38:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:46.924497+00:00 (in 8.912807 seconds) 2020-03-31 09:38:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:38 UTC)" (scheduled at 2020-03-31 09:38:38.010934+00:00) 2020-03-31 09:38:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:38 UTC)" executed successfully 2020-03-31 09:38:38,272 [527] [DEBUG] [app] Starting request: urn:request:e6599c04-fe69-4d86-9112-79e37ccedc2a (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:38,272 [527] [DEBUG] [app] Starting request: urn:request:e6599c04-fe69-4d86-9112-79e37ccedc2a (/health/instance) 2020-03-31 09:38:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:38,280 [550] [DEBUG] [app] Starting request: urn:request:20c784ca-c959-4696-9fae-74f053b9f670 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:38,280 [550] [DEBUG] [app] Starting request: urn:request:20c784ca-c959-4696-9fae-74f053b9f670 (/v1/_internal_ping) 2020-03-31 09:38:38,281 [550] [DEBUG] [app] Ending request: urn:request:20c784ca-c959-4696-9fae-74f053b9f670 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:38,281 [550] [DEBUG] [app] Ending request: urn:request:20c784ca-c959-4696-9fae-74f053b9f670 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:38,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:38:38,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:38,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:38:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:38:38,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:38,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:38,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:38,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:38,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:38,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:38,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:38,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:38,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:38,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:38,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:38,318 [527] [DEBUG] [app] Starting request: urn:request:e8ba3625-e246-40d1-bce5-e4d92411cf0b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:38,318 [527] [DEBUG] [app] Starting request: urn:request:e8ba3625-e246-40d1-bce5-e4d92411cf0b (/_internal_ping) 2020-03-31 09:38:38,319 [527] [DEBUG] [app] Ending request: urn:request:e8ba3625-e246-40d1-bce5-e4d92411cf0b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:38,319 [527] [DEBUG] [app] Ending request: urn:request:e8ba3625-e246-40d1-bce5-e4d92411cf0b (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:38:38,320 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:38,320 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:38:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:38:38,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:38,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:38:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:38,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:38,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:38,333 [541] [DEBUG] [app] Starting request: urn:request:1526e43c-62e2-4c85-bed3-310be61dd3a8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:38,333 [541] [DEBUG] [app] Starting request: urn:request:1526e43c-62e2-4c85-bed3-310be61dd3a8 (/c1/_internal_ping) 2020-03-31 09:38:38,334 [541] [DEBUG] [app] Ending request: urn:request:1526e43c-62e2-4c85-bed3-310be61dd3a8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:38,334 [541] [DEBUG] [app] Ending request: urn:request:1526e43c-62e2-4c85-bed3-310be61dd3a8 (/c1/_internal_ping) 2020-03-31 09:38:38,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:38,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:38,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:38:38,335 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:38,336 [527] [DEBUG] [app] Ending request: urn:request:e6599c04-fe69-4d86-9112-79e37ccedc2a (/health/instance) 2020-03-31 09:38:38,337 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:38,336 [527] [DEBUG] [app] Ending request: urn:request:e6599c04-fe69-4d86-9112-79e37ccedc2a (/health/instance) 2020-03-31 09:38:38,337 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:38,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:38:38,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:38:39,200 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:38:39,200 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy 2020-03-31 09:38:39,200 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:38:39,200 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:38:39,200 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:38:39,200 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:38:39,212 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) storagereplication stdout | 2020-03-31 09:38:39,212 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) 2020-03-31 09:38:39,218 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 43, 39, 211823), True, u'45d05454-9350-48bb-9e2a-fead2a46ef3d', 1]) storagereplication stdout | 2020-03-31 09:38:39,218 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 43, 39, 211823), True, u'45d05454-9350-48bb-9e2a-fead2a46ef3d', 1]) 2020-03-31 09:38:39,224 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 39, 222901), True, datetime.datetime(2020, 3, 31, 9, 38, 39, 222901), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:38:39,224 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 39, 222901), True, datetime.datetime(2020, 3, 31, 9, 38, 39, 222901), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:38:39,229 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:38:39,229 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:39,229 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:39,229 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:38:39,229 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:39,229 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:46 UTC)" executed successfully 2020-03-31 09:38:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:38:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:51 UTC)" (scheduled at 2020-03-31 09:38:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:38:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:51 UTC)" (scheduled at 2020-03-31 09:38:41.440707+00:00) 2020-03-31 09:38:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:38:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:38:41,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:51.440707+00:00 (in 9.999124 seconds) repositoryactioncounter stdout | 2020-03-31 09:38:41,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:51.440707+00:00 (in 9.999124 seconds) 2020-03-31 09:38:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:38:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:38:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:38:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:51 UTC)" executed successfully 2020-03-31 09:38:43,542 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:38:43,542 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:43,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:33.366791+00:00 (in 49.824415 seconds) 2020-03-31 09:38:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:43 UTC)" (scheduled at 2020-03-31 09:38:43.539999+00:00) 2020-03-31 09:38:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:38:43,542 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:33.366791+00:00 (in 49.824415 seconds) 2020-03-31 09:38:43,542 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:43 UTC)" (scheduled at 2020-03-31 09:38:43.539999+00:00) 2020-03-31 09:38:43,542 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:43,542 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:43 UTC)" executed successfully 2020-03-31 09:38:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:38:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:49.498966+00:00 (in 5.174524 seconds) 2020-03-31 09:38:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:54 UTC)" (scheduled at 2020-03-31 09:38:44.323352+00:00) 2020-03-31 09:38:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:38:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:49.498966+00:00 (in 5.174524 seconds) 2020-03-31 09:38:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:54 UTC)" (scheduled at 2020-03-31 09:38:44.323352+00:00) 2020-03-31 09:38:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 44, 325307), True, datetime.datetime(2020, 3, 31, 9, 38, 44, 325307), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:38:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 44, 325307), True, datetime.datetime(2020, 3, 31, 9, 38, 44, 325307), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:38:44,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:38:44,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:38:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:54 UTC)" executed successfully 2020-03-31 09:38:44,447 [518] [DEBUG] [app] Starting request: urn:request:bd36c75c-a319-478e-9b2d-704891911ab6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:44,447 [518] [DEBUG] [app] Starting request: urn:request:bd36c75c-a319-478e-9b2d-704891911ab6 (/health/instance) 2020-03-31 09:38:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:44,454 [538] [DEBUG] [app] Starting request: urn:request:7f7a9c75-3fd4-4870-94f0-e30ee7af6f66 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:44,454 [538] [DEBUG] [app] Starting request: urn:request:7f7a9c75-3fd4-4870-94f0-e30ee7af6f66 (/v1/_internal_ping) 2020-03-31 09:38:44,455 [538] [DEBUG] [app] Ending request: urn:request:7f7a9c75-3fd4-4870-94f0-e30ee7af6f66 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:44,455 [538] [DEBUG] [app] Ending request: urn:request:7f7a9c75-3fd4-4870-94f0-e30ee7af6f66 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:44,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:38:44,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:44,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:44,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:44,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:44,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:44,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:44,487 [518] [DEBUG] [app] Starting request: urn:request:2aa6efa5-81c1-41aa-b84d-ffce48f284a2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:44,487 [518] [DEBUG] [app] Starting request: urn:request:2aa6efa5-81c1-41aa-b84d-ffce48f284a2 (/_internal_ping) 2020-03-31 09:38:44,487 [518] [DEBUG] [app] Ending request: urn:request:2aa6efa5-81c1-41aa-b84d-ffce48f284a2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:44,487 [518] [DEBUG] [app] Ending request: urn:request:2aa6efa5-81c1-41aa-b84d-ffce48f284a2 (/_internal_ping) 2020-03-31 09:38:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:38:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:38:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:38:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:38:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:44,500 [541] [DEBUG] [app] Starting request: urn:request:a80503d9-48fc-4264-b866-d15e45ca9221 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:44,500 [541] [DEBUG] [app] Starting request: urn:request:a80503d9-48fc-4264-b866-d15e45ca9221 (/c1/_internal_ping) 2020-03-31 09:38:44,501 [541] [DEBUG] [app] Ending request: urn:request:a80503d9-48fc-4264-b866-d15e45ca9221 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:44,501 [541] [DEBUG] [app] Ending request: urn:request:a80503d9-48fc-4264-b866-d15e45ca9221 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:44,503 [518] [DEBUG] [app] Ending request: urn:request:bd36c75c-a319-478e-9b2d-704891911ab6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:44,503 [518] [DEBUG] [app] Ending request: urn:request:bd36c75c-a319-478e-9b2d-704891911ab6 (/health/instance) 2020-03-31 09:38:44,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:38:44,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:44,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-verbs stdout | 2020-03-31 09:38:44,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:38:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:15.064514+00:00 (in 29.999096 seconds) 2020-03-31 09:38:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:15 UTC)" (scheduled at 2020-03-31 09:38:45.064514+00:00) gcworker stdout | 2020-03-31 09:38:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:15.064514+00:00 (in 29.999096 seconds) 2020-03-31 09:38:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:15 UTC)" (scheduled at 2020-03-31 09:38:45.064514+00:00) 2020-03-31 09:38:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437925, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:38:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437925, 1209600, 500, 1, 0]) 2020-03-31 09:38:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:38:45,083 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:38:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:38:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:38:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:15 UTC)" executed successfully 2020-03-31 09:38:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:38:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:56.924497+00:00 (in 9.999184 seconds) 2020-03-31 09:38:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:56 UTC)" (scheduled at 2020-03-31 09:38:46.924497+00:00) 2020-03-31 09:38:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:38:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:56.924497+00:00 (in 9.999184 seconds) 2020-03-31 09:38:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:56 UTC)" (scheduled at 2020-03-31 09:38:46.924497+00:00) 2020-03-31 09:38:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 46, 926008), True, datetime.datetime(2020, 3, 31, 9, 38, 46, 926008), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:38:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 46, 926008), True, datetime.datetime(2020, 3, 31, 9, 38, 46, 926008), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:38:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:38:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:38:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:38:56 UTC)" executed successfully 2020-03-31 09:38:48,272 [527] [DEBUG] [app] Starting request: urn:request:e0d56dad-2e8a-4a75-a57f-adc64934af0f (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:48,272 [527] [DEBUG] [app] Starting request: urn:request:e0d56dad-2e8a-4a75-a57f-adc64934af0f (/health/instance) 2020-03-31 09:38:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:48,280 [538] [DEBUG] [app] Starting request: urn:request:2319a53f-4e45-4ef1-9ae2-07ac30a4619e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:48,280 [538] [DEBUG] [app] Starting request: urn:request:2319a53f-4e45-4ef1-9ae2-07ac30a4619e (/v1/_internal_ping) 2020-03-31 09:38:48,281 [538] [DEBUG] [app] Ending request: urn:request:2319a53f-4e45-4ef1-9ae2-07ac30a4619e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:48,281 [538] [DEBUG] [app] Ending request: urn:request:2319a53f-4e45-4ef1-9ae2-07ac30a4619e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:38:48,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:48,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:48,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:48,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:48,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:48,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:48,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:48,311 [527] [DEBUG] [app] Starting request: urn:request:2872e76f-5a81-4e64-90b5-c70b0906cfe3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:48,311 [527] [DEBUG] [app] Starting request: urn:request:2872e76f-5a81-4e64-90b5-c70b0906cfe3 (/_internal_ping) 2020-03-31 09:38:48,312 [527] [DEBUG] [app] Ending request: urn:request:2872e76f-5a81-4e64-90b5-c70b0906cfe3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:48,312 [527] [DEBUG] [app] Ending request: urn:request:2872e76f-5a81-4e64-90b5-c70b0906cfe3 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:38:48,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:48,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:48,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:48,316 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:38:48,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:38:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:38:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:38:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:48,325 [541] [DEBUG] [app] Starting request: urn:request:1ee73656-94fe-473f-9590-3953735df3d8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:48,325 [541] [DEBUG] [app] Starting request: urn:request:1ee73656-94fe-473f-9590-3953735df3d8 (/c1/_internal_ping) 2020-03-31 09:38:48,326 [541] [DEBUG] [app] Ending request: urn:request:1ee73656-94fe-473f-9590-3953735df3d8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:48,326 [541] [DEBUG] [app] Ending request: urn:request:1ee73656-94fe-473f-9590-3953735df3d8 (/c1/_internal_ping) 2020-03-31 09:38:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:48,327 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:38:48,327 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:48,328 [527] [DEBUG] [app] Ending request: urn:request:e0d56dad-2e8a-4a75-a57f-adc64934af0f (/health/instance) 2020-03-31 09:38:48,329 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:48,328 [527] [DEBUG] [app] Ending request: urn:request:e0d56dad-2e8a-4a75-a57f-adc64934af0f (/health/instance) 2020-03-31 09:38:48,329 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:48,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:38:48,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:38:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:38:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:54.323352+00:00 (in 4.823543 seconds) 2020-03-31 09:38:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:49 UTC)" (scheduled at 2020-03-31 09:38:49.498966+00:00) notificationworker stdout | 2020-03-31 09:38:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:54.323352+00:00 (in 4.823543 seconds) 2020-03-31 09:38:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:49 UTC)" (scheduled at 2020-03-31 09:38:49.498966+00:00) 2020-03-31 09:38:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:38:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:49 UTC)" executed successfully 2020-03-31 09:38:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:38:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:59.560326+00:00 (in 8.447755 seconds) 2020-03-31 09:38:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:51 UTC)" (scheduled at 2020-03-31 09:38:51.110610+00:00) 2020-03-31 09:38:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:38:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:38:59.560326+00:00 (in 8.447755 seconds) 2020-03-31 09:38:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:51 UTC)" (scheduled at 2020-03-31 09:38:51.110610+00:00) 2020-03-31 09:38:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 51, 113415), True, datetime.datetime(2020, 3, 31, 9, 38, 51, 113415), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:38:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 51, 113415), True, datetime.datetime(2020, 3, 31, 9, 38, 51, 113415), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:38:51,130 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:38:51,130 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:38:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:51 UTC)" executed successfully 2020-03-31 09:38:51,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:38:51,445 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:51,446 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:01.440707+00:00 (in 9.994979 seconds) 2020-03-31 09:38:51,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:01 UTC)" (scheduled at 2020-03-31 09:38:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:38:51,446 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:01.440707+00:00 (in 9.994979 seconds) 2020-03-31 09:38:51,446 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:01 UTC)" (scheduled at 2020-03-31 09:38:51.440707+00:00) 2020-03-31 09:38:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:38:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:38:51,468 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:38:51,468 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:38:51,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:51,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:38:51,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:51,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:01 UTC)" executed successfully 2020-03-31 09:38:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:38:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:22.113221+00:00 (in 29.999132 seconds) securityworker stdout | 2020-03-31 09:38:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:22.113221+00:00 (in 29.999132 seconds) 2020-03-31 09:38:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:22 UTC)" (scheduled at 2020-03-31 09:38:52.113221+00:00) 2020-03-31 09:38:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:38:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:22 UTC)" (scheduled at 2020-03-31 09:38:52.113221+00:00) securityworker stdout | 2020-03-31 09:38:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:38:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:38:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:38:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:38:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:38:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:38:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:22 UTC)" executed successfully 2020-03-31 09:38:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:38:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:04.323352+00:00 (in 9.998696 seconds) 2020-03-31 09:38:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:04 UTC)" (scheduled at 2020-03-31 09:38:54.323352+00:00) 2020-03-31 09:38:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:38:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:04.323352+00:00 (in 9.998696 seconds) 2020-03-31 09:38:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:04 UTC)" (scheduled at 2020-03-31 09:38:54.323352+00:00) 2020-03-31 09:38:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 54, 325676), True, datetime.datetime(2020, 3, 31, 9, 38, 54, 325676), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:38:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 54, 325676), True, datetime.datetime(2020, 3, 31, 9, 38, 54, 325676), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:38:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:38:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:38:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:04 UTC)" executed successfully 2020-03-31 09:38:54,445 [518] [DEBUG] [app] Starting request: urn:request:082ac947-b8f9-4f53-9eea-128599be74da (/health/instance) gunicorn-web stdout | 2020-03-31 09:38:54,445 [518] [DEBUG] [app] Starting request: urn:request:082ac947-b8f9-4f53-9eea-128599be74da (/health/instance) 2020-03-31 09:38:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:54,452 [555] [DEBUG] [app] Starting request: urn:request:99eea1b5-2532-4254-9bb6-f3c42ca8480a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:54,452 [555] [DEBUG] [app] Starting request: urn:request:99eea1b5-2532-4254-9bb6-f3c42ca8480a (/v1/_internal_ping) 2020-03-31 09:38:54,454 [555] [DEBUG] [app] Ending request: urn:request:99eea1b5-2532-4254-9bb6-f3c42ca8480a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:54,454 [555] [DEBUG] [app] Ending request: urn:request:99eea1b5-2532-4254-9bb6-f3c42ca8480a (/v1/_internal_ping) 2020-03-31 09:38:54,454 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:38:54,454 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:54,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:38:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:38:54,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:54,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:54,457 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:54,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:54,459 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:54,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:54,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:54,487 [518] [DEBUG] [app] Starting request: urn:request:a8a8f2c2-d311-4450-8020-bc9c2a8c6546 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:54,487 [518] [DEBUG] [app] Starting request: urn:request:a8a8f2c2-d311-4450-8020-bc9c2a8c6546 (/_internal_ping) 2020-03-31 09:38:54,488 [518] [DEBUG] [app] Ending request: urn:request:a8a8f2c2-d311-4450-8020-bc9c2a8c6546 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:54,488 [518] [DEBUG] [app] Ending request: urn:request:a8a8f2c2-d311-4450-8020-bc9c2a8c6546 (/_internal_ping) 2020-03-31 09:38:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:38:54,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:38:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:54,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:38:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:38:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:54,501 [541] [DEBUG] [app] Starting request: urn:request:536bcdfb-b367-42d1-b6bd-67e9c5115809 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:54,501 [541] [DEBUG] [app] Starting request: urn:request:536bcdfb-b367-42d1-b6bd-67e9c5115809 (/c1/_internal_ping) 2020-03-31 09:38:54,502 [541] [DEBUG] [app] Ending request: urn:request:536bcdfb-b367-42d1-b6bd-67e9c5115809 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:54,502 [541] [DEBUG] [app] Ending request: urn:request:536bcdfb-b367-42d1-b6bd-67e9c5115809 (/c1/_internal_ping) 2020-03-31 09:38:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:38:54,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:38:54,502 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:54,504 [518] [DEBUG] [app] Ending request: urn:request:082ac947-b8f9-4f53-9eea-128599be74da (/health/instance) 2020-03-31 09:38:54,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:54,504 [518] [DEBUG] [app] Ending request: urn:request:082ac947-b8f9-4f53-9eea-128599be74da (/health/instance) 2020-03-31 09:38:54,504 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:38:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:38:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:38:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:38:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:06.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:38:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:06 UTC)" (scheduled at 2020-03-31 09:38:56.924497+00:00) 2020-03-31 09:38:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:38:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:06.924497+00:00 (in 9.999199 seconds) 2020-03-31 09:38:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:06 UTC)" (scheduled at 2020-03-31 09:38:56.924497+00:00) 2020-03-31 09:38:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:38:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 56, 926008), True, datetime.datetime(2020, 3, 31, 9, 38, 56, 926008), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:38:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 38, 56, 926008), True, datetime.datetime(2020, 3, 31, 9, 38, 56, 926008), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:38:56,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:38:56,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:38:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:56,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:38:56,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:56,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:06 UTC)" executed successfully 2020-03-31 09:38:58,273 [518] [DEBUG] [app] Starting request: urn:request:af72402e-f4f9-45b5-96ce-5b0c44f2e541 (/health/instance) 2020-03-31 09:38:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:58,273 [518] [DEBUG] [app] Starting request: urn:request:af72402e-f4f9-45b5-96ce-5b0c44f2e541 (/health/instance) 2020-03-31 09:38:58,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:58,279 [556] [DEBUG] [app] Starting request: urn:request:7d49fc11-c162-445d-9016-2822cf3994a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:58,279 [556] [DEBUG] [app] Starting request: urn:request:7d49fc11-c162-445d-9016-2822cf3994a0 (/v1/_internal_ping) 2020-03-31 09:38:58,280 [556] [DEBUG] [app] Ending request: urn:request:7d49fc11-c162-445d-9016-2822cf3994a0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:38:58,280 [556] [DEBUG] [app] Ending request: urn:request:7d49fc11-c162-445d-9016-2822cf3994a0 (/v1/_internal_ping) 2020-03-31 09:38:58,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:38:58,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:38:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:38:58,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:38:58,283 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:38:58,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:38:58,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:38:58,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:38:58,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:38:58,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:38:58,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:38:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:38:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:38:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:58,310 [520] [DEBUG] [app] Starting request: urn:request:ed633b23-f9de-49ae-9f27-bdc713f0488e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:58,310 [520] [DEBUG] [app] Starting request: urn:request:ed633b23-f9de-49ae-9f27-bdc713f0488e (/_internal_ping) 2020-03-31 09:38:58,311 [520] [DEBUG] [app] Ending request: urn:request:ed633b23-f9de-49ae-9f27-bdc713f0488e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:38:58,311 [520] [DEBUG] [app] Ending request: urn:request:ed633b23-f9de-49ae-9f27-bdc713f0488e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:38:58,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:38:58,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:38:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:38:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:38:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:38:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:38:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:38:58,326 [541] [DEBUG] [app] Starting request: urn:request:c7ef2c7e-6e93-48fd-ba1b-85ca48216b03 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:58,326 [541] [DEBUG] [app] Starting request: urn:request:c7ef2c7e-6e93-48fd-ba1b-85ca48216b03 (/c1/_internal_ping) 2020-03-31 09:38:58,327 [541] [DEBUG] [app] Ending request: urn:request:c7ef2c7e-6e93-48fd-ba1b-85ca48216b03 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:38:58,327 [541] [DEBUG] [app] Ending request: urn:request:c7ef2c7e-6e93-48fd-ba1b-85ca48216b03 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:38:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:38:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:38:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:38:58,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:38:58,330 [518] [DEBUG] [app] Ending request: urn:request:af72402e-f4f9-45b5-96ce-5b0c44f2e541 (/health/instance) 2020-03-31 09:38:58,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:38:58,330 [518] [DEBUG] [app] Ending request: urn:request:af72402e-f4f9-45b5-96ce-5b0c44f2e541 (/health/instance) 2020-03-31 09:38:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:38:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:38:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:38:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:38:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:38:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:38:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:38:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:51.110610+00:00 (in 51.549206 seconds) 2020-03-31 09:38:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:59 UTC)" (scheduled at 2020-03-31 09:38:59.560326+00:00) 2020-03-31 09:38:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:38:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:51.110610+00:00 (in 51.549206 seconds) 2020-03-31 09:38:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:59 UTC)" (scheduled at 2020-03-31 09:38:59.560326+00:00) 2020-03-31 09:38:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:38:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:38:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:59 UTC)" executed successfully 2020-03-31 09:39:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:39:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:30.337699+00:00 (in 29.999296 seconds) 2020-03-31 09:39:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:30 UTC)" (scheduled at 2020-03-31 09:39:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:39:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:30.337699+00:00 (in 29.999296 seconds) 2020-03-31 09:39:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:30 UTC)" (scheduled at 2020-03-31 09:39:00.337699+00:00) 2020-03-31 09:39:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 39, 0, 339160), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:39:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 39, 0, 339160), False, 50, 1, 0]) 2020-03-31 09:39:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:39:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:39:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:39:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:30 UTC)" executed successfully 2020-03-31 09:39:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:39:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:11.440707+00:00 (in 9.998743 seconds) 2020-03-31 09:39:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:11 UTC)" (scheduled at 2020-03-31 09:39:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:39:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:11.440707+00:00 (in 9.998743 seconds) 2020-03-31 09:39:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:11 UTC)" (scheduled at 2020-03-31 09:39:01.440707+00:00) 2020-03-31 09:39:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:39:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:39:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:39:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:39:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:39:01,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:01,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:11 UTC)" executed successfully 2020-03-31 09:39:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:39:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:23.410371+00:00 (in 20.079351 seconds) 2020-03-31 09:39:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:33 UTC)" (scheduled at 2020-03-31 09:39:03.330112+00:00) 2020-03-31 09:39:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:39:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:23.410371+00:00 (in 20.079351 seconds) 2020-03-31 09:39:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:33 UTC)" (scheduled at 2020-03-31 09:39:03.330112+00:00) 2020-03-31 09:39:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 3, 331868), True, datetime.datetime(2020, 3, 31, 9, 39, 3, 331868), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:39:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 3, 331868), True, datetime.datetime(2020, 3, 31, 9, 39, 3, 331868), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:39:03,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:39:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:39:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:33 UTC)" executed successfully 2020-03-31 09:39:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:39:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:04 UTC)" (scheduled at 2020-03-31 09:39:04.323352+00:00) 2020-03-31 09:39:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:14.323352+00:00 (in 9.998753 seconds) notificationworker stdout | 2020-03-31 09:39:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:04 UTC)" (scheduled at 2020-03-31 09:39:04.323352+00:00) 2020-03-31 09:39:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:14.323352+00:00 (in 9.998753 seconds) 2020-03-31 09:39:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 4, 325384), True, datetime.datetime(2020, 3, 31, 9, 39, 4, 325384), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:39:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 4, 325384), True, datetime.datetime(2020, 3, 31, 9, 39, 4, 325384), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:39:04,347 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:39:04,347 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:04,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:04,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:39:04,347 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:04,347 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:14 UTC)" executed successfully 2020-03-31 09:39:04,445 [526] [DEBUG] [app] Starting request: urn:request:ee75b266-cc47-443d-8f16-831373787aa0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:04,445 [526] [DEBUG] [app] Starting request: urn:request:ee75b266-cc47-443d-8f16-831373787aa0 (/health/instance) 2020-03-31 09:39:04,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:04,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:04,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:04,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:04,453 [553] [DEBUG] [app] Starting request: urn:request:7dbed730-8991-4201-ac3d-e8f1ea166d4d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:04,453 [553] [DEBUG] [app] Starting request: urn:request:7dbed730-8991-4201-ac3d-e8f1ea166d4d (/v1/_internal_ping) 2020-03-31 09:39:04,454 [553] [DEBUG] [app] Ending request: urn:request:7dbed730-8991-4201-ac3d-e8f1ea166d4d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:04,454 [553] [DEBUG] [app] Ending request: urn:request:7dbed730-8991-4201-ac3d-e8f1ea166d4d (/v1/_internal_ping) 2020-03-31 09:39:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:04,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:39:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:04,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:04,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:39:04,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:04,458 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:04,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:04,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:04,465 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:04,465 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:04,479 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:04,479 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:04,483 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:04,483 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:04,485 [526] [DEBUG] [app] Starting request: urn:request:ebb48343-8da6-449e-ba62-2fcf3713b75e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:04,485 [526] [DEBUG] [app] Starting request: urn:request:ebb48343-8da6-449e-ba62-2fcf3713b75e (/_internal_ping) 2020-03-31 09:39:04,486 [526] [DEBUG] [app] Ending request: urn:request:ebb48343-8da6-449e-ba62-2fcf3713b75e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:04,486 [526] [DEBUG] [app] Ending request: urn:request:ebb48343-8da6-449e-ba62-2fcf3713b75e (/_internal_ping) 2020-03-31 09:39:04,486 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:04,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:04,486 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:04,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:39:04,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:04,491 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:04,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:04,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:04,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:04,495 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:39:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:04,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:04,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:04,502 [534] [DEBUG] [app] Starting request: urn:request:94411558-dec2-4be6-835b-7c72d3501ed0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:04,502 [534] [DEBUG] [app] Starting request: urn:request:94411558-dec2-4be6-835b-7c72d3501ed0 (/c1/_internal_ping) 2020-03-31 09:39:04,503 [534] [DEBUG] [app] Ending request: urn:request:94411558-dec2-4be6-835b-7c72d3501ed0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:04,503 [534] [DEBUG] [app] Ending request: urn:request:94411558-dec2-4be6-835b-7c72d3501ed0 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:04,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:39:04,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:04,505 [526] [DEBUG] [app] Ending request: urn:request:ee75b266-cc47-443d-8f16-831373787aa0 (/health/instance) 2020-03-31 09:39:04,505 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:04,505 [526] [DEBUG] [app] Ending request: urn:request:ee75b266-cc47-443d-8f16-831373787aa0 (/health/instance) 2020-03-31 09:39:04,505 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:04,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:39:04,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:39:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:39:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:16.924497+00:00 (in 9.999274 seconds) 2020-03-31 09:39:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:16 UTC)" (scheduled at 2020-03-31 09:39:06.924497+00:00) 2020-03-31 09:39:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:39:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:16.924497+00:00 (in 9.999274 seconds) 2020-03-31 09:39:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:16 UTC)" (scheduled at 2020-03-31 09:39:06.924497+00:00) 2020-03-31 09:39:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 6, 925941), True, datetime.datetime(2020, 3, 31, 9, 39, 6, 925941), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:39:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 6, 925941), True, datetime.datetime(2020, 3, 31, 9, 39, 6, 925941), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:39:06,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:39:06,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:06,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:06,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:39:06,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:06,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:16 UTC)" executed successfully 2020-03-31 09:39:08,272 [518] [DEBUG] [app] Starting request: urn:request:d130303a-1971-4663-8f23-e5817b14d7a1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:08,272 [518] [DEBUG] [app] Starting request: urn:request:d130303a-1971-4663-8f23-e5817b14d7a1 (/health/instance) 2020-03-31 09:39:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:08,279 [548] [DEBUG] [app] Starting request: urn:request:4ae1d126-fd73-490f-b5dc-8519d19c1648 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:08,279 [548] [DEBUG] [app] Starting request: urn:request:4ae1d126-fd73-490f-b5dc-8519d19c1648 (/v1/_internal_ping) 2020-03-31 09:39:08,280 [548] [DEBUG] [app] Ending request: urn:request:4ae1d126-fd73-490f-b5dc-8519d19c1648 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:08,280 [548] [DEBUG] [app] Ending request: urn:request:4ae1d126-fd73-490f-b5dc-8519d19c1648 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:08,281 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:39:08,281 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:08,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:08,284 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:08,285 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:08,285 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:08,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:08,286 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:08,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:08,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:08,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:08,311 [518] [DEBUG] [app] Starting request: urn:request:a4dc5016-cf4d-4f31-8b5c-8696a30ed60d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:08,311 [518] [DEBUG] [app] Starting request: urn:request:a4dc5016-cf4d-4f31-8b5c-8696a30ed60d (/_internal_ping) 2020-03-31 09:39:08,312 [518] [DEBUG] [app] Ending request: urn:request:a4dc5016-cf4d-4f31-8b5c-8696a30ed60d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:08,312 [518] [DEBUG] [app] Ending request: urn:request:a4dc5016-cf4d-4f31-8b5c-8696a30ed60d (/_internal_ping) 2020-03-31 09:39:08,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:08,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:08,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:39:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:08,326 [541] [DEBUG] [app] Starting request: urn:request:1c75b25d-1797-4415-9fae-31344e7ba8de (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:08,326 [541] [DEBUG] [app] Starting request: urn:request:1c75b25d-1797-4415-9fae-31344e7ba8de (/c1/_internal_ping) 2020-03-31 09:39:08,327 [541] [DEBUG] [app] Ending request: urn:request:1c75b25d-1797-4415-9fae-31344e7ba8de (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:08,327 [541] [DEBUG] [app] Ending request: urn:request:1c75b25d-1797-4415-9fae-31344e7ba8de (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:08,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:39:08,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:08,329 [518] [DEBUG] [app] Ending request: urn:request:d130303a-1971-4663-8f23-e5817b14d7a1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:08,329 [518] [DEBUG] [app] Ending request: urn:request:d130303a-1971-4663-8f23-e5817b14d7a1 (/health/instance) 2020-03-31 09:39:08,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:39:08,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:08,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:39:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:39:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:21.440707+00:00 (in 9.999045 seconds) 2020-03-31 09:39:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:21 UTC)" (scheduled at 2020-03-31 09:39:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:39:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:21.440707+00:00 (in 9.999045 seconds) 2020-03-31 09:39:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:21 UTC)" (scheduled at 2020-03-31 09:39:11.440707+00:00) 2020-03-31 09:39:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:39:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:39:11,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:39:11,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:39:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:39:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:21 UTC)" executed successfully 2020-03-31 09:39:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:39:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:24.323352+00:00 (in 9.998448 seconds) 2020-03-31 09:39:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:24 UTC)" (scheduled at 2020-03-31 09:39:14.323352+00:00) 2020-03-31 09:39:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:39:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:24.323352+00:00 (in 9.998448 seconds) 2020-03-31 09:39:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:24 UTC)" (scheduled at 2020-03-31 09:39:14.323352+00:00) 2020-03-31 09:39:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 14, 325746), True, datetime.datetime(2020, 3, 31, 9, 39, 14, 325746), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:39:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 14, 325746), True, datetime.datetime(2020, 3, 31, 9, 39, 14, 325746), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:39:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:39:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:39:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:24 UTC)" executed successfully 2020-03-31 09:39:14,445 [526] [DEBUG] [app] Starting request: urn:request:6c55d5f7-c112-4d48-8e5f-6ef0a4084d02 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:14,445 [526] [DEBUG] [app] Starting request: urn:request:6c55d5f7-c112-4d48-8e5f-6ef0a4084d02 (/health/instance) 2020-03-31 09:39:14,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:14,447 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:14,453 [538] [DEBUG] [app] Starting request: urn:request:f59a10dc-45ec-42a0-a3f1-4f9522eff107 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:14,453 [538] [DEBUG] [app] Starting request: urn:request:f59a10dc-45ec-42a0-a3f1-4f9522eff107 (/v1/_internal_ping) 2020-03-31 09:39:14,454 [538] [DEBUG] [app] Ending request: urn:request:f59a10dc-45ec-42a0-a3f1-4f9522eff107 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:14,454 [538] [DEBUG] [app] Ending request: urn:request:f59a10dc-45ec-42a0-a3f1-4f9522eff107 (/v1/_internal_ping) 2020-03-31 09:39:14,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:39:14,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:14,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:14,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:14,458 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:14,458 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:14,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:14,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:14,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:14,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:14,483 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:14,487 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:14,487 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:14,488 [526] [DEBUG] [app] Starting request: urn:request:934de4f5-643a-43cb-b164-612c7423b17a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:14,488 [526] [DEBUG] [app] Starting request: urn:request:934de4f5-643a-43cb-b164-612c7423b17a (/_internal_ping) 2020-03-31 09:39:14,489 [526] [DEBUG] [app] Ending request: urn:request:934de4f5-643a-43cb-b164-612c7423b17a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:14,489 [526] [DEBUG] [app] Ending request: urn:request:934de4f5-643a-43cb-b164-612c7423b17a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:39:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:14,490 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:14,489 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:14,490 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:14,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:14,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:39:14,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:39:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:14,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:14,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:39:14,497 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:39:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:14,501 [534] [DEBUG] [app] Starting request: urn:request:5caebfe0-3020-4783-b4e4-77dff134abcc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:14,501 [534] [DEBUG] [app] Starting request: urn:request:5caebfe0-3020-4783-b4e4-77dff134abcc (/c1/_internal_ping) 2020-03-31 09:39:14,503 [534] [DEBUG] [app] Ending request: urn:request:5caebfe0-3020-4783-b4e4-77dff134abcc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:14,503 [534] [DEBUG] [app] Ending request: urn:request:5caebfe0-3020-4783-b4e4-77dff134abcc (/c1/_internal_ping) 2020-03-31 09:39:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:14,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:14,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:39:14,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:14,505 [526] [DEBUG] [app] Ending request: urn:request:6c55d5f7-c112-4d48-8e5f-6ef0a4084d02 (/health/instance) 2020-03-31 09:39:14,505 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:14,505 [526] [DEBUG] [app] Ending request: urn:request:6c55d5f7-c112-4d48-8e5f-6ef0a4084d02 (/health/instance) 2020-03-31 09:39:14,505 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:39:14,505 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:39:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:39:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:45.064514+00:00 (in 29.999036 seconds) 2020-03-31 09:39:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:45 UTC)" (scheduled at 2020-03-31 09:39:15.064514+00:00) gcworker stdout | 2020-03-31 09:39:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:45.064514+00:00 (in 29.999036 seconds) 2020-03-31 09:39:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:45 UTC)" (scheduled at 2020-03-31 09:39:15.064514+00:00) 2020-03-31 09:39:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437955, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:39:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437955, 1209600, 500, 1, 0]) 2020-03-31 09:39:15,088 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:39:15,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:39:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:15,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:39:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:15,089 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:45 UTC)" executed successfully 2020-03-31 09:39:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:39:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:26.924497+00:00 (in 9.999264 seconds) 2020-03-31 09:39:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:26 UTC)" (scheduled at 2020-03-31 09:39:16.924497+00:00) 2020-03-31 09:39:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:39:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:26.924497+00:00 (in 9.999264 seconds) 2020-03-31 09:39:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:26 UTC)" (scheduled at 2020-03-31 09:39:16.924497+00:00) 2020-03-31 09:39:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 16, 925910), True, datetime.datetime(2020, 3, 31, 9, 39, 16, 925910), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:39:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 16, 925910), True, datetime.datetime(2020, 3, 31, 9, 39, 16, 925910), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:39:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:39:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:39:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:26 UTC)" executed successfully 2020-03-31 09:39:18,272 [525] [DEBUG] [app] Starting request: urn:request:32d3f82f-52b9-406b-b83e-fe8cefe65011 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:18,272 [525] [DEBUG] [app] Starting request: urn:request:32d3f82f-52b9-406b-b83e-fe8cefe65011 (/health/instance) 2020-03-31 09:39:18,274 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:18,274 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:18,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:18,278 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:18,279 [545] [DEBUG] [app] Starting request: urn:request:0ec0f1ed-41e3-4a4b-acf8-f87cb0857a7e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:18,279 [545] [DEBUG] [app] Starting request: urn:request:0ec0f1ed-41e3-4a4b-acf8-f87cb0857a7e (/v1/_internal_ping) 2020-03-31 09:39:18,280 [545] [DEBUG] [app] Ending request: urn:request:0ec0f1ed-41e3-4a4b-acf8-f87cb0857a7e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:18,280 [545] [DEBUG] [app] Ending request: urn:request:0ec0f1ed-41e3-4a4b-acf8-f87cb0857a7e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:18,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:39:18,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:18,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:18,282 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:18,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:18,283 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:18,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:18,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:18,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:18,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:18,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:18,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:18,297 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:18,297 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:18,311 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:18,311 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:18,315 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:18,315 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:18,316 [525] [DEBUG] [app] Starting request: urn:request:a8b27c42-8c62-4631-a0c7-825a510ed23e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:18,316 [525] [DEBUG] [app] Starting request: urn:request:a8b27c42-8c62-4631-a0c7-825a510ed23e (/_internal_ping) 2020-03-31 09:39:18,317 [525] [DEBUG] [app] Ending request: urn:request:a8b27c42-8c62-4631-a0c7-825a510ed23e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:18,317 [525] [DEBUG] [app] Ending request: urn:request:a8b27c42-8c62-4631-a0c7-825a510ed23e (/_internal_ping) 2020-03-31 09:39:18,317 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:39:18,317 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:18,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:39:18,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:18,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:18,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:18,321 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:39:18,321 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:39:18,324 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:18,324 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:39:18,325 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:18,325 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:18,328 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:18,328 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:18,329 [537] [DEBUG] [app] Starting request: urn:request:1a3e430b-43a8-4684-bbba-7b48b59425df (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:18,329 [537] [DEBUG] [app] Starting request: urn:request:1a3e430b-43a8-4684-bbba-7b48b59425df (/c1/_internal_ping) 2020-03-31 09:39:18,330 [537] [DEBUG] [app] Ending request: urn:request:1a3e430b-43a8-4684-bbba-7b48b59425df (/c1/_internal_ping) 2020-03-31 09:39:18,331 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:18,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:39:18,330 [537] [DEBUG] [app] Ending request: urn:request:1a3e430b-43a8-4684-bbba-7b48b59425df (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:18,331 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:18,332 [525] [DEBUG] [app] Ending request: urn:request:32d3f82f-52b9-406b-b83e-fe8cefe65011 (/health/instance) 2020-03-31 09:39:18,333 [525] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:39:18,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:39:18,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:18,332 [525] [DEBUG] [app] Ending request: urn:request:32d3f82f-52b9-406b-b83e-fe8cefe65011 (/health/instance) 2020-03-31 09:39:18,333 [525] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:39:18,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:39:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:39:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:31.440707+00:00 (in 9.998077 seconds) 2020-03-31 09:39:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:31 UTC)" (scheduled at 2020-03-31 09:39:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:39:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:31.440707+00:00 (in 9.998077 seconds) 2020-03-31 09:39:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:31 UTC)" (scheduled at 2020-03-31 09:39:21.440707+00:00) 2020-03-31 09:39:21,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:39:21,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:39:21,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:39:21,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:39:21,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:39:21,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:21,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:31 UTC)" executed successfully 2020-03-31 09:39:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:39:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:52 UTC)" (scheduled at 2020-03-31 09:39:22.113221+00:00) securityworker stdout | 2020-03-31 09:39:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:52 UTC)" (scheduled at 2020-03-31 09:39:22.113221+00:00) 2020-03-31 09:39:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:52.113221+00:00 (in 29.999326 seconds) 2020-03-31 09:39:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:39:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:52.113221+00:00 (in 29.999326 seconds) 2020-03-31 09:39:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:39:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:39:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:39:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:39:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:39:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:39:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:39:52 UTC)" executed successfully 2020-03-31 09:39:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:39:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:23 UTC)" (scheduled at 2020-03-31 09:39:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:39:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:39:23 UTC)" (scheduled at 2020-03-31 09:39:23.410371+00:00) 2020-03-31 09:39:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:39:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), u'security_notification/%']) 2020-03-31 09:39:23,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:24.410927+00:00 (in 0.999612 seconds) security_notification_worker stdout | 2020-03-31 09:39:23,413 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:24.410927+00:00 (in 0.999612 seconds) 2020-03-31 09:39:23,426 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), True, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:39:23,426 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), True, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), 0, u'security_notification/%']) 2020-03-31 09:39:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), True, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:39:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), True, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 39, 23, 411766), u'security_notification/%']) 2020-03-31 09:39:23,436 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:39:23,436 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:23,439 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:44:23 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:39:23,439 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:44:23 UTC)" executed successfully 2020-03-31 09:39:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:39:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:39:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:39:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:39:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:39:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:34.323352+00:00 (in 9.999080 seconds) 2020-03-31 09:39:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:34 UTC)" (scheduled at 2020-03-31 09:39:24.323352+00:00) 2020-03-31 09:39:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:39:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:34.323352+00:00 (in 9.999080 seconds) 2020-03-31 09:39:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:34 UTC)" (scheduled at 2020-03-31 09:39:24.323352+00:00) 2020-03-31 09:39:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 24, 325282), True, datetime.datetime(2020, 3, 31, 9, 39, 24, 325282), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:39:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 24, 325282), True, datetime.datetime(2020, 3, 31, 9, 39, 24, 325282), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:39:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:24,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:39:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:39:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:34 UTC)" executed successfully 2020-03-31 09:39:24,413 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:39:24,413 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:24,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:33.330112+00:00 (in 8.916153 seconds) 2020-03-31 09:39:24,414 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:24 UTC)" (scheduled at 2020-03-31 09:39:24.410927+00:00) 2020-03-31 09:39:24,414 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:24,414 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:39:24,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:33.330112+00:00 (in 8.916153 seconds) 2020-03-31 09:39:24,414 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:24 UTC)" (scheduled at 2020-03-31 09:39:24.410927+00:00) 2020-03-31 09:39:24,414 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:24,414 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:24 UTC)" executed successfully 2020-03-31 09:39:24,445 [518] [DEBUG] [app] Starting request: urn:request:6c2e8aa6-3f39-4993-872c-4906759e87f2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:24,445 [518] [DEBUG] [app] Starting request: urn:request:6c2e8aa6-3f39-4993-872c-4906759e87f2 (/health/instance) 2020-03-31 09:39:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:24,453 [546] [DEBUG] [app] Starting request: urn:request:1a3096c2-51cb-4275-9d10-345219875399 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:24,453 [546] [DEBUG] [app] Starting request: urn:request:1a3096c2-51cb-4275-9d10-345219875399 (/v1/_internal_ping) 2020-03-31 09:39:24,453 [546] [DEBUG] [app] Ending request: urn:request:1a3096c2-51cb-4275-9d10-345219875399 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:24,453 [546] [DEBUG] [app] Ending request: urn:request:1a3096c2-51cb-4275-9d10-345219875399 (/v1/_internal_ping) 2020-03-31 09:39:24,454 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:39:24,454 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:24,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:24,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:24,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:24,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:24,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:24,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:24,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:24,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:24,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:24,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:24,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:24,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:24,484 [518] [DEBUG] [app] Starting request: urn:request:73954793-c0a5-46b9-8e39-4a4c04f8a0a6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:24,484 [518] [DEBUG] [app] Starting request: urn:request:73954793-c0a5-46b9-8e39-4a4c04f8a0a6 (/_internal_ping) 2020-03-31 09:39:24,484 [518] [DEBUG] [app] Ending request: urn:request:73954793-c0a5-46b9-8e39-4a4c04f8a0a6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:24,484 [518] [DEBUG] [app] Ending request: urn:request:73954793-c0a5-46b9-8e39-4a4c04f8a0a6 (/_internal_ping) 2020-03-31 09:39:24,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:39:24,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:24,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:39:24,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:24,488 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:24,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:24,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:39:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:24,497 [537] [DEBUG] [app] Starting request: urn:request:9be72060-a1ea-48ee-9428-c92af866d210 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:24,497 [537] [DEBUG] [app] Starting request: urn:request:9be72060-a1ea-48ee-9428-c92af866d210 (/c1/_internal_ping) 2020-03-31 09:39:24,498 [537] [DEBUG] [app] Ending request: urn:request:9be72060-a1ea-48ee-9428-c92af866d210 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:24,498 [537] [DEBUG] [app] Ending request: urn:request:9be72060-a1ea-48ee-9428-c92af866d210 (/c1/_internal_ping) 2020-03-31 09:39:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:24,498 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:24,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:24,500 [518] [DEBUG] [app] Ending request: urn:request:6c2e8aa6-3f39-4993-872c-4906759e87f2 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:39:24,498 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:24,500 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:24,500 [518] [DEBUG] [app] Ending request: urn:request:6c2e8aa6-3f39-4993-872c-4906759e87f2 (/health/instance) 2020-03-31 09:39:24,500 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:39:24,500 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:24,500 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:39:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:36.924497+00:00 (in 9.999215 seconds) 2020-03-31 09:39:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:36 UTC)" (scheduled at 2020-03-31 09:39:26.924497+00:00) 2020-03-31 09:39:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 26, 926046), True, datetime.datetime(2020, 3, 31, 9, 39, 26, 926046), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:39:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:36.924497+00:00 (in 9.999215 seconds) 2020-03-31 09:39:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:36 UTC)" (scheduled at 2020-03-31 09:39:26.924497+00:00) 2020-03-31 09:39:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 26, 926046), True, datetime.datetime(2020, 3, 31, 9, 39, 26, 926046), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:39:26,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:39:26,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:39:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:36 UTC)" executed successfully 2020-03-31 09:39:28,272 [518] [DEBUG] [app] Starting request: urn:request:15dd7e73-e599-4d72-b693-d90a49476ff6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:28,272 [518] [DEBUG] [app] Starting request: urn:request:15dd7e73-e599-4d72-b693-d90a49476ff6 (/health/instance) 2020-03-31 09:39:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:28,278 [549] [DEBUG] [app] Starting request: urn:request:6a31a240-eeec-4649-8173-d1cf180834ba (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:28,278 [549] [DEBUG] [app] Starting request: urn:request:6a31a240-eeec-4649-8173-d1cf180834ba (/v1/_internal_ping) 2020-03-31 09:39:28,279 [549] [DEBUG] [app] Ending request: urn:request:6a31a240-eeec-4649-8173-d1cf180834ba (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:28,279 [549] [DEBUG] [app] Ending request: urn:request:6a31a240-eeec-4649-8173-d1cf180834ba (/v1/_internal_ping) 2020-03-31 09:39:28,280 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:39:28,280 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:28,282 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:39:28,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:39:28,282 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:28,284 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:28,284 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:28,285 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:28,285 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:28,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:28,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:28,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:28,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:28,312 [518] [DEBUG] [app] Starting request: urn:request:7cfef38d-8ec8-438f-aac8-3f10fd4b9c1b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:28,312 [518] [DEBUG] [app] Starting request: urn:request:7cfef38d-8ec8-438f-aac8-3f10fd4b9c1b (/_internal_ping) 2020-03-31 09:39:28,313 [518] [DEBUG] [app] Ending request: urn:request:7cfef38d-8ec8-438f-aac8-3f10fd4b9c1b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:28,313 [518] [DEBUG] [app] Ending request: urn:request:7cfef38d-8ec8-438f-aac8-3f10fd4b9c1b (/_internal_ping) 2020-03-31 09:39:28,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:39:28,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:28,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:39:28,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:28,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:39:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:39:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:28,327 [541] [DEBUG] [app] Starting request: urn:request:b56dbc5e-1e8c-4f2e-8817-d7d3c7f58e10 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:28,327 [541] [DEBUG] [app] Starting request: urn:request:b56dbc5e-1e8c-4f2e-8817-d7d3c7f58e10 (/c1/_internal_ping) 2020-03-31 09:39:28,327 [541] [DEBUG] [app] Ending request: urn:request:b56dbc5e-1e8c-4f2e-8817-d7d3c7f58e10 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:28,327 [541] [DEBUG] [app] Ending request: urn:request:b56dbc5e-1e8c-4f2e-8817-d7d3c7f58e10 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:28,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:39:28,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:28,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:28,330 [518] [DEBUG] [app] Ending request: urn:request:15dd7e73-e599-4d72-b693-d90a49476ff6 (/health/instance) 2020-03-31 09:39:28,330 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:39:28,330 [518] [DEBUG] [app] Ending request: urn:request:15dd7e73-e599-4d72-b693-d90a49476ff6 (/health/instance) 2020-03-31 09:39:28,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:28,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:39:28,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:39:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:39:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:00.337699+00:00 (in 29.999244 seconds) 2020-03-31 09:39:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:00 UTC)" (scheduled at 2020-03-31 09:39:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:39:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:00.337699+00:00 (in 29.999244 seconds) 2020-03-31 09:39:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:00 UTC)" (scheduled at 2020-03-31 09:39:30.337699+00:00) 2020-03-31 09:39:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 39, 30, 339306), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:39:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 39, 30, 339306), False, 50, 1, 0]) 2020-03-31 09:39:30,367 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:39:30,367 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:39:30,367 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:30,367 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:39:30,367 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:30,367 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:00 UTC)" executed successfully 2020-03-31 09:39:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:39:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:41.440707+00:00 (in 9.999213 seconds) 2020-03-31 09:39:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:41 UTC)" (scheduled at 2020-03-31 09:39:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:39:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:41.440707+00:00 (in 9.999213 seconds) 2020-03-31 09:39:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:41 UTC)" (scheduled at 2020-03-31 09:39:31.440707+00:00) 2020-03-31 09:39:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:39:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:39:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:39:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:39:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:39:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:41 UTC)" executed successfully 2020-03-31 09:39:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:39:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:03.330112+00:00 (in 29.999166 seconds) 2020-03-31 09:39:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:03 UTC)" (scheduled at 2020-03-31 09:39:33.330112+00:00) 2020-03-31 09:39:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:39:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:03.330112+00:00 (in 29.999166 seconds) 2020-03-31 09:39:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:03 UTC)" (scheduled at 2020-03-31 09:39:33.330112+00:00) 2020-03-31 09:39:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 33, 331678), True, datetime.datetime(2020, 3, 31, 9, 39, 33, 331678), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:39:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 33, 331678), True, datetime.datetime(2020, 3, 31, 9, 39, 33, 331678), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:39:33,355 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:39:33,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:33,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:33,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:39:33,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:33,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:03 UTC)" executed successfully 2020-03-31 09:39:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:39:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:43.539999+00:00 (in 10.172618 seconds) 2020-03-31 09:39:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:33 UTC)" (scheduled at 2020-03-31 09:39:33.366791+00:00) 2020-03-31 09:39:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:39:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:43.539999+00:00 (in 10.172618 seconds) 2020-03-31 09:39:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:33 UTC)" (scheduled at 2020-03-31 09:39:33.366791+00:00) 2020-03-31 09:39:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 33, 368176), True, datetime.datetime(2020, 3, 31, 9, 39, 33, 368176), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:39:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 33, 368176), True, datetime.datetime(2020, 3, 31, 9, 39, 33, 368176), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:39:33,386 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:39:33,386 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:39:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:33 UTC)" executed successfully 2020-03-31 09:39:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:39:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:44 UTC)" (scheduled at 2020-03-31 09:39:34.323352+00:00) 2020-03-31 09:39:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:39:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:44 UTC)" (scheduled at 2020-03-31 09:39:34.323352+00:00) 2020-03-31 09:39:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 34, 325159), True, datetime.datetime(2020, 3, 31, 9, 39, 34, 325159), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:39:34,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 34, 325159), True, datetime.datetime(2020, 3, 31, 9, 39, 34, 325159), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:39:34,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:44.323352+00:00 (in 9.998948 seconds) notificationworker stdout | 2020-03-31 09:39:34,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:44.323352+00:00 (in 9.998948 seconds) 2020-03-31 09:39:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:39:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:39:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:44 UTC)" executed successfully 2020-03-31 09:39:34,445 [518] [DEBUG] [app] Starting request: urn:request:0e966bcb-e5e7-45c6-9a40-ab0ed2ba5fc4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:34,445 [518] [DEBUG] [app] Starting request: urn:request:0e966bcb-e5e7-45c6-9a40-ab0ed2ba5fc4 (/health/instance) 2020-03-31 09:39:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:34,452 [556] [DEBUG] [app] Starting request: urn:request:bd9b19b0-de7f-4bbd-b361-79dd50b5878c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:34,452 [556] [DEBUG] [app] Starting request: urn:request:bd9b19b0-de7f-4bbd-b361-79dd50b5878c (/v1/_internal_ping) 2020-03-31 09:39:34,453 [556] [DEBUG] [app] Ending request: urn:request:bd9b19b0-de7f-4bbd-b361-79dd50b5878c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:34,453 [556] [DEBUG] [app] Ending request: urn:request:bd9b19b0-de7f-4bbd-b361-79dd50b5878c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:34,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:39:34,454 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:34,456 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:39:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:39:34,456 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:34,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:34,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:34,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:34,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:34,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:34,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:34,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:34,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:34,487 [518] [DEBUG] [app] Starting request: urn:request:74394be6-7f06-459c-af0d-4376fab7024a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:34,487 [518] [DEBUG] [app] Starting request: urn:request:74394be6-7f06-459c-af0d-4376fab7024a (/_internal_ping) 2020-03-31 09:39:34,488 [518] [DEBUG] [app] Ending request: urn:request:74394be6-7f06-459c-af0d-4376fab7024a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:34,488 [518] [DEBUG] [app] Ending request: urn:request:74394be6-7f06-459c-af0d-4376fab7024a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:39:34,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:34,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:34,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:34,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:39:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:34,501 [537] [DEBUG] [app] Starting request: urn:request:4c6bc444-d720-4d48-aaa0-5adfdf225a7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:34,501 [537] [DEBUG] [app] Starting request: urn:request:4c6bc444-d720-4d48-aaa0-5adfdf225a7c (/c1/_internal_ping) 2020-03-31 09:39:34,502 [537] [DEBUG] [app] Ending request: urn:request:4c6bc444-d720-4d48-aaa0-5adfdf225a7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:34,502 [537] [DEBUG] [app] Ending request: urn:request:4c6bc444-d720-4d48-aaa0-5adfdf225a7c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:34,503 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:39:34,503 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:34,505 [518] [DEBUG] [app] Ending request: urn:request:0e966bcb-e5e7-45c6-9a40-ab0ed2ba5fc4 (/health/instance) 2020-03-31 09:39:34,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:34,505 [518] [DEBUG] [app] Ending request: urn:request:0e966bcb-e5e7-45c6-9a40-ab0ed2ba5fc4 (/health/instance) 2020-03-31 09:39:34,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:39:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:39:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:38.010934+00:00 (in 1.085568 seconds) 2020-03-31 09:39:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:46 UTC)" (scheduled at 2020-03-31 09:39:36.924497+00:00) 2020-03-31 09:39:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:39:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:39:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:38.010934+00:00 (in 1.085568 seconds) 2020-03-31 09:39:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:46 UTC)" (scheduled at 2020-03-31 09:39:36.924497+00:00) 2020-03-31 09:39:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 36, 926078), True, datetime.datetime(2020, 3, 31, 9, 39, 36, 926078), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:39:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 36, 926078), True, datetime.datetime(2020, 3, 31, 9, 39, 36, 926078), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:39:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:39:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:39:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:46 UTC)" executed successfully 2020-03-31 09:39:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:39:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:46.924497+00:00 (in 8.912731 seconds) 2020-03-31 09:39:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:38 UTC)" (scheduled at 2020-03-31 09:39:38.010934+00:00) 2020-03-31 09:39:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:39:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:46.924497+00:00 (in 8.912731 seconds) 2020-03-31 09:39:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:38 UTC)" (scheduled at 2020-03-31 09:39:38.010934+00:00) 2020-03-31 09:39:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:38 UTC)" executed successfully 2020-03-31 09:39:38,273 [526] [DEBUG] [app] Starting request: urn:request:85e1e2b0-dbe2-42d3-83aa-35acd3368901 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:38,273 [526] [DEBUG] [app] Starting request: urn:request:85e1e2b0-dbe2-42d3-83aa-35acd3368901 (/health/instance) 2020-03-31 09:39:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:38,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:38,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:38,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:38,280 [538] [DEBUG] [app] Starting request: urn:request:24c56a9f-cf49-4e58-b1cb-157a53a8f1a9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:38,280 [538] [DEBUG] [app] Starting request: urn:request:24c56a9f-cf49-4e58-b1cb-157a53a8f1a9 (/v1/_internal_ping) 2020-03-31 09:39:38,281 [538] [DEBUG] [app] Ending request: urn:request:24c56a9f-cf49-4e58-b1cb-157a53a8f1a9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:38,281 [538] [DEBUG] [app] Ending request: urn:request:24c56a9f-cf49-4e58-b1cb-157a53a8f1a9 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:38,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:38,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:38,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:39:38,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:38,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:38,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:38,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:38,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:38,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:38,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:38,292 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:38,317 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:38,317 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:38,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:38,322 [526] [DEBUG] [app] Starting request: urn:request:7d7a384f-3b2c-4cbe-aa30-e60d58d35760 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:38,322 [526] [DEBUG] [app] Starting request: urn:request:7d7a384f-3b2c-4cbe-aa30-e60d58d35760 (/_internal_ping) 2020-03-31 09:39:38,323 [526] [DEBUG] [app] Ending request: urn:request:7d7a384f-3b2c-4cbe-aa30-e60d58d35760 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:38,323 [526] [DEBUG] [app] Ending request: urn:request:7d7a384f-3b2c-4cbe-aa30-e60d58d35760 (/_internal_ping) 2020-03-31 09:39:38,324 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:38,324 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:38,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:38,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:38,328 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:38,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:38,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:38,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:38,332 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:39:38,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:38,333 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:38,337 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:38,337 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:38,339 [541] [DEBUG] [app] Starting request: urn:request:0478c159-19e8-4b4a-8a9b-b81781c00e0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:38,339 [541] [DEBUG] [app] Starting request: urn:request:0478c159-19e8-4b4a-8a9b-b81781c00e0b (/c1/_internal_ping) 2020-03-31 09:39:38,340 [541] [DEBUG] [app] Ending request: urn:request:0478c159-19e8-4b4a-8a9b-b81781c00e0b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:38,340 [541] [DEBUG] [app] Ending request: urn:request:0478c159-19e8-4b4a-8a9b-b81781c00e0b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:38,340 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:38,340 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:38,340 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:39:38,340 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:38,342 [526] [DEBUG] [app] Ending request: urn:request:85e1e2b0-dbe2-42d3-83aa-35acd3368901 (/health/instance) 2020-03-31 09:39:38,342 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:38,342 [526] [DEBUG] [app] Ending request: urn:request:85e1e2b0-dbe2-42d3-83aa-35acd3368901 (/health/instance) 2020-03-31 09:39:38,342 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:38,343 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 09:39:38,343 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:39:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:39:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:51.440707+00:00 (in 9.999163 seconds) 2020-03-31 09:39:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:51 UTC)" (scheduled at 2020-03-31 09:39:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:39:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:51.440707+00:00 (in 9.999163 seconds) 2020-03-31 09:39:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:51 UTC)" (scheduled at 2020-03-31 09:39:41.440707+00:00) 2020-03-31 09:39:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:39:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:39:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:39:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:39:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:39:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:51 UTC)" executed successfully 2020-03-31 09:39:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:39:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:43 UTC)" (scheduled at 2020-03-31 09:39:43.539999+00:00) 2020-03-31 09:39:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:39:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:39:43 UTC)" (scheduled at 2020-03-31 09:39:43.539999+00:00) 2020-03-31 09:39:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:14.539408+00:00 (in 30.998381 seconds) 2020-03-31 09:39:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:39:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:14.539408+00:00 (in 30.998381 seconds) 2020-03-31 09:39:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:43 UTC)" executed successfully 2020-03-31 09:39:44,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:39:44,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:44,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:49.498966+00:00 (in 5.170908 seconds) 2020-03-31 09:39:44,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:54 UTC)" (scheduled at 2020-03-31 09:39:44.323352+00:00) 2020-03-31 09:39:44,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:39:44,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:49.498966+00:00 (in 5.170908 seconds) 2020-03-31 09:39:44,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:54 UTC)" (scheduled at 2020-03-31 09:39:44.323352+00:00) 2020-03-31 09:39:44,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:44,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 44, 328913), True, datetime.datetime(2020, 3, 31, 9, 39, 44, 328913), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:39:44,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 44, 328913), True, datetime.datetime(2020, 3, 31, 9, 39, 44, 328913), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:39:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:39:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:39:44,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:44,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:54 UTC)" executed successfully 2020-03-31 09:39:44,446 [518] [DEBUG] [app] Starting request: urn:request:da7c176e-e3c4-4b31-9812-1b9c734ac4e2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:44,446 [518] [DEBUG] [app] Starting request: urn:request:da7c176e-e3c4-4b31-9812-1b9c734ac4e2 (/health/instance) 2020-03-31 09:39:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:44,455 [538] [DEBUG] [app] Starting request: urn:request:e5032389-4e52-4598-be61-fc537eec9180 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:44,455 [538] [DEBUG] [app] Starting request: urn:request:e5032389-4e52-4598-be61-fc537eec9180 (/v1/_internal_ping) 2020-03-31 09:39:44,456 [538] [DEBUG] [app] Ending request: urn:request:e5032389-4e52-4598-be61-fc537eec9180 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:44,456 [538] [DEBUG] [app] Ending request: urn:request:e5032389-4e52-4598-be61-fc537eec9180 (/v1/_internal_ping) 2020-03-31 09:39:44,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:39:44,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:39:44,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:39:44,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:44,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:44,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:44,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:44,461 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:44,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:44,462 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:44,477 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:44,477 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:44,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:44,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:44,502 [526] [DEBUG] [app] Starting request: urn:request:cf0c37b8-e8b5-401c-86bd-83eaf4d9e7f9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:44,502 [526] [DEBUG] [app] Starting request: urn:request:cf0c37b8-e8b5-401c-86bd-83eaf4d9e7f9 (/_internal_ping) 2020-03-31 09:39:44,502 [526] [DEBUG] [app] Ending request: urn:request:cf0c37b8-e8b5-401c-86bd-83eaf4d9e7f9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:44,502 [526] [DEBUG] [app] Ending request: urn:request:cf0c37b8-e8b5-401c-86bd-83eaf4d9e7f9 (/_internal_ping) 2020-03-31 09:39:44,503 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:39:44,503 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:39:44,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:44,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:44,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:39:44,507 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:39:44,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:44,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:39:44,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:44,511 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:44,514 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:44,514 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:44,515 [537] [DEBUG] [app] Starting request: urn:request:ff37bd1c-007f-4452-b8b7-b7cff2d0b8de (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:44,515 [537] [DEBUG] [app] Starting request: urn:request:ff37bd1c-007f-4452-b8b7-b7cff2d0b8de (/c1/_internal_ping) 2020-03-31 09:39:44,516 [537] [DEBUG] [app] Ending request: urn:request:ff37bd1c-007f-4452-b8b7-b7cff2d0b8de (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:44,516 [537] [DEBUG] [app] Ending request: urn:request:ff37bd1c-007f-4452-b8b7-b7cff2d0b8de (/c1/_internal_ping) 2020-03-31 09:39:44,517 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:44,517 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:44,517 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:39:44,518 [518] [DEBUG] [app] Ending request: urn:request:da7c176e-e3c4-4b31-9812-1b9c734ac4e2 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:39:44,517 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:44,518 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:44,519 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) gunicorn-web stdout | 2020-03-31 09:39:44,518 [518] [DEBUG] [app] Ending request: urn:request:da7c176e-e3c4-4b31-9812-1b9c734ac4e2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:44,518 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:44,519 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) 2020-03-31 09:39:45,064 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:39:45,064 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:15.064514+00:00 (in 29.999372 seconds) 2020-03-31 09:39:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:15 UTC)" (scheduled at 2020-03-31 09:39:45.064514+00:00) gcworker stdout | 2020-03-31 09:39:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:15.064514+00:00 (in 29.999372 seconds) 2020-03-31 09:39:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:15 UTC)" (scheduled at 2020-03-31 09:39:45.064514+00:00) 2020-03-31 09:39:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437985, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:39:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584437985, 1209600, 500, 1, 0]) 2020-03-31 09:39:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:39:45,083 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:39:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:39:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:39:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:15 UTC)" executed successfully 2020-03-31 09:39:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:39:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:56.924497+00:00 (in 9.999183 seconds) 2020-03-31 09:39:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:56 UTC)" (scheduled at 2020-03-31 09:39:46.924497+00:00) storagereplication stdout | 2020-03-31 09:39:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:56.924497+00:00 (in 9.999183 seconds) 2020-03-31 09:39:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:56 UTC)" (scheduled at 2020-03-31 09:39:46.924497+00:00) 2020-03-31 09:39:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:39:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 46, 926665), True, datetime.datetime(2020, 3, 31, 9, 39, 46, 926665), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:39:46,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 46, 926665), True, datetime.datetime(2020, 3, 31, 9, 39, 46, 926665), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:39:46,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:39:46,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:39:46,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:39:56 UTC)" executed successfully 2020-03-31 09:39:48,273 [518] [DEBUG] [app] Starting request: urn:request:3f3f1ed8-c8b9-4ed1-a939-5797433e290a (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:48,273 [518] [DEBUG] [app] Starting request: urn:request:3f3f1ed8-c8b9-4ed1-a939-5797433e290a (/health/instance) 2020-03-31 09:39:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:48,278 [551] [DEBUG] [app] Starting request: urn:request:6d6b3ee2-d476-48d9-8f67-bddcaccd4192 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:48,278 [551] [DEBUG] [app] Starting request: urn:request:6d6b3ee2-d476-48d9-8f67-bddcaccd4192 (/v1/_internal_ping) 2020-03-31 09:39:48,279 [551] [DEBUG] [app] Ending request: urn:request:6d6b3ee2-d476-48d9-8f67-bddcaccd4192 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:48,279 [551] [DEBUG] [app] Ending request: urn:request:6d6b3ee2-d476-48d9-8f67-bddcaccd4192 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:48,279 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:48,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:48,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:39:48,279 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:48,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:48,282 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:39:48,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:39:48,282 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:48,283 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:48,284 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:48,283 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:48,284 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:48,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:48,311 [518] [DEBUG] [app] Starting request: urn:request:2b3c4690-ed10-4ded-8d62-1bc756006a3b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:48,311 [518] [DEBUG] [app] Starting request: urn:request:2b3c4690-ed10-4ded-8d62-1bc756006a3b (/_internal_ping) 2020-03-31 09:39:48,312 [518] [DEBUG] [app] Ending request: urn:request:2b3c4690-ed10-4ded-8d62-1bc756006a3b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:48,312 [518] [DEBUG] [app] Ending request: urn:request:2b3c4690-ed10-4ded-8d62-1bc756006a3b (/_internal_ping) 2020-03-31 09:39:48,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:48,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:39:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:39:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:48,325 [534] [DEBUG] [app] Starting request: urn:request:f4e9cf32-c1d7-497a-858f-8142d0477694 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:48,325 [534] [DEBUG] [app] Starting request: urn:request:f4e9cf32-c1d7-497a-858f-8142d0477694 (/c1/_internal_ping) 2020-03-31 09:39:48,326 [534] [DEBUG] [app] Ending request: urn:request:f4e9cf32-c1d7-497a-858f-8142d0477694 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:48,326 [534] [DEBUG] [app] Ending request: urn:request:f4e9cf32-c1d7-497a-858f-8142d0477694 (/c1/_internal_ping) 2020-03-31 09:39:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:48,326 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:48,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:39:48,326 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:48,328 [518] [DEBUG] [app] Ending request: urn:request:3f3f1ed8-c8b9-4ed1-a939-5797433e290a (/health/instance) 2020-03-31 09:39:48,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:48,328 [518] [DEBUG] [app] Ending request: urn:request:3f3f1ed8-c8b9-4ed1-a939-5797433e290a (/health/instance) 2020-03-31 09:39:48,328 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:39:48,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:39:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:39:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:49 UTC)" (scheduled at 2020-03-31 09:39:49.498966+00:00) 2020-03-31 09:39:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:49 UTC)" executed successfully 2020-03-31 09:39:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:54.323352+00:00 (in 4.823534 seconds) notificationworker stdout | 2020-03-31 09:39:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:49 UTC)" (scheduled at 2020-03-31 09:39:49.498966+00:00) 2020-03-31 09:39:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:49 UTC)" executed successfully 2020-03-31 09:39:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:54.323352+00:00 (in 4.823534 seconds) 2020-03-31 09:39:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:39:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:53.556282+00:00 (in 2.444761 seconds) 2020-03-31 09:39:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:51 UTC)" (scheduled at 2020-03-31 09:39:51.110610+00:00) 2020-03-31 09:39:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:39:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:53.556282+00:00 (in 2.444761 seconds) 2020-03-31 09:39:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:51 UTC)" (scheduled at 2020-03-31 09:39:51.110610+00:00) 2020-03-31 09:39:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 51, 112484), True, datetime.datetime(2020, 3, 31, 9, 39, 51, 112484), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:39:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 51, 112484), True, datetime.datetime(2020, 3, 31, 9, 39, 51, 112484), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:39:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:39:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:39:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:51 UTC)" executed successfully 2020-03-31 09:39:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:39:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:01.440707+00:00 (in 9.996284 seconds) 2020-03-31 09:39:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:01 UTC)" (scheduled at 2020-03-31 09:39:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:39:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:01.440707+00:00 (in 9.996284 seconds) 2020-03-31 09:39:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:01 UTC)" (scheduled at 2020-03-31 09:39:51.440707+00:00) 2020-03-31 09:39:51,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:39:51,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:39:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:39:51,464 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:39:51,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:39:51,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:39:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:01 UTC)" executed successfully 2020-03-31 09:39:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:39:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:22.113221+00:00 (in 29.997572 seconds) 2020-03-31 09:39:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:22 UTC)" (scheduled at 2020-03-31 09:39:52.113221+00:00) securityworker stdout | 2020-03-31 09:39:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:22.113221+00:00 (in 29.997572 seconds) 2020-03-31 09:39:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:22 UTC)" (scheduled at 2020-03-31 09:39:52.113221+00:00) 2020-03-31 09:39:52,118 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:39:52,118 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:39:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:39:52,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:39:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:39:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:39:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:39:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:22 UTC)" executed successfully 2020-03-31 09:39:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:39:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:59.560326+00:00 (in 6.003266 seconds) 2020-03-31 09:39:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:44:53 UTC)" (scheduled at 2020-03-31 09:39:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:39:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:39:59.560326+00:00 (in 6.003266 seconds) 2020-03-31 09:39:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:44:53 UTC)" (scheduled at 2020-03-31 09:39:53.556282+00:00) 2020-03-31 09:39:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:39:53,558 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), u'namespacegc/%']) 2020-03-31 09:39:53,572 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), True, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:39:53,572 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), True, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), 0, u'namespacegc/%']) 2020-03-31 09:39:53,576 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), True, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:39:53,576 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), True, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 39, 53, 557894), u'namespacegc/%']) 2020-03-31 09:39:53,580 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:39:53,580 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:53,581 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:44:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:39:53,581 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:44:53 UTC)" executed successfully 2020-03-31 09:39:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:39:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:53,584 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:39:53,584 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:53,585 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:39:53,585 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:39:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:39:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:04.323352+00:00 (in 9.998437 seconds) 2020-03-31 09:39:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:04 UTC)" (scheduled at 2020-03-31 09:39:54.323352+00:00) 2020-03-31 09:39:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:39:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:04.323352+00:00 (in 9.998437 seconds) 2020-03-31 09:39:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:04 UTC)" (scheduled at 2020-03-31 09:39:54.323352+00:00) 2020-03-31 09:39:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 54, 325882), True, datetime.datetime(2020, 3, 31, 9, 39, 54, 325882), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:39:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 54, 325882), True, datetime.datetime(2020, 3, 31, 9, 39, 54, 325882), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:39:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:39:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:39:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:54,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:04 UTC)" executed successfully 2020-03-31 09:39:54,446 [518] [DEBUG] [app] Starting request: urn:request:02c0ed2e-21c4-45eb-bc51-46a11032b0c6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:54,446 [518] [DEBUG] [app] Starting request: urn:request:02c0ed2e-21c4-45eb-bc51-46a11032b0c6 (/health/instance) 2020-03-31 09:39:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:54,455 [547] [DEBUG] [app] Starting request: urn:request:74573bb0-eba9-4a30-8ddf-a2e318d3db23 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:54,455 [547] [DEBUG] [app] Starting request: urn:request:74573bb0-eba9-4a30-8ddf-a2e318d3db23 (/v1/_internal_ping) 2020-03-31 09:39:54,456 [547] [DEBUG] [app] Ending request: urn:request:74573bb0-eba9-4a30-8ddf-a2e318d3db23 (/v1/_internal_ping) 2020-03-31 09:39:54,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:39:54,456 [547] [DEBUG] [app] Ending request: urn:request:74573bb0-eba9-4a30-8ddf-a2e318d3db23 (/v1/_internal_ping) 2020-03-31 09:39:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:39:54,456 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:39:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:54,459 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:54,461 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:54,459 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:54,461 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:54,462 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:54,462 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:54,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:54,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:54,487 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:54,492 [518] [DEBUG] [app] Starting request: urn:request:4e020262-91e1-4623-9d7f-83883eac5b56 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:54,492 [518] [DEBUG] [app] Starting request: urn:request:4e020262-91e1-4623-9d7f-83883eac5b56 (/_internal_ping) 2020-03-31 09:39:54,493 [518] [DEBUG] [app] Ending request: urn:request:4e020262-91e1-4623-9d7f-83883eac5b56 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:54,493 [518] [DEBUG] [app] Ending request: urn:request:4e020262-91e1-4623-9d7f-83883eac5b56 (/_internal_ping) 2020-03-31 09:39:54,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:54,494 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:39:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:39:54,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:54,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:39:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:54,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:54,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:54,508 [534] [DEBUG] [app] Starting request: urn:request:75193db5-9855-457e-9f4d-85bce27a2a90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:54,508 [534] [DEBUG] [app] Starting request: urn:request:75193db5-9855-457e-9f4d-85bce27a2a90 (/c1/_internal_ping) 2020-03-31 09:39:54,509 [534] [DEBUG] [app] Ending request: urn:request:75193db5-9855-457e-9f4d-85bce27a2a90 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:54,509 [534] [DEBUG] [app] Ending request: urn:request:75193db5-9855-457e-9f4d-85bce27a2a90 (/c1/_internal_ping) 2020-03-31 09:39:54,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:54,510 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:54,510 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:39:54,510 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:54,512 [518] [DEBUG] [app] Ending request: urn:request:02c0ed2e-21c4-45eb-bc51-46a11032b0c6 (/health/instance) 2020-03-31 09:39:54,512 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:54,512 [518] [DEBUG] [app] Ending request: urn:request:02c0ed2e-21c4-45eb-bc51-46a11032b0c6 (/health/instance) 2020-03-31 09:39:54,512 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:54,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) gunicorn-web stdout | 2020-03-31 09:39:54,513 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.068 49 0.068) 2020-03-31 09:39:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:39:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:06.924497+00:00 (in 9.999216 seconds) 2020-03-31 09:39:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:06 UTC)" (scheduled at 2020-03-31 09:39:56.924497+00:00) 2020-03-31 09:39:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:39:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:06.924497+00:00 (in 9.999216 seconds) 2020-03-31 09:39:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:06 UTC)" (scheduled at 2020-03-31 09:39:56.924497+00:00) 2020-03-31 09:39:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:39:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 56, 926135), True, datetime.datetime(2020, 3, 31, 9, 39, 56, 926135), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:39:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 39, 56, 926135), True, datetime.datetime(2020, 3, 31, 9, 39, 56, 926135), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:39:57,081 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:39:57,081 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:39:57,081 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:57,082 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:39:57,081 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:57,082 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:06 UTC)" executed successfully 2020-03-31 09:39:58,272 [518] [DEBUG] [app] Starting request: urn:request:cf2119cb-962a-46eb-852a-32917afaa6ea (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:58,272 [518] [DEBUG] [app] Starting request: urn:request:cf2119cb-962a-46eb-852a-32917afaa6ea (/health/instance) 2020-03-31 09:39:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:58,278 [538] [DEBUG] [app] Starting request: urn:request:0bb470cf-44f3-40d1-be3e-89a0d2eff337 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:58,278 [538] [DEBUG] [app] Starting request: urn:request:0bb470cf-44f3-40d1-be3e-89a0d2eff337 (/v1/_internal_ping) 2020-03-31 09:39:58,280 [538] [DEBUG] [app] Ending request: urn:request:0bb470cf-44f3-40d1-be3e-89a0d2eff337 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:39:58,280 [538] [DEBUG] [app] Ending request: urn:request:0bb470cf-44f3-40d1-be3e-89a0d2eff337 (/v1/_internal_ping) 2020-03-31 09:39:58,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:39:58,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:39:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:39:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:39:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:39:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:39:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:39:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:39:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:39:58,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:39:58,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:39:58,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:39:58,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:39:58,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:58,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:58,312 [527] [DEBUG] [app] Starting request: urn:request:022f8498-790b-432c-abe1-669405be176f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:58,312 [527] [DEBUG] [app] Starting request: urn:request:022f8498-790b-432c-abe1-669405be176f (/_internal_ping) 2020-03-31 09:39:58,313 [527] [DEBUG] [app] Ending request: urn:request:022f8498-790b-432c-abe1-669405be176f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:58,313 [527] [DEBUG] [app] Ending request: urn:request:022f8498-790b-432c-abe1-669405be176f (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:39:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:39:58,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:39:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:39:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:39:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:39:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:39:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:39:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:39:58,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:39:58,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:39:58,326 [534] [DEBUG] [app] Starting request: urn:request:ed54b613-a49b-47e1-801c-40a3759171ee (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:39:58,326 [534] [DEBUG] [app] Starting request: urn:request:ed54b613-a49b-47e1-801c-40a3759171ee (/c1/_internal_ping) 2020-03-31 09:39:58,327 [534] [DEBUG] [app] Ending request: urn:request:ed54b613-a49b-47e1-801c-40a3759171ee (/c1/_internal_ping) 2020-03-31 09:39:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:39:58,328 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:39:58,329 [518] [DEBUG] [app] Ending request: urn:request:cf2119cb-962a-46eb-852a-32917afaa6ea (/health/instance) 2020-03-31 09:39:58,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:39:58,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-verbs stdout | 2020-03-31 09:39:58,327 [534] [DEBUG] [app] Ending request: urn:request:ed54b613-a49b-47e1-801c-40a3759171ee (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:39:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:39:58,328 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:39:58,329 [518] [DEBUG] [app] Ending request: urn:request:cf2119cb-962a-46eb-852a-32917afaa6ea (/health/instance) gunicorn-web stdout | 2020-03-31 09:39:58,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:39:58,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:39:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:39:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:39:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:39:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:39:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:51.110610+00:00 (in 51.549324 seconds) 2020-03-31 09:39:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:59 UTC)" (scheduled at 2020-03-31 09:39:59.560326+00:00) 2020-03-31 09:39:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:39:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:51.110610+00:00 (in 51.549324 seconds) 2020-03-31 09:39:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:59 UTC)" (scheduled at 2020-03-31 09:39:59.560326+00:00) 2020-03-31 09:39:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:39:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:40:59 UTC)" executed successfully 2020-03-31 09:40:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:40:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:30.337699+00:00 (in 29.999119 seconds) 2020-03-31 09:40:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:30 UTC)" (scheduled at 2020-03-31 09:40:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:40:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:30.337699+00:00 (in 29.999119 seconds) 2020-03-31 09:40:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:30 UTC)" (scheduled at 2020-03-31 09:40:00.337699+00:00) 2020-03-31 09:40:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 40, 0, 339424), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:40:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 40, 0, 339424), False, 50, 1, 0]) 2020-03-31 09:40:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:40:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:40:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:40:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:30 UTC)" executed successfully 2020-03-31 09:40:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:40:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:11.440707+00:00 (in 9.999105 seconds) 2020-03-31 09:40:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:11 UTC)" (scheduled at 2020-03-31 09:40:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:40:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:11.440707+00:00 (in 9.999105 seconds) 2020-03-31 09:40:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:11 UTC)" (scheduled at 2020-03-31 09:40:01.440707+00:00) 2020-03-31 09:40:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:40:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:40:01,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:40:01,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:40:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:40:01,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:11 UTC)" executed successfully 2020-03-31 09:40:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:40:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:24.410927+00:00 (in 21.079948 seconds) 2020-03-31 09:40:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:33 UTC)" (scheduled at 2020-03-31 09:40:03.330112+00:00) 2020-03-31 09:40:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:40:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:24.410927+00:00 (in 21.079948 seconds) 2020-03-31 09:40:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:33 UTC)" (scheduled at 2020-03-31 09:40:03.330112+00:00) 2020-03-31 09:40:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 3, 331668), True, datetime.datetime(2020, 3, 31, 9, 40, 3, 331668), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:40:03,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 3, 331668), True, datetime.datetime(2020, 3, 31, 9, 40, 3, 331668), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:40:03,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:40:03,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:03,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:33 UTC)" executed successfully 2020-03-31 09:40:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:40:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:14.323352+00:00 (in 9.999112 seconds) 2020-03-31 09:40:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:14 UTC)" (scheduled at 2020-03-31 09:40:04.323352+00:00) 2020-03-31 09:40:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:40:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:14.323352+00:00 (in 9.999112 seconds) 2020-03-31 09:40:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:14 UTC)" (scheduled at 2020-03-31 09:40:04.323352+00:00) 2020-03-31 09:40:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 4, 325095), True, datetime.datetime(2020, 3, 31, 9, 40, 4, 325095), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:40:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 4, 325095), True, datetime.datetime(2020, 3, 31, 9, 40, 4, 325095), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:40:04,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:40:04,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:40:04,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:14 UTC)" executed successfully 2020-03-31 09:40:04,446 [518] [DEBUG] [app] Starting request: urn:request:fc0dd9a4-3128-4f8a-89e8-4a15d36e2ac4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:04,446 [518] [DEBUG] [app] Starting request: urn:request:fc0dd9a4-3128-4f8a-89e8-4a15d36e2ac4 (/health/instance) 2020-03-31 09:40:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:04,454 [538] [DEBUG] [app] Starting request: urn:request:3db729d0-ac11-4d4b-b5c5-c44c3bf2b4f2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:04,454 [538] [DEBUG] [app] Starting request: urn:request:3db729d0-ac11-4d4b-b5c5-c44c3bf2b4f2 (/v1/_internal_ping) 2020-03-31 09:40:04,455 [538] [DEBUG] [app] Ending request: urn:request:3db729d0-ac11-4d4b-b5c5-c44c3bf2b4f2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:04,455 [538] [DEBUG] [app] Ending request: urn:request:3db729d0-ac11-4d4b-b5c5-c44c3bf2b4f2 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:04,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:40:04,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:40:04,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:40:04,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:04,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:04,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:04,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:04,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:04,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:04,487 [518] [DEBUG] [app] Starting request: urn:request:9ece51bb-925e-4404-8397-13a09251ab23 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:04,487 [518] [DEBUG] [app] Starting request: urn:request:9ece51bb-925e-4404-8397-13a09251ab23 (/_internal_ping) 2020-03-31 09:40:04,487 [518] [DEBUG] [app] Ending request: urn:request:9ece51bb-925e-4404-8397-13a09251ab23 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:04,487 [518] [DEBUG] [app] Ending request: urn:request:9ece51bb-925e-4404-8397-13a09251ab23 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:40:04,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:04,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:40:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:40:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:04,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:04,501 [534] [DEBUG] [app] Starting request: urn:request:d58ab6db-1ffc-49cf-a396-943354bf283c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:04,501 [534] [DEBUG] [app] Starting request: urn:request:d58ab6db-1ffc-49cf-a396-943354bf283c (/c1/_internal_ping) 2020-03-31 09:40:04,502 [534] [DEBUG] [app] Ending request: urn:request:d58ab6db-1ffc-49cf-a396-943354bf283c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:04,502 [534] [DEBUG] [app] Ending request: urn:request:d58ab6db-1ffc-49cf-a396-943354bf283c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:04,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:40:04,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:04,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:04,504 [518] [DEBUG] [app] Ending request: urn:request:fc0dd9a4-3128-4f8a-89e8-4a15d36e2ac4 (/health/instance) 2020-03-31 09:40:04,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:04,504 [518] [DEBUG] [app] Ending request: urn:request:fc0dd9a4-3128-4f8a-89e8-4a15d36e2ac4 (/health/instance) 2020-03-31 09:40:04,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:40:04,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:40:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:40:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:16.924497+00:00 (in 9.999162 seconds) storagereplication stdout | 2020-03-31 09:40:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:16.924497+00:00 (in 9.999162 seconds) 2020-03-31 09:40:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:16 UTC)" (scheduled at 2020-03-31 09:40:06.924497+00:00) storagereplication stdout | 2020-03-31 09:40:06,927 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:16 UTC)" (scheduled at 2020-03-31 09:40:06.924497+00:00) 2020-03-31 09:40:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:40:06,927 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 6, 927426), True, datetime.datetime(2020, 3, 31, 9, 40, 6, 927426), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:40:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 6, 927426), True, datetime.datetime(2020, 3, 31, 9, 40, 6, 927426), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:40:06,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:40:06,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:40:06,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:16 UTC)" executed successfully 2020-03-31 09:40:08,273 [527] [DEBUG] [app] Starting request: urn:request:49e12bc4-ae19-425d-9969-b51c988f9d01 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:08,273 [527] [DEBUG] [app] Starting request: urn:request:49e12bc4-ae19-425d-9969-b51c988f9d01 (/health/instance) 2020-03-31 09:40:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:08,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:08,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:08,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:08,283 [540] [DEBUG] [app] Starting request: urn:request:4969641f-4805-413f-a84b-d048eee93f57 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:08,283 [540] [DEBUG] [app] Starting request: urn:request:4969641f-4805-413f-a84b-d048eee93f57 (/v1/_internal_ping) 2020-03-31 09:40:08,284 [540] [DEBUG] [app] Ending request: urn:request:4969641f-4805-413f-a84b-d048eee93f57 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:08,284 [540] [DEBUG] [app] Ending request: urn:request:4969641f-4805-413f-a84b-d048eee93f57 (/v1/_internal_ping) 2020-03-31 09:40:08,285 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:08,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:40:08,285 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:08,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:08,285 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:08,287 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:40:08,286 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:40:08,288 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:08,287 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:08,289 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:08,288 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:08,289 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:08,314 [526] [DEBUG] [app] Starting request: urn:request:40a703bb-70cd-45b5-b8b4-03c2a99855e2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:08,314 [526] [DEBUG] [app] Starting request: urn:request:40a703bb-70cd-45b5-b8b4-03c2a99855e2 (/_internal_ping) 2020-03-31 09:40:08,315 [526] [DEBUG] [app] Ending request: urn:request:40a703bb-70cd-45b5-b8b4-03c2a99855e2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:08,315 [526] [DEBUG] [app] Ending request: urn:request:40a703bb-70cd-45b5-b8b4-03c2a99855e2 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:40:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:08,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:08,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:08,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:08,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:40:08,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:08,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:08,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:08,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:08,328 [537] [DEBUG] [app] Starting request: urn:request:ce37face-ff8a-4478-b72d-0f3c8e92bd95 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:08,328 [537] [DEBUG] [app] Starting request: urn:request:ce37face-ff8a-4478-b72d-0f3c8e92bd95 (/c1/_internal_ping) 2020-03-31 09:40:08,329 [537] [DEBUG] [app] Ending request: urn:request:ce37face-ff8a-4478-b72d-0f3c8e92bd95 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:08,329 [537] [DEBUG] [app] Ending request: urn:request:ce37face-ff8a-4478-b72d-0f3c8e92bd95 (/c1/_internal_ping) 2020-03-31 09:40:08,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:40:08,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:08,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-verbs stdout | 2020-03-31 09:40:08,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:08,334 [527] [DEBUG] [app] Ending request: urn:request:49e12bc4-ae19-425d-9969-b51c988f9d01 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:08,334 [527] [DEBUG] [app] Ending request: urn:request:49e12bc4-ae19-425d-9969-b51c988f9d01 (/health/instance) 2020-03-31 09:40:08,335 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:08,335 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:08,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:40:08,335 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:40:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:40:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:21.440707+00:00 (in 9.998861 seconds) 2020-03-31 09:40:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:21 UTC)" (scheduled at 2020-03-31 09:40:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:40:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:21.440707+00:00 (in 9.998861 seconds) 2020-03-31 09:40:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:21 UTC)" (scheduled at 2020-03-31 09:40:11.440707+00:00) 2020-03-31 09:40:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:40:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:40:11,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:40:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:40:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:40:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:21 UTC)" executed successfully 2020-03-31 09:40:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:40:14,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:24.323352+00:00 (in 9.997786 seconds) 2020-03-31 09:40:14,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:24 UTC)" (scheduled at 2020-03-31 09:40:14.323352+00:00) notificationworker stdout | 2020-03-31 09:40:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:24.323352+00:00 (in 9.997786 seconds) 2020-03-31 09:40:14,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:24 UTC)" (scheduled at 2020-03-31 09:40:14.323352+00:00) 2020-03-31 09:40:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:40:14,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 14, 326577), True, datetime.datetime(2020, 3, 31, 9, 40, 14, 326577), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:40:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 14, 326577), True, datetime.datetime(2020, 3, 31, 9, 40, 14, 326577), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:40:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:40:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:40:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:24 UTC)" executed successfully 2020-03-31 09:40:14,446 [518] [DEBUG] [app] Starting request: urn:request:9a467ae9-7602-40ed-98ab-2ec947350545 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:14,446 [518] [DEBUG] [app] Starting request: urn:request:9a467ae9-7602-40ed-98ab-2ec947350545 (/health/instance) 2020-03-31 09:40:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:14,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:14,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:14,453 [550] [DEBUG] [app] Starting request: urn:request:85f39c75-ac41-4230-a361-d3084eb86ff7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:14,453 [550] [DEBUG] [app] Starting request: urn:request:85f39c75-ac41-4230-a361-d3084eb86ff7 (/v1/_internal_ping) 2020-03-31 09:40:14,454 [550] [DEBUG] [app] Ending request: urn:request:85f39c75-ac41-4230-a361-d3084eb86ff7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:14,454 [550] [DEBUG] [app] Ending request: urn:request:85f39c75-ac41-4230-a361-d3084eb86ff7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:40:14,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:40:14,457 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:40:14,457 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:14,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:14,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:14,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:14,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:14,465 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:14,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:14,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:14,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:14,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:14,486 [518] [DEBUG] [app] Starting request: urn:request:ace307c6-6199-49aa-92dd-f4d11f5dfa5b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:14,486 [518] [DEBUG] [app] Starting request: urn:request:ace307c6-6199-49aa-92dd-f4d11f5dfa5b (/_internal_ping) 2020-03-31 09:40:14,487 [518] [DEBUG] [app] Ending request: urn:request:ace307c6-6199-49aa-92dd-f4d11f5dfa5b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:14,487 [518] [DEBUG] [app] Ending request: urn:request:ace307c6-6199-49aa-92dd-f4d11f5dfa5b (/_internal_ping) 2020-03-31 09:40:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:40:14,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:40:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:40:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:14,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:14,503 [539] [DEBUG] [app] Starting request: urn:request:9ddfb88d-e9ee-407b-a447-1d3835fd91c5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:14,503 [539] [DEBUG] [app] Starting request: urn:request:9ddfb88d-e9ee-407b-a447-1d3835fd91c5 (/c1/_internal_ping) 2020-03-31 09:40:14,504 [539] [DEBUG] [app] Ending request: urn:request:9ddfb88d-e9ee-407b-a447-1d3835fd91c5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:14,504 [539] [DEBUG] [app] Ending request: urn:request:9ddfb88d-e9ee-407b-a447-1d3835fd91c5 (/c1/_internal_ping) 2020-03-31 09:40:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:14,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:14,507 [518] [DEBUG] [app] Ending request: urn:request:9a467ae9-7602-40ed-98ab-2ec947350545 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:40:14,505 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:14,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:40:14,507 [518] [DEBUG] [app] Ending request: urn:request:9a467ae9-7602-40ed-98ab-2ec947350545 (/health/instance) 10.131.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:40:14,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:14,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:40:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:40:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:33.366791+00:00 (in 18.825990 seconds) 2020-03-31 09:40:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:45:14 UTC)" (scheduled at 2020-03-31 09:40:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:40:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:33.366791+00:00 (in 18.825990 seconds) 2020-03-31 09:40:14,541 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:45:14 UTC)" (scheduled at 2020-03-31 09:40:14.539408+00:00) 2020-03-31 09:40:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:40:14,543 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), u'exportactionlogs/%']) 2020-03-31 09:40:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), True, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:40:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), True, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), 0, u'exportactionlogs/%']) 2020-03-31 09:40:14,564 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), True, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:40:14,564 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), True, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 40, 14, 541870), u'exportactionlogs/%']) 2020-03-31 09:40:14,568 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:40:14,568 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:14,568 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:45:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:40:14,568 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:45:14 UTC)" executed successfully 2020-03-31 09:40:14,571 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:40:14,571 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:14,572 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:40:14,572 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:14,573 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:40:14,573 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:40:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:40:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:45.064514+00:00 (in 29.999235 seconds) 2020-03-31 09:40:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:45 UTC)" (scheduled at 2020-03-31 09:40:15.064514+00:00) gcworker stdout | 2020-03-31 09:40:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:45.064514+00:00 (in 29.999235 seconds) 2020-03-31 09:40:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:45 UTC)" (scheduled at 2020-03-31 09:40:15.064514+00:00) 2020-03-31 09:40:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438015, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:40:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438015, 1209600, 500, 1, 0]) 2020-03-31 09:40:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:40:15,082 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:40:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:40:15,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:40:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:45 UTC)" executed successfully 2020-03-31 09:40:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:40:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:26.924497+00:00 (in 9.999251 seconds) storagereplication stdout | 2020-03-31 09:40:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:26.924497+00:00 (in 9.999251 seconds) 2020-03-31 09:40:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:26 UTC)" (scheduled at 2020-03-31 09:40:16.924497+00:00) 2020-03-31 09:40:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 16, 926640), True, datetime.datetime(2020, 3, 31, 9, 40, 16, 926640), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:40:16,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:26 UTC)" (scheduled at 2020-03-31 09:40:16.924497+00:00) 2020-03-31 09:40:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 16, 926640), True, datetime.datetime(2020, 3, 31, 9, 40, 16, 926640), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:40:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:40:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:40:16,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:26 UTC)" executed successfully 2020-03-31 09:40:18,272 [526] [DEBUG] [app] Starting request: urn:request:f8d243b2-b078-4854-a37e-f9a24abc8c85 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:18,272 [526] [DEBUG] [app] Starting request: urn:request:f8d243b2-b078-4854-a37e-f9a24abc8c85 (/health/instance) 2020-03-31 09:40:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:18,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:18,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:18,280 [550] [DEBUG] [app] Starting request: urn:request:3b7d7c94-a19b-4048-87e6-3a0ce1bbdc36 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:18,280 [550] [DEBUG] [app] Starting request: urn:request:3b7d7c94-a19b-4048-87e6-3a0ce1bbdc36 (/v1/_internal_ping) 2020-03-31 09:40:18,281 [550] [DEBUG] [app] Ending request: urn:request:3b7d7c94-a19b-4048-87e6-3a0ce1bbdc36 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:18,281 [550] [DEBUG] [app] Ending request: urn:request:3b7d7c94-a19b-4048-87e6-3a0ce1bbdc36 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:18,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:18,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:40:18,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:18,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:18,282 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:18,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:40:18,283 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:40:18,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:18,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:18,286 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:18,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:18,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:18,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:18,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:18,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:18,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:18,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:18,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:18,314 [518] [DEBUG] [app] Starting request: urn:request:f8300465-8c3f-4e0c-b695-56e556dcfae1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:18,314 [518] [DEBUG] [app] Starting request: urn:request:f8300465-8c3f-4e0c-b695-56e556dcfae1 (/_internal_ping) 2020-03-31 09:40:18,314 [518] [DEBUG] [app] Ending request: urn:request:f8300465-8c3f-4e0c-b695-56e556dcfae1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:18,314 [518] [DEBUG] [app] Ending request: urn:request:f8300465-8c3f-4e0c-b695-56e556dcfae1 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:40:18,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:18,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:18,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:18,315 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:18,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:18,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:18,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:40:18,319 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:40:18,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:18,330 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:18,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:18,331 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:18,334 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:18,334 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:18,336 [537] [DEBUG] [app] Starting request: urn:request:ab71d540-1554-4478-9b8e-68a2ddff6eb4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:18,336 [537] [DEBUG] [app] Starting request: urn:request:ab71d540-1554-4478-9b8e-68a2ddff6eb4 (/c1/_internal_ping) 2020-03-31 09:40:18,337 [537] [DEBUG] [app] Ending request: urn:request:ab71d540-1554-4478-9b8e-68a2ddff6eb4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:18,337 [537] [DEBUG] [app] Ending request: urn:request:ab71d540-1554-4478-9b8e-68a2ddff6eb4 (/c1/_internal_ping) 2020-03-31 09:40:18,338 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:18,338 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:18,338 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:40:18,338 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:18,339 [526] [DEBUG] [app] Ending request: urn:request:f8d243b2-b078-4854-a37e-f9a24abc8c85 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:18,339 [526] [DEBUG] [app] Ending request: urn:request:f8d243b2-b078-4854-a37e-f9a24abc8c85 (/health/instance) 2020-03-31 09:40:18,340 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:18,340 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:18,340 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) gunicorn-web stdout | 2020-03-31 09:40:18,340 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.069 49 0.069) 2020-03-31 09:40:21,449 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:21,450 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:31.440707+00:00 (in 9.990840 seconds) 2020-03-31 09:40:21,450 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:31 UTC)" (scheduled at 2020-03-31 09:40:21.440707+00:00) 2020-03-31 09:40:21,453 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:40:21,449 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:21,450 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:31.440707+00:00 (in 9.990840 seconds) 2020-03-31 09:40:21,450 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:31 UTC)" (scheduled at 2020-03-31 09:40:21.440707+00:00) 2020-03-31 09:40:21,453 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:40:21,517 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:40:21,517 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:21,517 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:40:21,517 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:40:21,517 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:21,517 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:31 UTC)" executed successfully 2020-03-31 09:40:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:40:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:52.113221+00:00 (in 29.998958 seconds) 2020-03-31 09:40:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:52 UTC)" (scheduled at 2020-03-31 09:40:22.113221+00:00) securityworker stdout | 2020-03-31 09:40:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:52.113221+00:00 (in 29.998958 seconds) 2020-03-31 09:40:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:52 UTC)" (scheduled at 2020-03-31 09:40:22.113221+00:00) 2020-03-31 09:40:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:40:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:40:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:40:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:40:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:40:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:40:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:40:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:40:52 UTC)" executed successfully 2020-03-31 09:40:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:34.323352+00:00 (in 9.995872 seconds) 2020-03-31 09:40:24,330 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:34 UTC)" (scheduled at 2020-03-31 09:40:24.323352+00:00) 2020-03-31 09:40:24,330 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:24,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 24, 330653), True, datetime.datetime(2020, 3, 31, 9, 40, 24, 330653), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:40:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:34.323352+00:00 (in 9.995872 seconds) 2020-03-31 09:40:24,330 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:34 UTC)" (scheduled at 2020-03-31 09:40:24.323352+00:00) 2020-03-31 09:40:24,330 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:24,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 24, 330653), True, datetime.datetime(2020, 3, 31, 9, 40, 24, 330653), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:40:24,391 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:24,391 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:24,391 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:40:24,391 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:24,391 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:24,391 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:34 UTC)" executed successfully 2020-03-31 09:40:24,414 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:24,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:33.330112+00:00 (in 8.915641 seconds) 2020-03-31 09:40:24,416 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:24 UTC)" (scheduled at 2020-03-31 09:40:24.410927+00:00) 2020-03-31 09:40:24,419 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:24,419 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:40:24,414 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:24,414 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:33.330112+00:00 (in 8.915641 seconds) 2020-03-31 09:40:24,416 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:24 UTC)" (scheduled at 2020-03-31 09:40:24.410927+00:00) 2020-03-31 09:40:24,419 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:24,419 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:24 UTC)" executed successfully 2020-03-31 09:40:24,491 [525] [DEBUG] [app] Starting request: urn:request:2d5c74af-70de-4722-89da-9d4ab9716e10 (/health/instance) 2020-03-31 09:40:24,494 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:24,491 [525] [DEBUG] [app] Starting request: urn:request:2d5c74af-70de-4722-89da-9d4ab9716e10 (/health/instance) 2020-03-31 09:40:24,494 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:24,502 [555] [DEBUG] [app] Starting request: urn:request:1e04cd30-4e2a-42b8-b0a6-d06b88e55580 (/v1/_internal_ping) 2020-03-31 09:40:24,500 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-registry stdout | 2020-03-31 09:40:24,502 [555] [DEBUG] [app] Starting request: urn:request:1e04cd30-4e2a-42b8-b0a6-d06b88e55580 (/v1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:24,500 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:24,508 [555] [DEBUG] [app] Ending request: urn:request:1e04cd30-4e2a-42b8-b0a6-d06b88e55580 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:24,508 [555] [DEBUG] [app] Ending request: urn:request:1e04cd30-4e2a-42b8-b0a6-d06b88e55580 (/v1/_internal_ping) 2020-03-31 09:40:24,513 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.012 162 0.012) gunicorn-web stdout | 2020-03-31 09:40:24,513 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:24,515 [525] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.012 162 0.012) gunicorn-web stdout | 2020-03-31 09:40:24,515 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:40:24,513 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:40:24,513 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:24,524 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:40:24,524 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:24,528 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:24,528 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:24,531 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:24,531 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:24,534 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:24,534 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:24,542 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:24,542 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:24,547 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:24,547 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:24,549 [519] [DEBUG] [app] Starting request: urn:request:cbd96e74-ebe1-45b5-80a1-66a28a652271 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:24,549 [519] [DEBUG] [app] Starting request: urn:request:cbd96e74-ebe1-45b5-80a1-66a28a652271 (/_internal_ping) 2020-03-31 09:40:24,551 [519] [DEBUG] [app] Ending request: urn:request:cbd96e74-ebe1-45b5-80a1-66a28a652271 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:24,551 [519] [DEBUG] [app] Ending request: urn:request:cbd96e74-ebe1-45b5-80a1-66a28a652271 (/_internal_ping) 2020-03-31 09:40:24,551 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:40:24,551 [519] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:24,552 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:40:24,552 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:24,556 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:24,557 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:40:24,556 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:24,557 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:24,560 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:24,560 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:24,561 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:40:24,561 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:24,715 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:24,715 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:24,717 [535] [DEBUG] [app] Starting request: urn:request:c069ea5b-c03a-460e-b201-9e9dee790d10 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:24,717 [535] [DEBUG] [app] Starting request: urn:request:c069ea5b-c03a-460e-b201-9e9dee790d10 (/c1/_internal_ping) 2020-03-31 09:40:24,718 [535] [DEBUG] [app] Ending request: urn:request:c069ea5b-c03a-460e-b201-9e9dee790d10 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:24,718 [535] [DEBUG] [app] Ending request: urn:request:c069ea5b-c03a-460e-b201-9e9dee790d10 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:24,719 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:24,720 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:40:24,719 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:24,720 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:24,722 [525] [DEBUG] [app] Ending request: urn:request:2d5c74af-70de-4722-89da-9d4ab9716e10 (/health/instance) 2020-03-31 09:40:24,722 [525] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:24,722 [525] [DEBUG] [app] Ending request: urn:request:2d5c74af-70de-4722-89da-9d4ab9716e10 (/health/instance) 2020-03-31 09:40:24,722 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:24,723 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.249 49 0.249) gunicorn-web stdout | 2020-03-31 09:40:24,723 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.249 49 0.249) 2020-03-31 09:40:26,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:40:26,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:36.924497+00:00 (in 9.999352 seconds) 2020-03-31 09:40:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:36 UTC)" (scheduled at 2020-03-31 09:40:26.924497+00:00) 2020-03-31 09:40:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:40:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:36.924497+00:00 (in 9.999352 seconds) 2020-03-31 09:40:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:36 UTC)" (scheduled at 2020-03-31 09:40:26.924497+00:00) 2020-03-31 09:40:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 26, 925873), True, datetime.datetime(2020, 3, 31, 9, 40, 26, 925873), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:40:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 26, 925873), True, datetime.datetime(2020, 3, 31, 9, 40, 26, 925873), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:40:26,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:40:26,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:40:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:26,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:36 UTC)" executed successfully 2020-03-31 09:40:28,275 [523] [DEBUG] [app] Starting request: urn:request:f49678be-1818-41ee-b3eb-6c2626c9c84e (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:28,275 [523] [DEBUG] [app] Starting request: urn:request:f49678be-1818-41ee-b3eb-6c2626c9c84e (/health/instance) 2020-03-31 09:40:28,278 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:28,278 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:28,300 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:28,300 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:28,304 [551] [DEBUG] [app] Starting request: urn:request:42750dae-3cc4-4947-8f1d-1401dcfa270b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:28,304 [551] [DEBUG] [app] Starting request: urn:request:42750dae-3cc4-4947-8f1d-1401dcfa270b (/v1/_internal_ping) 2020-03-31 09:40:28,307 [551] [DEBUG] [app] Ending request: urn:request:42750dae-3cc4-4947-8f1d-1401dcfa270b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:28,307 [551] [DEBUG] [app] Ending request: urn:request:42750dae-3cc4-4947-8f1d-1401dcfa270b (/v1/_internal_ping) 2020-03-31 09:40:28,310 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:40:28,310 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:28,312 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:28,313 [523] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.011 162 0.011) gunicorn-web stdout | 2020-03-31 09:40:28,312 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:28,313 [523] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.011 162 0.011) 2020-03-31 09:40:28,318 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:40:28,318 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:28,322 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:28,322 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:28,325 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:28,325 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:28,328 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:28,328 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:28,357 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:28,357 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:28,369 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:28,372 [526] [DEBUG] [app] Starting request: urn:request:c43ea299-2ae7-480e-897a-3c1332815bbb (/_internal_ping) 2020-03-31 09:40:28,375 [526] [DEBUG] [app] Ending request: urn:request:c43ea299-2ae7-480e-897a-3c1332815bbb (/_internal_ping) 2020-03-31 09:40:28,376 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 159 0.006) gunicorn-web stdout | 2020-03-31 09:40:28,369 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:28,372 [526] [DEBUG] [app] Starting request: urn:request:c43ea299-2ae7-480e-897a-3c1332815bbb (/_internal_ping) 2020-03-31 09:40:28,375 [526] [DEBUG] [app] Ending request: urn:request:c43ea299-2ae7-480e-897a-3c1332815bbb (/_internal_ping) 2020-03-31 09:40:28,377 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:28,376 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 159 0.006) gunicorn-web stdout | 2020-03-31 09:40:28,377 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:28,381 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:28,381 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:28,381 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:40:28,381 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:40:28,384 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:28,384 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:40:28,385 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:28,385 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:28,390 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:28,390 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:28,391 [539] [DEBUG] [app] Starting request: urn:request:5608df2b-2eb8-44e0-9476-325a29db3fc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:28,391 [539] [DEBUG] [app] Starting request: urn:request:5608df2b-2eb8-44e0-9476-325a29db3fc2 (/c1/_internal_ping) 2020-03-31 09:40:28,392 [539] [DEBUG] [app] Ending request: urn:request:5608df2b-2eb8-44e0-9476-325a29db3fc2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:28,392 [539] [DEBUG] [app] Ending request: urn:request:5608df2b-2eb8-44e0-9476-325a29db3fc2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:28,392 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:28,392 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:40:28,392 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:28,392 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:28,394 [523] [DEBUG] [app] Ending request: urn:request:f49678be-1818-41ee-b3eb-6c2626c9c84e (/health/instance) 2020-03-31 09:40:28,394 [523] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:28,394 [523] [DEBUG] [app] Ending request: urn:request:f49678be-1818-41ee-b3eb-6c2626c9c84e (/health/instance) 2020-03-31 09:40:28,394 [523] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:28,395 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.121 49 0.121) gunicorn-web stdout | 2020-03-31 09:40:28,395 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.121 49 0.121) 2020-03-31 09:40:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:40:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:00.337699+00:00 (in 29.998764 seconds) 2020-03-31 09:40:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:00 UTC)" (scheduled at 2020-03-31 09:40:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:40:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:00.337699+00:00 (in 29.998764 seconds) 2020-03-31 09:40:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:00 UTC)" (scheduled at 2020-03-31 09:40:30.337699+00:00) 2020-03-31 09:40:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 40, 30, 340159), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:40:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 40, 30, 340159), False, 50, 1, 0]) 2020-03-31 09:40:30,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:40:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:40:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:40:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:00 UTC)" executed successfully 2020-03-31 09:40:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:40:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:41.440707+00:00 (in 9.998871 seconds) 2020-03-31 09:40:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:41 UTC)" (scheduled at 2020-03-31 09:40:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:40:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:41.440707+00:00 (in 9.998871 seconds) 2020-03-31 09:40:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:41 UTC)" (scheduled at 2020-03-31 09:40:31.440707+00:00) 2020-03-31 09:40:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:40:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:40:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:40:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:40:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:40:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:41 UTC)" executed successfully 2020-03-31 09:40:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:40:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:03.330112+00:00 (in 29.999268 seconds) 2020-03-31 09:40:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:03 UTC)" (scheduled at 2020-03-31 09:40:33.330112+00:00) 2020-03-31 09:40:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:40:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:03.330112+00:00 (in 29.999268 seconds) 2020-03-31 09:40:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:03 UTC)" (scheduled at 2020-03-31 09:40:33.330112+00:00) 2020-03-31 09:40:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 33, 331690), True, datetime.datetime(2020, 3, 31, 9, 40, 33, 331690), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:40:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 33, 331690), True, datetime.datetime(2020, 3, 31, 9, 40, 33, 331690), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:40:33,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:40:33,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:40:33,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:33,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:03 UTC)" executed successfully 2020-03-31 09:40:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:40:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:33 UTC)" (scheduled at 2020-03-31 09:40:33.366791+00:00) 2020-03-31 09:40:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:40:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:33 UTC)" (scheduled at 2020-03-31 09:40:33.366791+00:00) 2020-03-31 09:40:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:43.539999+00:00 (in 10.172206 seconds) exportactionlogsworker stdout | 2020-03-31 09:40:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:43.539999+00:00 (in 10.172206 seconds) 2020-03-31 09:40:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 33, 368696), True, datetime.datetime(2020, 3, 31, 9, 40, 33, 368696), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:40:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 33, 368696), True, datetime.datetime(2020, 3, 31, 9, 40, 33, 368696), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:40:33,395 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:40:33,395 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:33,395 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:33,395 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:40:33,395 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:33,395 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:33 UTC)" executed successfully 2020-03-31 09:40:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:40:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:44.323352+00:00 (in 9.999081 seconds) 2020-03-31 09:40:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:44 UTC)" (scheduled at 2020-03-31 09:40:34.323352+00:00) 2020-03-31 09:40:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:40:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:44.323352+00:00 (in 9.999081 seconds) 2020-03-31 09:40:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:44 UTC)" (scheduled at 2020-03-31 09:40:34.323352+00:00) 2020-03-31 09:40:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 34, 325312), True, datetime.datetime(2020, 3, 31, 9, 40, 34, 325312), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:40:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 34, 325312), True, datetime.datetime(2020, 3, 31, 9, 40, 34, 325312), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:40:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:40:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:40:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:44 UTC)" executed successfully 2020-03-31 09:40:34,445 [518] [DEBUG] [app] Starting request: urn:request:cc35f9b9-8ef5-4fdb-a83f-9a35a5ac72f7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:34,445 [518] [DEBUG] [app] Starting request: urn:request:cc35f9b9-8ef5-4fdb-a83f-9a35a5ac72f7 (/health/instance) 2020-03-31 09:40:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:34,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:34,453 [542] [DEBUG] [app] Starting request: urn:request:ec9e4602-2734-4307-94a0-c4e1d34617c5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:34,453 [542] [DEBUG] [app] Starting request: urn:request:ec9e4602-2734-4307-94a0-c4e1d34617c5 (/v1/_internal_ping) 2020-03-31 09:40:34,454 [542] [DEBUG] [app] Ending request: urn:request:ec9e4602-2734-4307-94a0-c4e1d34617c5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:34,454 [542] [DEBUG] [app] Ending request: urn:request:ec9e4602-2734-4307-94a0-c4e1d34617c5 (/v1/_internal_ping) 2020-03-31 09:40:34,454 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:40:34,454 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:34,456 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:40:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:34,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:40:34,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:40:34,456 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:34,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:34,458 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:34,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:34,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:34,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:34,484 [518] [DEBUG] [app] Starting request: urn:request:6280e21d-cdae-4392-abe7-2beb980ee6a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:34,484 [518] [DEBUG] [app] Starting request: urn:request:6280e21d-cdae-4392-abe7-2beb980ee6a0 (/_internal_ping) 2020-03-31 09:40:34,485 [518] [DEBUG] [app] Ending request: urn:request:6280e21d-cdae-4392-abe7-2beb980ee6a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:34,485 [518] [DEBUG] [app] Ending request: urn:request:6280e21d-cdae-4392-abe7-2beb980ee6a0 (/_internal_ping) 2020-03-31 09:40:34,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:40:34,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:34,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:40:34,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:40:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:34,498 [534] [DEBUG] [app] Starting request: urn:request:d8dcc93d-9259-4bbb-8928-e968ee64a333 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:34,498 [534] [DEBUG] [app] Starting request: urn:request:d8dcc93d-9259-4bbb-8928-e968ee64a333 (/c1/_internal_ping) 2020-03-31 09:40:34,499 [534] [DEBUG] [app] Ending request: urn:request:d8dcc93d-9259-4bbb-8928-e968ee64a333 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:34,499 [534] [DEBUG] [app] Ending request: urn:request:d8dcc93d-9259-4bbb-8928-e968ee64a333 (/c1/_internal_ping) 2020-03-31 09:40:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:34,499 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:34,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:40:34,499 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:34,501 [518] [DEBUG] [app] Ending request: urn:request:cc35f9b9-8ef5-4fdb-a83f-9a35a5ac72f7 (/health/instance) 2020-03-31 09:40:34,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:34,501 [518] [DEBUG] [app] Ending request: urn:request:cc35f9b9-8ef5-4fdb-a83f-9a35a5ac72f7 (/health/instance) 2020-03-31 09:40:34,501 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:34,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:40:34,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:40:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:40:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:38.010934+00:00 (in 1.085675 seconds) 2020-03-31 09:40:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:46 UTC)" (scheduled at 2020-03-31 09:40:36.924497+00:00) 2020-03-31 09:40:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:40:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:38.010934+00:00 (in 1.085675 seconds) 2020-03-31 09:40:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:46 UTC)" (scheduled at 2020-03-31 09:40:36.924497+00:00) 2020-03-31 09:40:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 36, 925901), True, datetime.datetime(2020, 3, 31, 9, 40, 36, 925901), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:40:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 36, 925901), True, datetime.datetime(2020, 3, 31, 9, 40, 36, 925901), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:40:36,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:40:36,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:40:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:46 UTC)" executed successfully 2020-03-31 09:40:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:40:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:46.924497+00:00 (in 8.912855 seconds) 2020-03-31 09:40:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:38 UTC)" (scheduled at 2020-03-31 09:40:38.010934+00:00) 2020-03-31 09:40:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:40:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:46.924497+00:00 (in 8.912855 seconds) 2020-03-31 09:40:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:38 UTC)" (scheduled at 2020-03-31 09:40:38.010934+00:00) 2020-03-31 09:40:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:38 UTC)" executed successfully 2020-03-31 09:40:38,272 [527] [DEBUG] [app] Starting request: urn:request:8ce92e86-79e7-40e3-8628-84b45c46e1d7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:38,272 [527] [DEBUG] [app] Starting request: urn:request:8ce92e86-79e7-40e3-8628-84b45c46e1d7 (/health/instance) 2020-03-31 09:40:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:38,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:38,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:38,280 [555] [DEBUG] [app] Starting request: urn:request:cddcf319-a539-4b7b-80cd-37ab1a1ab364 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:38,280 [555] [DEBUG] [app] Starting request: urn:request:cddcf319-a539-4b7b-80cd-37ab1a1ab364 (/v1/_internal_ping) 2020-03-31 09:40:38,282 [555] [DEBUG] [app] Ending request: urn:request:cddcf319-a539-4b7b-80cd-37ab1a1ab364 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:38,282 [555] [DEBUG] [app] Ending request: urn:request:cddcf319-a539-4b7b-80cd-37ab1a1ab364 (/v1/_internal_ping) 2020-03-31 09:40:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:40:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:38,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:40:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:40:38,284 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:38,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:38,286 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:38,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:38,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:38,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:38,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:38,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:38,315 [526] [DEBUG] [app] Starting request: urn:request:3d027daf-56ba-4414-a11f-89450f4c022f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:38,315 [526] [DEBUG] [app] Starting request: urn:request:3d027daf-56ba-4414-a11f-89450f4c022f (/_internal_ping) 2020-03-31 09:40:38,316 [526] [DEBUG] [app] Ending request: urn:request:3d027daf-56ba-4414-a11f-89450f4c022f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:38,316 [526] [DEBUG] [app] Ending request: urn:request:3d027daf-56ba-4414-a11f-89450f4c022f (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:40:38,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:38,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:38,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:38,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:40:38,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:40:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:38,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:38,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:38,329 [541] [DEBUG] [app] Starting request: urn:request:497c6d5b-bca1-491a-ab51-6f3c32fac205 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:38,329 [541] [DEBUG] [app] Starting request: urn:request:497c6d5b-bca1-491a-ab51-6f3c32fac205 (/c1/_internal_ping) 2020-03-31 09:40:38,330 [541] [DEBUG] [app] Ending request: urn:request:497c6d5b-bca1-491a-ab51-6f3c32fac205 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:38,330 [541] [DEBUG] [app] Ending request: urn:request:497c6d5b-bca1-491a-ab51-6f3c32fac205 (/c1/_internal_ping) 2020-03-31 09:40:38,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:38,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:38,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:40:38,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:38,332 [527] [DEBUG] [app] Ending request: urn:request:8ce92e86-79e7-40e3-8628-84b45c46e1d7 (/health/instance) 2020-03-31 09:40:38,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:38,332 [527] [DEBUG] [app] Ending request: urn:request:8ce92e86-79e7-40e3-8628-84b45c46e1d7 (/health/instance) 2020-03-31 09:40:38,333 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:40:38,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:40:38,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:40:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:40:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:51.440707+00:00 (in 9.999183 seconds) 2020-03-31 09:40:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:51 UTC)" (scheduled at 2020-03-31 09:40:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:40:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:51.440707+00:00 (in 9.999183 seconds) 2020-03-31 09:40:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:51 UTC)" (scheduled at 2020-03-31 09:40:41.440707+00:00) 2020-03-31 09:40:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:40:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:40:41,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:40:41,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:40:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:40:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:51 UTC)" executed successfully 2020-03-31 09:40:43,542 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:40:43,542 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:43,543 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:33.366791+00:00 (in 49.823735 seconds) 2020-03-31 09:40:43,543 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:43 UTC)" (scheduled at 2020-03-31 09:40:43.539999+00:00) 2020-03-31 09:40:43,543 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:43,543 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:40:43,543 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:33.366791+00:00 (in 49.823735 seconds) 2020-03-31 09:40:43,543 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:43 UTC)" (scheduled at 2020-03-31 09:40:43.539999+00:00) 2020-03-31 09:40:43,543 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:43,543 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:43 UTC)" executed successfully 2020-03-31 09:40:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:40:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:49.498966+00:00 (in 5.174546 seconds) 2020-03-31 09:40:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:54 UTC)" (scheduled at 2020-03-31 09:40:44.323352+00:00) 2020-03-31 09:40:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:40:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:49.498966+00:00 (in 5.174546 seconds) 2020-03-31 09:40:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:54 UTC)" (scheduled at 2020-03-31 09:40:44.323352+00:00) 2020-03-31 09:40:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 44, 325361), True, datetime.datetime(2020, 3, 31, 9, 40, 44, 325361), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:40:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 44, 325361), True, datetime.datetime(2020, 3, 31, 9, 40, 44, 325361), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:40:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:40:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:40:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:54 UTC)" executed successfully 2020-03-31 09:40:44,446 [518] [DEBUG] [app] Starting request: urn:request:bb8867b2-e933-497a-96ce-39752e3ddd6e (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:44,446 [518] [DEBUG] [app] Starting request: urn:request:bb8867b2-e933-497a-96ce-39752e3ddd6e (/health/instance) 2020-03-31 09:40:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:44,454 [556] [DEBUG] [app] Starting request: urn:request:2af3be37-10b7-4930-b16a-05e96fe91b4b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:44,454 [556] [DEBUG] [app] Starting request: urn:request:2af3be37-10b7-4930-b16a-05e96fe91b4b (/v1/_internal_ping) 2020-03-31 09:40:44,455 [556] [DEBUG] [app] Ending request: urn:request:2af3be37-10b7-4930-b16a-05e96fe91b4b (/v1/_internal_ping) 2020-03-31 09:40:44,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:40:44,455 [556] [DEBUG] [app] Ending request: urn:request:2af3be37-10b7-4930-b16a-05e96fe91b4b (/v1/_internal_ping) 2020-03-31 09:40:44,455 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:44,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:40:44,457 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:44,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:40:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:40:44,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:44,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:44,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:44,487 [518] [DEBUG] [app] Starting request: urn:request:4d1f07c2-32db-43d7-99e7-7dd17da1219d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:44,487 [518] [DEBUG] [app] Starting request: urn:request:4d1f07c2-32db-43d7-99e7-7dd17da1219d (/_internal_ping) 2020-03-31 09:40:44,488 [518] [DEBUG] [app] Ending request: urn:request:4d1f07c2-32db-43d7-99e7-7dd17da1219d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:44,488 [518] [DEBUG] [app] Ending request: urn:request:4d1f07c2-32db-43d7-99e7-7dd17da1219d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:40:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:40:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:40:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:44,502 [541] [DEBUG] [app] Starting request: urn:request:3b6b34c6-ff9c-4afd-b7d8-7b27f356a084 (/c1/_internal_ping) 2020-03-31 09:40:44,503 [541] [DEBUG] [app] Ending request: urn:request:3b6b34c6-ff9c-4afd-b7d8-7b27f356a084 (/c1/_internal_ping) 2020-03-31 09:40:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:44,503 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:40:44,502 [541] [DEBUG] [app] Starting request: urn:request:3b6b34c6-ff9c-4afd-b7d8-7b27f356a084 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:44,503 [541] [DEBUG] [app] Ending request: urn:request:3b6b34c6-ff9c-4afd-b7d8-7b27f356a084 (/c1/_internal_ping) 2020-03-31 09:40:44,504 [518] [DEBUG] [app] Ending request: urn:request:bb8867b2-e933-497a-96ce-39752e3ddd6e (/health/instance) 2020-03-31 09:40:44,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:44,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:40:44,503 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:44,504 [518] [DEBUG] [app] Ending request: urn:request:bb8867b2-e933-497a-96ce-39752e3ddd6e (/health/instance) 2020-03-31 09:40:44,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:44,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:40:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:40:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:15.064514+00:00 (in 29.999219 seconds) 2020-03-31 09:40:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:15 UTC)" (scheduled at 2020-03-31 09:40:45.064514+00:00) gcworker stdout | 2020-03-31 09:40:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:15.064514+00:00 (in 29.999219 seconds) 2020-03-31 09:40:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:15 UTC)" (scheduled at 2020-03-31 09:40:45.064514+00:00) 2020-03-31 09:40:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438045, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:40:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438045, 1209600, 500, 1, 0]) 2020-03-31 09:40:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:40:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:40:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:40:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:15 UTC)" executed successfully 2020-03-31 09:40:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:40:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:56.924497+00:00 (in 9.999233 seconds) 2020-03-31 09:40:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:56 UTC)" (scheduled at 2020-03-31 09:40:46.924497+00:00) 2020-03-31 09:40:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 46, 926105), True, datetime.datetime(2020, 3, 31, 9, 40, 46, 926105), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:40:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:56.924497+00:00 (in 9.999233 seconds) 2020-03-31 09:40:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:56 UTC)" (scheduled at 2020-03-31 09:40:46.924497+00:00) 2020-03-31 09:40:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:40:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 46, 926105), True, datetime.datetime(2020, 3, 31, 9, 40, 46, 926105), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:40:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:40:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:40:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:40:56 UTC)" executed successfully 2020-03-31 09:40:48,273 [527] [DEBUG] [app] Starting request: urn:request:ac01662d-7343-43ae-bb15-ec02bde9e3a3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:48,273 [527] [DEBUG] [app] Starting request: urn:request:ac01662d-7343-43ae-bb15-ec02bde9e3a3 (/health/instance) 2020-03-31 09:40:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:48,281 [546] [DEBUG] [app] Starting request: urn:request:8e32fa5a-1d5d-4deb-a724-40c6133be8b5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:48,281 [546] [DEBUG] [app] Starting request: urn:request:8e32fa5a-1d5d-4deb-a724-40c6133be8b5 (/v1/_internal_ping) 2020-03-31 09:40:48,282 [546] [DEBUG] [app] Ending request: urn:request:8e32fa5a-1d5d-4deb-a724-40c6133be8b5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:48,282 [546] [DEBUG] [app] Ending request: urn:request:8e32fa5a-1d5d-4deb-a724-40c6133be8b5 (/v1/_internal_ping) 2020-03-31 09:40:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:40:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:48,284 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:40:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:40:48,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:48,284 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:40:48,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:48,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:48,286 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:48,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:48,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:48,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:48,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:48,315 [527] [DEBUG] [app] Starting request: urn:request:81a8bde8-9a2e-4fd9-8410-947c3b61e897 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:48,315 [527] [DEBUG] [app] Starting request: urn:request:81a8bde8-9a2e-4fd9-8410-947c3b61e897 (/_internal_ping) 2020-03-31 09:40:48,317 [527] [DEBUG] [app] Ending request: urn:request:81a8bde8-9a2e-4fd9-8410-947c3b61e897 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:48,317 [527] [DEBUG] [app] Ending request: urn:request:81a8bde8-9a2e-4fd9-8410-947c3b61e897 (/_internal_ping) 2020-03-31 09:40:48,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:40:48,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:40:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:40:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:40:48,327 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:48,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:48,331 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:48,333 [537] [DEBUG] [app] Starting request: urn:request:59bd2e31-143f-4909-bcaa-c42ea37c2435 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:48,333 [537] [DEBUG] [app] Starting request: urn:request:59bd2e31-143f-4909-bcaa-c42ea37c2435 (/c1/_internal_ping) 2020-03-31 09:40:48,334 [537] [DEBUG] [app] Ending request: urn:request:59bd2e31-143f-4909-bcaa-c42ea37c2435 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:48,334 [537] [DEBUG] [app] Ending request: urn:request:59bd2e31-143f-4909-bcaa-c42ea37c2435 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:40:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:48,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:40:48,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:48,336 [527] [DEBUG] [app] Ending request: urn:request:ac01662d-7343-43ae-bb15-ec02bde9e3a3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:48,336 [527] [DEBUG] [app] Ending request: urn:request:ac01662d-7343-43ae-bb15-ec02bde9e3a3 (/health/instance) 2020-03-31 09:40:48,337 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:48,337 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:48,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:40:48,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:40:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:40:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:49,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:54.323352+00:00 (in 4.823661 seconds) 2020-03-31 09:40:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:49 UTC)" (scheduled at 2020-03-31 09:40:49.498966+00:00) 2020-03-31 09:40:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:40:49,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:54.323352+00:00 (in 4.823661 seconds) 2020-03-31 09:40:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:49 UTC)" (scheduled at 2020-03-31 09:40:49.498966+00:00) 2020-03-31 09:40:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:49 UTC)" executed successfully 2020-03-31 09:40:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:40:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:59.560326+00:00 (in 8.448818 seconds) 2020-03-31 09:40:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:51 UTC)" (scheduled at 2020-03-31 09:40:51.110610+00:00) 2020-03-31 09:40:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:40:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:40:59.560326+00:00 (in 8.448818 seconds) 2020-03-31 09:40:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:51 UTC)" (scheduled at 2020-03-31 09:40:51.110610+00:00) 2020-03-31 09:40:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 51, 112459), True, datetime.datetime(2020, 3, 31, 9, 40, 51, 112459), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:40:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 51, 112459), True, datetime.datetime(2020, 3, 31, 9, 40, 51, 112459), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:40:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:40:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:40:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:51 UTC)" executed successfully 2020-03-31 09:40:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:40:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:01.440707+00:00 (in 9.998877 seconds) 2020-03-31 09:40:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:01 UTC)" (scheduled at 2020-03-31 09:40:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:40:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:01.440707+00:00 (in 9.998877 seconds) 2020-03-31 09:40:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:01 UTC)" (scheduled at 2020-03-31 09:40:51.440707+00:00) 2020-03-31 09:40:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:40:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:40:51,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:40:51,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:40:51,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:40:51,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:51,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:01 UTC)" executed successfully 2020-03-31 09:40:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:40:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:22.113221+00:00 (in 29.999186 seconds) 2020-03-31 09:40:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:22 UTC)" (scheduled at 2020-03-31 09:40:52.113221+00:00) securityworker stdout | 2020-03-31 09:40:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:22.113221+00:00 (in 29.999186 seconds) 2020-03-31 09:40:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:22 UTC)" (scheduled at 2020-03-31 09:40:52.113221+00:00) 2020-03-31 09:40:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:40:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:40:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:40:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:40:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:40:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:40:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:40:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:22 UTC)" executed successfully 2020-03-31 09:40:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:40:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:04.323352+00:00 (in 9.999291 seconds) 2020-03-31 09:40:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:04 UTC)" (scheduled at 2020-03-31 09:40:54.323352+00:00) 2020-03-31 09:40:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:40:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:04.323352+00:00 (in 9.999291 seconds) 2020-03-31 09:40:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:04 UTC)" (scheduled at 2020-03-31 09:40:54.323352+00:00) 2020-03-31 09:40:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 54, 324864), True, datetime.datetime(2020, 3, 31, 9, 40, 54, 324864), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:40:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 54, 324864), True, datetime.datetime(2020, 3, 31, 9, 40, 54, 324864), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:40:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:40:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:40:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:04 UTC)" executed successfully 2020-03-31 09:40:54,446 [527] [DEBUG] [app] Starting request: urn:request:7d890092-6c54-44fd-a664-5ef121fb890e (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:54,446 [527] [DEBUG] [app] Starting request: urn:request:7d890092-6c54-44fd-a664-5ef121fb890e (/health/instance) 2020-03-31 09:40:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:54,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:54,453 [552] [DEBUG] [app] Starting request: urn:request:3d722e10-5449-40d8-a2cb-f87a30c6f974 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:54,453 [552] [DEBUG] [app] Starting request: urn:request:3d722e10-5449-40d8-a2cb-f87a30c6f974 (/v1/_internal_ping) 2020-03-31 09:40:54,454 [552] [DEBUG] [app] Ending request: urn:request:3d722e10-5449-40d8-a2cb-f87a30c6f974 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:54,454 [552] [DEBUG] [app] Ending request: urn:request:3d722e10-5449-40d8-a2cb-f87a30c6f974 (/v1/_internal_ping) 2020-03-31 09:40:54,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:40:54,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:40:54,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:40:54,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:54,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:54,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:54,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:54,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:54,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:54,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:54,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:54,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:54,487 [527] [DEBUG] [app] Starting request: urn:request:938f9b63-3aa7-435a-b733-c46101816db2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:54,487 [527] [DEBUG] [app] Starting request: urn:request:938f9b63-3aa7-435a-b733-c46101816db2 (/_internal_ping) 2020-03-31 09:40:54,487 [527] [DEBUG] [app] Ending request: urn:request:938f9b63-3aa7-435a-b733-c46101816db2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:54,487 [527] [DEBUG] [app] Ending request: urn:request:938f9b63-3aa7-435a-b733-c46101816db2 (/_internal_ping) 2020-03-31 09:40:54,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:40:54,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:54,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:40:54,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:54,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:54,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:40:54,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:40:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:54,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:54,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:54,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:54,500 [539] [DEBUG] [app] Starting request: urn:request:0505fe75-b435-4d9b-ad09-a82ae4e0580b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:54,500 [539] [DEBUG] [app] Starting request: urn:request:0505fe75-b435-4d9b-ad09-a82ae4e0580b (/c1/_internal_ping) 2020-03-31 09:40:54,501 [539] [DEBUG] [app] Ending request: urn:request:0505fe75-b435-4d9b-ad09-a82ae4e0580b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:54,501 [539] [DEBUG] [app] Ending request: urn:request:0505fe75-b435-4d9b-ad09-a82ae4e0580b (/c1/_internal_ping) 2020-03-31 09:40:54,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:54,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:54,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:54,503 [527] [DEBUG] [app] Ending request: urn:request:7d890092-6c54-44fd-a664-5ef121fb890e (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:40:54,501 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:54,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:54,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:40:54,503 [527] [DEBUG] [app] Ending request: urn:request:7d890092-6c54-44fd-a664-5ef121fb890e (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:54,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:54,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:40:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:40:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:06 UTC)" (scheduled at 2020-03-31 09:40:56.924497+00:00) 2020-03-31 09:40:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:40:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:06 UTC)" (scheduled at 2020-03-31 09:40:56.924497+00:00) 2020-03-31 09:40:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:40:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:06.924497+00:00 (in 9.999327 seconds) storagereplication stdout | 2020-03-31 09:40:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:06.924497+00:00 (in 9.999327 seconds) 2020-03-31 09:40:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 56, 925943), True, datetime.datetime(2020, 3, 31, 9, 40, 56, 925943), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:40:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 40, 56, 925943), True, datetime.datetime(2020, 3, 31, 9, 40, 56, 925943), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:40:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:40:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:40:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:40:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:06 UTC)" executed successfully 2020-03-31 09:40:58,273 [518] [DEBUG] [app] Starting request: urn:request:51116e6a-1523-4fdd-9ba8-38542c8be337 (/health/instance) gunicorn-web stdout | 2020-03-31 09:40:58,273 [518] [DEBUG] [app] Starting request: urn:request:51116e6a-1523-4fdd-9ba8-38542c8be337 (/health/instance) 2020-03-31 09:40:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:58,281 [556] [DEBUG] [app] Starting request: urn:request:0b0bd7fa-2f31-4c0f-b2ae-f42de6f2c05d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:58,281 [556] [DEBUG] [app] Starting request: urn:request:0b0bd7fa-2f31-4c0f-b2ae-f42de6f2c05d (/v1/_internal_ping) 2020-03-31 09:40:58,282 [556] [DEBUG] [app] Ending request: urn:request:0b0bd7fa-2f31-4c0f-b2ae-f42de6f2c05d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:40:58,282 [556] [DEBUG] [app] Ending request: urn:request:0b0bd7fa-2f31-4c0f-b2ae-f42de6f2c05d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:58,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:40:58,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:40:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:40:58,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:40:58,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:40:58,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:40:58,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:40:58,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:40:58,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:40:58,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:40:58,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:40:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:40:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:40:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:58,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:58,314 [526] [DEBUG] [app] Starting request: urn:request:e8db5863-8de3-4ef2-8381-8cc347f58011 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:58,314 [526] [DEBUG] [app] Starting request: urn:request:e8db5863-8de3-4ef2-8381-8cc347f58011 (/_internal_ping) 2020-03-31 09:40:58,315 [526] [DEBUG] [app] Ending request: urn:request:e8db5863-8de3-4ef2-8381-8cc347f58011 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:40:58,315 [526] [DEBUG] [app] Ending request: urn:request:e8db5863-8de3-4ef2-8381-8cc347f58011 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:40:58,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:40:58,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:58,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:40:58,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:40:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:58,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:40:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:40:58,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:40:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:40:58,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:40:58,327 [539] [DEBUG] [app] Starting request: urn:request:e16bad67-073f-479a-9fa9-9d6eba5bf251 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:58,327 [539] [DEBUG] [app] Starting request: urn:request:e16bad67-073f-479a-9fa9-9d6eba5bf251 (/c1/_internal_ping) 2020-03-31 09:40:58,328 [539] [DEBUG] [app] Ending request: urn:request:e16bad67-073f-479a-9fa9-9d6eba5bf251 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:40:58,328 [539] [DEBUG] [app] Ending request: urn:request:e16bad67-073f-479a-9fa9-9d6eba5bf251 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:40:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:40:58,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:40:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:40:58,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:40:58,330 [518] [DEBUG] [app] Ending request: urn:request:51116e6a-1523-4fdd-9ba8-38542c8be337 (/health/instance) 2020-03-31 09:40:58,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:40:58,330 [518] [DEBUG] [app] Ending request: urn:request:51116e6a-1523-4fdd-9ba8-38542c8be337 (/health/instance) 2020-03-31 09:40:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:40:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:40:58,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:40:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:40:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:40:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:40:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:40:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:59 UTC)" (scheduled at 2020-03-31 09:40:59.560326+00:00) 2020-03-31 09:40:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:40:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:59 UTC)" (scheduled at 2020-03-31 09:40:59.560326+00:00) 2020-03-31 09:40:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:40:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:51.110610+00:00 (in 51.549206 seconds) 2020-03-31 09:40:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:40:59,562 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:51.110610+00:00 (in 51.549206 seconds) 2020-03-31 09:40:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:41:59 UTC)" executed successfully 2020-03-31 09:41:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:41:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:30 UTC)" (scheduled at 2020-03-31 09:41:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:41:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:30 UTC)" (scheduled at 2020-03-31 09:41:00.337699+00:00) 2020-03-31 09:41:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 41, 0, 339702), False, 50, 1, 0]) 2020-03-31 09:41:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:30.337699+00:00 (in 29.998723 seconds) buildlogsarchiver stdout | 2020-03-31 09:41:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 41, 0, 339702), False, 50, 1, 0]) 2020-03-31 09:41:00,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:30.337699+00:00 (in 29.998723 seconds) 2020-03-31 09:41:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:41:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:41:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:41:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:30 UTC)" executed successfully 2020-03-31 09:41:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:41:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:11.440707+00:00 (in 9.998896 seconds) 2020-03-31 09:41:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:11 UTC)" (scheduled at 2020-03-31 09:41:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:41:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:11.440707+00:00 (in 9.998896 seconds) 2020-03-31 09:41:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:11 UTC)" (scheduled at 2020-03-31 09:41:01.440707+00:00) 2020-03-31 09:41:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:41:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:41:01,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:41:01,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:41:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:41:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:11 UTC)" executed successfully 2020-03-31 09:41:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:41:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:24.410927+00:00 (in 21.079943 seconds) 2020-03-31 09:41:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:33 UTC)" (scheduled at 2020-03-31 09:41:03.330112+00:00) 2020-03-31 09:41:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:41:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:24.410927+00:00 (in 21.079943 seconds) 2020-03-31 09:41:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:33 UTC)" (scheduled at 2020-03-31 09:41:03.330112+00:00) 2020-03-31 09:41:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 3, 331900), True, datetime.datetime(2020, 3, 31, 9, 41, 3, 331900), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:41:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 3, 331900), True, datetime.datetime(2020, 3, 31, 9, 41, 3, 331900), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:41:03,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:41:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:41:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:03,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:33 UTC)" executed successfully 2020-03-31 09:41:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:41:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:14 UTC)" (scheduled at 2020-03-31 09:41:04.323352+00:00) 2020-03-31 09:41:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:41:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:14 UTC)" (scheduled at 2020-03-31 09:41:04.323352+00:00) 2020-03-31 09:41:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:14.323352+00:00 (in 9.999052 seconds) notificationworker stdout | 2020-03-31 09:41:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:14.323352+00:00 (in 9.999052 seconds) 2020-03-31 09:41:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 4, 325266), True, datetime.datetime(2020, 3, 31, 9, 41, 4, 325266), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:41:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 4, 325266), True, datetime.datetime(2020, 3, 31, 9, 41, 4, 325266), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:41:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:41:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:41:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:14 UTC)" executed successfully 2020-03-31 09:41:04,446 [527] [DEBUG] [app] Starting request: urn:request:91831ead-783d-44e0-a107-df68a9f88ffa (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:04,446 [527] [DEBUG] [app] Starting request: urn:request:91831ead-783d-44e0-a107-df68a9f88ffa (/health/instance) 2020-03-31 09:41:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:04,454 [547] [DEBUG] [app] Starting request: urn:request:da683af6-acb3-42a8-880d-123ab85a307e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:04,454 [547] [DEBUG] [app] Starting request: urn:request:da683af6-acb3-42a8-880d-123ab85a307e (/v1/_internal_ping) 2020-03-31 09:41:04,455 [547] [DEBUG] [app] Ending request: urn:request:da683af6-acb3-42a8-880d-123ab85a307e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:04,455 [547] [DEBUG] [app] Ending request: urn:request:da683af6-acb3-42a8-880d-123ab85a307e (/v1/_internal_ping) 2020-03-31 09:41:04,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:41:04,455 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:04,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:41:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:41:04,458 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:04,459 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:04,459 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:04,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:04,460 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:04,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:04,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:04,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:04,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:04,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:04,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:04,486 [518] [DEBUG] [app] Starting request: urn:request:54b75649-d38e-4af5-8635-c23fa57122ca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:04,486 [518] [DEBUG] [app] Starting request: urn:request:54b75649-d38e-4af5-8635-c23fa57122ca (/_internal_ping) 2020-03-31 09:41:04,487 [518] [DEBUG] [app] Ending request: urn:request:54b75649-d38e-4af5-8635-c23fa57122ca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:04,487 [518] [DEBUG] [app] Ending request: urn:request:54b75649-d38e-4af5-8635-c23fa57122ca (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:41:04,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:04,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:41:04,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:04,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:04,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:04,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:04,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:41:04,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:04,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:04,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:04,501 [539] [DEBUG] [app] Starting request: urn:request:d11238cc-9d90-48c9-9326-97edc933ef8f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:04,501 [539] [DEBUG] [app] Starting request: urn:request:d11238cc-9d90-48c9-9326-97edc933ef8f (/c1/_internal_ping) 2020-03-31 09:41:04,502 [539] [DEBUG] [app] Ending request: urn:request:d11238cc-9d90-48c9-9326-97edc933ef8f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:04,502 [539] [DEBUG] [app] Ending request: urn:request:d11238cc-9d90-48c9-9326-97edc933ef8f (/c1/_internal_ping) 2020-03-31 09:41:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:04,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:04,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:04,505 [527] [DEBUG] [app] Ending request: urn:request:91831ead-783d-44e0-a107-df68a9f88ffa (/health/instance) 2020-03-31 09:41:04,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:41:04,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:04,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:41:04,505 [527] [DEBUG] [app] Ending request: urn:request:91831ead-783d-44e0-a107-df68a9f88ffa (/health/instance) 2020-03-31 09:41:04,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:04,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:41:06,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:41:06,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:16.924497+00:00 (in 9.996187 seconds) 2020-03-31 09:41:06,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:16 UTC)" (scheduled at 2020-03-31 09:41:06.924497+00:00) 2020-03-31 09:41:06,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:06,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 6, 929010), True, datetime.datetime(2020, 3, 31, 9, 41, 6, 929010), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:41:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:16.924497+00:00 (in 9.996187 seconds) 2020-03-31 09:41:06,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:16 UTC)" (scheduled at 2020-03-31 09:41:06.924497+00:00) 2020-03-31 09:41:06,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:41:06,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 6, 929010), True, datetime.datetime(2020, 3, 31, 9, 41, 6, 929010), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:41:06,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:41:06,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:41:06,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:06,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:16 UTC)" executed successfully 2020-03-31 09:41:08,273 [527] [DEBUG] [app] Starting request: urn:request:de09cf8f-c5e8-4c5e-ab21-1748ad11cd1e (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:08,273 [527] [DEBUG] [app] Starting request: urn:request:de09cf8f-c5e8-4c5e-ab21-1748ad11cd1e (/health/instance) 2020-03-31 09:41:08,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:08,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:08,279 [553] [DEBUG] [app] Starting request: urn:request:d458d353-6eb7-4dc1-8bec-072837f9f24e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:08,279 [553] [DEBUG] [app] Starting request: urn:request:d458d353-6eb7-4dc1-8bec-072837f9f24e (/v1/_internal_ping) 2020-03-31 09:41:08,280 [553] [DEBUG] [app] Ending request: urn:request:d458d353-6eb7-4dc1-8bec-072837f9f24e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:08,280 [553] [DEBUG] [app] Ending request: urn:request:d458d353-6eb7-4dc1-8bec-072837f9f24e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:08,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:08,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:41:08,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:08,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:08,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:08,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:08,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:41:08,283 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:08,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:08,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:08,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:08,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:08,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:08,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:08,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:08,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:08,314 [527] [DEBUG] [app] Starting request: urn:request:ae5ed74d-d9e6-4303-b8f8-d9ffb5d02559 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:08,314 [527] [DEBUG] [app] Starting request: urn:request:ae5ed74d-d9e6-4303-b8f8-d9ffb5d02559 (/_internal_ping) 2020-03-31 09:41:08,315 [527] [DEBUG] [app] Ending request: urn:request:ae5ed74d-d9e6-4303-b8f8-d9ffb5d02559 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:08,315 [527] [DEBUG] [app] Ending request: urn:request:ae5ed74d-d9e6-4303-b8f8-d9ffb5d02559 (/_internal_ping) 2020-03-31 09:41:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:08,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:08,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:08,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:41:08,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:41:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:08,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:08,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:41:08,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:08,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:08,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:08,327 [537] [DEBUG] [app] Starting request: urn:request:228cdb1e-1f79-4415-a321-28e769d19057 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:08,327 [537] [DEBUG] [app] Starting request: urn:request:228cdb1e-1f79-4415-a321-28e769d19057 (/c1/_internal_ping) 2020-03-31 09:41:08,327 [537] [DEBUG] [app] Ending request: urn:request:228cdb1e-1f79-4415-a321-28e769d19057 (/c1/_internal_ping) 2020-03-31 09:41:08,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:08,328 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:41:08,327 [537] [DEBUG] [app] Ending request: urn:request:228cdb1e-1f79-4415-a321-28e769d19057 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:08,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:08,330 [527] [DEBUG] [app] Ending request: urn:request:de09cf8f-c5e8-4c5e-ab21-1748ad11cd1e (/health/instance) 2020-03-31 09:41:08,330 [527] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:08,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-verbs stdout | 2020-03-31 09:41:08,328 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:08,330 [527] [DEBUG] [app] Ending request: urn:request:de09cf8f-c5e8-4c5e-ab21-1748ad11cd1e (/health/instance) 2020-03-31 09:41:08,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:08,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:41:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:41:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:21.440707+00:00 (in 9.999039 seconds) 2020-03-31 09:41:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:21 UTC)" (scheduled at 2020-03-31 09:41:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:41:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:21.440707+00:00 (in 9.999039 seconds) 2020-03-31 09:41:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:21 UTC)" (scheduled at 2020-03-31 09:41:11.440707+00:00) 2020-03-31 09:41:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:41:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:41:11,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:41:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:41:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:41:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:11,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:21 UTC)" executed successfully 2020-03-31 09:41:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:41:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:14,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:24 UTC)" (scheduled at 2020-03-31 09:41:14.323352+00:00) 2020-03-31 09:41:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:24.323352+00:00 (in 9.996136 seconds) 2020-03-31 09:41:14,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:41:14,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:24 UTC)" (scheduled at 2020-03-31 09:41:14.323352+00:00) 2020-03-31 09:41:14,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:24.323352+00:00 (in 9.996136 seconds) 2020-03-31 09:41:14,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 14, 328212), True, datetime.datetime(2020, 3, 31, 9, 41, 14, 328212), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:41:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 14, 328212), True, datetime.datetime(2020, 3, 31, 9, 41, 14, 328212), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:41:14,349 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:41:14,349 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:14,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:14,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:41:14,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:14,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:24 UTC)" executed successfully 2020-03-31 09:41:14,446 [527] [DEBUG] [app] Starting request: urn:request:97ad59ef-10ba-41de-8bd8-7df3750f0de1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:14,446 [527] [DEBUG] [app] Starting request: urn:request:97ad59ef-10ba-41de-8bd8-7df3750f0de1 (/health/instance) 2020-03-31 09:41:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:14,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:14,454 [544] [DEBUG] [app] Starting request: urn:request:eebd30b3-3835-4e0b-998c-9e9d755994b2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:14,454 [544] [DEBUG] [app] Starting request: urn:request:eebd30b3-3835-4e0b-998c-9e9d755994b2 (/v1/_internal_ping) 2020-03-31 09:41:14,455 [544] [DEBUG] [app] Ending request: urn:request:eebd30b3-3835-4e0b-998c-9e9d755994b2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:14,455 [544] [DEBUG] [app] Ending request: urn:request:eebd30b3-3835-4e0b-998c-9e9d755994b2 (/v1/_internal_ping) 2020-03-31 09:41:14,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:41:14,456 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:14,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:41:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:14,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:14,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:41:14,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:14,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:14,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:14,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:14,473 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:14,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:14,486 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:14,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:14,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:14,491 [525] [DEBUG] [app] Starting request: urn:request:82d41169-c0aa-468a-b3a7-5d6b0afb3023 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:14,491 [525] [DEBUG] [app] Starting request: urn:request:82d41169-c0aa-468a-b3a7-5d6b0afb3023 (/_internal_ping) 2020-03-31 09:41:14,492 [525] [DEBUG] [app] Ending request: urn:request:82d41169-c0aa-468a-b3a7-5d6b0afb3023 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:14,492 [525] [DEBUG] [app] Ending request: urn:request:82d41169-c0aa-468a-b3a7-5d6b0afb3023 (/_internal_ping) 2020-03-31 09:41:14,492 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:14,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:14,492 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:14,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:14,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:14,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:14,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:41:14,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:14,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:14,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:14,504 [537] [DEBUG] [app] Starting request: urn:request:738b9f86-e7ee-4d5e-9c63-509650c061f9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:14,504 [537] [DEBUG] [app] Starting request: urn:request:738b9f86-e7ee-4d5e-9c63-509650c061f9 (/c1/_internal_ping) 2020-03-31 09:41:14,505 [537] [DEBUG] [app] Ending request: urn:request:738b9f86-e7ee-4d5e-9c63-509650c061f9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:14,505 [537] [DEBUG] [app] Ending request: urn:request:738b9f86-e7ee-4d5e-9c63-509650c061f9 (/c1/_internal_ping) 2020-03-31 09:41:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:14,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:14,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:14,507 [527] [DEBUG] [app] Ending request: urn:request:97ad59ef-10ba-41de-8bd8-7df3750f0de1 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:41:14,505 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:14,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:14,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:41:14,507 [527] [DEBUG] [app] Ending request: urn:request:97ad59ef-10ba-41de-8bd8-7df3750f0de1 (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:14,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:14,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:41:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:41:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:45.064514+00:00 (in 29.999196 seconds) 2020-03-31 09:41:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:45 UTC)" (scheduled at 2020-03-31 09:41:15.064514+00:00) gcworker stdout | 2020-03-31 09:41:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:45.064514+00:00 (in 29.999196 seconds) 2020-03-31 09:41:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:45 UTC)" (scheduled at 2020-03-31 09:41:15.064514+00:00) 2020-03-31 09:41:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438075, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:41:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438075, 1209600, 500, 1, 0]) 2020-03-31 09:41:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:41:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:41:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:41:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:45 UTC)" executed successfully 2020-03-31 09:41:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:41:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:26.924497+00:00 (in 9.999269 seconds) 2020-03-31 09:41:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:26 UTC)" (scheduled at 2020-03-31 09:41:16.924497+00:00) 2020-03-31 09:41:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:41:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:26.924497+00:00 (in 9.999269 seconds) 2020-03-31 09:41:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:26 UTC)" (scheduled at 2020-03-31 09:41:16.924497+00:00) 2020-03-31 09:41:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 16, 925884), True, datetime.datetime(2020, 3, 31, 9, 41, 16, 925884), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:41:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 16, 925884), True, datetime.datetime(2020, 3, 31, 9, 41, 16, 925884), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:41:16,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:41:16,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:41:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:26 UTC)" executed successfully 2020-03-31 09:41:18,272 [518] [DEBUG] [app] Starting request: urn:request:67e899e0-f612-44d9-bd99-b5a355eb3a53 (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:18,272 [518] [DEBUG] [app] Starting request: urn:request:67e899e0-f612-44d9-bd99-b5a355eb3a53 (/health/instance) 2020-03-31 09:41:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:18,280 [538] [DEBUG] [app] Starting request: urn:request:b9a698be-aea9-4d5e-8614-a36b60a4405f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:18,280 [538] [DEBUG] [app] Starting request: urn:request:b9a698be-aea9-4d5e-8614-a36b60a4405f (/v1/_internal_ping) 2020-03-31 09:41:18,282 [538] [DEBUG] [app] Ending request: urn:request:b9a698be-aea9-4d5e-8614-a36b60a4405f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:18,282 [538] [DEBUG] [app] Ending request: urn:request:b9a698be-aea9-4d5e-8614-a36b60a4405f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:41:18,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:41:18,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:18,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:18,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:18,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:18,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:18,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:18,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:18,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:18,319 [518] [DEBUG] [app] Starting request: urn:request:b131f991-d960-4c77-87bd-88686bba9851 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:18,319 [518] [DEBUG] [app] Starting request: urn:request:b131f991-d960-4c77-87bd-88686bba9851 (/_internal_ping) 2020-03-31 09:41:18,320 [518] [DEBUG] [app] Ending request: urn:request:b131f991-d960-4c77-87bd-88686bba9851 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:18,320 [518] [DEBUG] [app] Ending request: urn:request:b131f991-d960-4c77-87bd-88686bba9851 (/_internal_ping) 2020-03-31 09:41:18,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:18,320 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:41:18,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:41:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:41:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:18,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:18,332 [537] [DEBUG] [app] Starting request: urn:request:9583fd53-4890-4106-9a2c-b459c4208aa9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:18,332 [537] [DEBUG] [app] Starting request: urn:request:9583fd53-4890-4106-9a2c-b459c4208aa9 (/c1/_internal_ping) 2020-03-31 09:41:18,333 [537] [DEBUG] [app] Ending request: urn:request:9583fd53-4890-4106-9a2c-b459c4208aa9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:18,333 [537] [DEBUG] [app] Ending request: urn:request:9583fd53-4890-4106-9a2c-b459c4208aa9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:18,334 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:41:18,334 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:18,335 [518] [DEBUG] [app] Ending request: urn:request:67e899e0-f612-44d9-bd99-b5a355eb3a53 (/health/instance) 2020-03-31 09:41:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:18,335 [518] [DEBUG] [app] Ending request: urn:request:67e899e0-f612-44d9-bd99-b5a355eb3a53 (/health/instance) 2020-03-31 09:41:18,335 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:41:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:41:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:31.440707+00:00 (in 9.998973 seconds) repositoryactioncounter stdout | 2020-03-31 09:41:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:31.440707+00:00 (in 9.998973 seconds) 2020-03-31 09:41:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:31 UTC)" (scheduled at 2020-03-31 09:41:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:41:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:31 UTC)" (scheduled at 2020-03-31 09:41:21.440707+00:00) 2020-03-31 09:41:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:41:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:41:21,538 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:41:21,538 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:41:21,538 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:21,538 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:41:21,538 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:21,538 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:31 UTC)" executed successfully 2020-03-31 09:41:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:41:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:52.113221+00:00 (in 29.999182 seconds) 2020-03-31 09:41:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:52 UTC)" (scheduled at 2020-03-31 09:41:22.113221+00:00) securityworker stdout | 2020-03-31 09:41:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:52.113221+00:00 (in 29.999182 seconds) 2020-03-31 09:41:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:52 UTC)" (scheduled at 2020-03-31 09:41:22.113221+00:00) 2020-03-31 09:41:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:41:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:41:22,134 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:41:22,134 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:41:22,141 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:41:22,141 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:41:22,141 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:22,142 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:41:22,141 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:22,142 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:41:52 UTC)" executed successfully 2020-03-31 09:41:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:41:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:34.323352+00:00 (in 9.998729 seconds) 2020-03-31 09:41:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:34 UTC)" (scheduled at 2020-03-31 09:41:24.323352+00:00) 2020-03-31 09:41:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:41:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:34.323352+00:00 (in 9.998729 seconds) 2020-03-31 09:41:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:34 UTC)" (scheduled at 2020-03-31 09:41:24.323352+00:00) 2020-03-31 09:41:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 24, 325437), True, datetime.datetime(2020, 3, 31, 9, 41, 24, 325437), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:41:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 24, 325437), True, datetime.datetime(2020, 3, 31, 9, 41, 24, 325437), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:41:24,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:41:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:41:24,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:24,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:34 UTC)" executed successfully 2020-03-31 09:41:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:41:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:33.330112+00:00 (in 8.918349 seconds) 2020-03-31 09:41:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:24 UTC)" (scheduled at 2020-03-31 09:41:24.410927+00:00) 2020-03-31 09:41:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:41:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:33.330112+00:00 (in 8.918349 seconds) 2020-03-31 09:41:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:24 UTC)" (scheduled at 2020-03-31 09:41:24.410927+00:00) 2020-03-31 09:41:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:24 UTC)" executed successfully 2020-03-31 09:41:24,446 [518] [DEBUG] [app] Starting request: urn:request:5a398cbd-338d-481c-a271-94c48945174e (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:24,446 [518] [DEBUG] [app] Starting request: urn:request:5a398cbd-338d-481c-a271-94c48945174e (/health/instance) 2020-03-31 09:41:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:24,455 [546] [DEBUG] [app] Starting request: urn:request:f226f18e-544b-45d8-b6ff-ba6cf503269a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:24,455 [546] [DEBUG] [app] Starting request: urn:request:f226f18e-544b-45d8-b6ff-ba6cf503269a (/v1/_internal_ping) 2020-03-31 09:41:24,456 [546] [DEBUG] [app] Ending request: urn:request:f226f18e-544b-45d8-b6ff-ba6cf503269a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:24,456 [546] [DEBUG] [app] Ending request: urn:request:f226f18e-544b-45d8-b6ff-ba6cf503269a (/v1/_internal_ping) 2020-03-31 09:41:24,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:41:24,457 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:41:24,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:24,461 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:24,459 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:24,462 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:41:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:41:24,461 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:24,462 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:24,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:24,495 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:24,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:24,502 [527] [DEBUG] [app] Starting request: urn:request:6632cada-418c-48d5-92eb-cb6897d80cea (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:24,502 [527] [DEBUG] [app] Starting request: urn:request:6632cada-418c-48d5-92eb-cb6897d80cea (/_internal_ping) 2020-03-31 09:41:24,503 [527] [DEBUG] [app] Ending request: urn:request:6632cada-418c-48d5-92eb-cb6897d80cea (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:24,503 [527] [DEBUG] [app] Ending request: urn:request:6632cada-418c-48d5-92eb-cb6897d80cea (/_internal_ping) 2020-03-31 09:41:24,503 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:41:24,503 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:41:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:24,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:24,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:24,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:41:24,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:24,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:24,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:41:24,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:24,518 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:24,518 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:24,520 [535] [DEBUG] [app] Starting request: urn:request:1a65f6fb-4b0f-4b17-99bd-ddf58eda9b1b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:24,520 [535] [DEBUG] [app] Starting request: urn:request:1a65f6fb-4b0f-4b17-99bd-ddf58eda9b1b (/c1/_internal_ping) 2020-03-31 09:41:24,521 [535] [DEBUG] [app] Ending request: urn:request:1a65f6fb-4b0f-4b17-99bd-ddf58eda9b1b (/c1/_internal_ping) 2020-03-31 09:41:24,522 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:24,522 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:41:24,521 [535] [DEBUG] [app] Ending request: urn:request:1a65f6fb-4b0f-4b17-99bd-ddf58eda9b1b (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:24,522 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:24,524 [518] [DEBUG] [app] Ending request: urn:request:5a398cbd-338d-481c-a271-94c48945174e (/health/instance) 2020-03-31 09:41:24,524 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:24,525 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) gunicorn-verbs stdout | 2020-03-31 09:41:24,522 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:24,524 [518] [DEBUG] [app] Ending request: urn:request:5a398cbd-338d-481c-a271-94c48945174e (/health/instance) 2020-03-31 09:41:24,524 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:24,525 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.080 49 0.080) 2020-03-31 09:41:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:41:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:26 UTC)" (scheduled at 2020-03-31 09:41:26.924497+00:00) 2020-03-31 09:41:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:41:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:26 UTC)" (scheduled at 2020-03-31 09:41:26.924497+00:00) 2020-03-31 09:41:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 26, 925907), True, datetime.datetime(2020, 3, 31, 9, 41, 26, 925907), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:41:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 26, 925907), True, datetime.datetime(2020, 3, 31, 9, 41, 26, 925907), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:41:26,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:36.924497+00:00 (in 9.999173 seconds) storagereplication stdout | 2020-03-31 09:41:26,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:36.924497+00:00 (in 9.999173 seconds) 2020-03-31 09:41:26,953 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:41:26,953 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:26,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:26,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:41:26,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:26,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:36 UTC)" executed successfully 2020-03-31 09:41:28,314 [523] [DEBUG] [app] Starting request: urn:request:b35d519c-f820-46c2-97df-87edb92e762d (/health/instance) 2020-03-31 09:41:28,316 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:28,314 [523] [DEBUG] [app] Starting request: urn:request:b35d519c-f820-46c2-97df-87edb92e762d (/health/instance) 2020-03-31 09:41:28,316 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:28,324 [554] [DEBUG] [app] Starting request: urn:request:1a69ca0f-03b7-4273-b020-43e0971fc28d (/v1/_internal_ping) 2020-03-31 09:41:28,325 [554] [DEBUG] [app] Ending request: urn:request:1a69ca0f-03b7-4273-b020-43e0971fc28d (/v1/_internal_ping) 2020-03-31 09:41:28,326 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:28,331 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:28,332 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:28,333 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:28,322 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:28,328 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:28,329 [523] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2020-03-31 09:41:28,324 [554] [DEBUG] [app] Starting request: urn:request:1a69ca0f-03b7-4273-b020-43e0971fc28d (/v1/_internal_ping) 2020-03-31 09:41:28,325 [554] [DEBUG] [app] Ending request: urn:request:1a69ca0f-03b7-4273-b020-43e0971fc28d (/v1/_internal_ping) 2020-03-31 09:41:28,326 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:28,331 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:28,332 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:28,333 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:41:28,322 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:28,328 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:28,329 [523] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) 2020-03-31 09:41:28,370 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:28,370 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:28,496 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:28,496 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:28,525 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:28,528 [527] [DEBUG] [app] Starting request: urn:request:0873d9bd-5cf1-475e-9196-5b7691f3f625 (/_internal_ping) 2020-03-31 09:41:28,529 [527] [DEBUG] [app] Ending request: urn:request:0873d9bd-5cf1-475e-9196-5b7691f3f625 (/_internal_ping) 2020-03-31 09:41:28,529 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:28,530 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:41:28,525 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:28,528 [527] [DEBUG] [app] Starting request: urn:request:0873d9bd-5cf1-475e-9196-5b7691f3f625 (/_internal_ping) 2020-03-31 09:41:28,529 [527] [DEBUG] [app] Ending request: urn:request:0873d9bd-5cf1-475e-9196-5b7691f3f625 (/_internal_ping) 2020-03-31 09:41:28,529 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:28,530 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:28,544 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:28,582 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:41:28,544 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:28,611 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:28,613 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:41:28,582 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:41:28,611 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:41:28,613 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:28,618 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:28,618 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:28,622 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:28,625 [523] [DEBUG] [app] Ending request: urn:request:b35d519c-f820-46c2-97df-87edb92e762d (/health/instance) 2020-03-31 09:41:28,625 [523] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:28,626 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 10.131.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.314 49 0.314) 2020-03-31 09:41:28,620 [535] [DEBUG] [app] Starting request: urn:request:dbf8b5ed-8d1d-4ca5-9306-0e3ff7566d42 (/c1/_internal_ping) 2020-03-31 09:41:28,621 [535] [DEBUG] [app] Ending request: urn:request:dbf8b5ed-8d1d-4ca5-9306-0e3ff7566d42 (/c1/_internal_ping) 2020-03-31 09:41:28,622 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:28,622 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:28,625 [523] [DEBUG] [app] Ending request: urn:request:b35d519c-f820-46c2-97df-87edb92e762d (/health/instance) 2020-03-31 09:41:28,625 [523] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:28,626 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 10.131.0.1 () - - [31/Mar/2020:09:41:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.314 49 0.314) gunicorn-verbs stdout | 2020-03-31 09:41:28,620 [535] [DEBUG] [app] Starting request: urn:request:dbf8b5ed-8d1d-4ca5-9306-0e3ff7566d42 (/c1/_internal_ping) 2020-03-31 09:41:28,621 [535] [DEBUG] [app] Ending request: urn:request:dbf8b5ed-8d1d-4ca5-9306-0e3ff7566d42 (/c1/_internal_ping) 2020-03-31 09:41:28,622 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:41:30,340 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:00.337699+00:00 (in 29.996897 seconds) 2020-03-31 09:41:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:00 UTC)" (scheduled at 2020-03-31 09:41:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:41:30,341 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:00.337699+00:00 (in 29.996897 seconds) 2020-03-31 09:41:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:00 UTC)" (scheduled at 2020-03-31 09:41:30.337699+00:00) 2020-03-31 09:41:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 41, 30, 341614), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:41:30,342 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 41, 30, 341614), False, 50, 1, 0]) 2020-03-31 09:41:30,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:41:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:41:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:41:30,357 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:00 UTC)" executed successfully 2020-03-31 09:41:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:41:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:41.440707+00:00 (in 9.999180 seconds) 2020-03-31 09:41:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:41 UTC)" (scheduled at 2020-03-31 09:41:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:41:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:41.440707+00:00 (in 9.999180 seconds) 2020-03-31 09:41:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:41 UTC)" (scheduled at 2020-03-31 09:41:31.440707+00:00) 2020-03-31 09:41:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:41:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:41:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:41:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:41:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:41:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:41 UTC)" executed successfully 2020-03-31 09:41:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:41:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:03.330112+00:00 (in 29.999178 seconds) 2020-03-31 09:41:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:03 UTC)" (scheduled at 2020-03-31 09:41:33.330112+00:00) 2020-03-31 09:41:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:41:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:03.330112+00:00 (in 29.999178 seconds) 2020-03-31 09:41:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:03 UTC)" (scheduled at 2020-03-31 09:41:33.330112+00:00) 2020-03-31 09:41:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 33, 331822), True, datetime.datetime(2020, 3, 31, 9, 41, 33, 331822), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:41:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 33, 331822), True, datetime.datetime(2020, 3, 31, 9, 41, 33, 331822), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:41:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:41:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:41:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:03 UTC)" executed successfully 2020-03-31 09:41:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:41:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:43.539999+00:00 (in 10.172425 seconds) 2020-03-31 09:41:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:33 UTC)" (scheduled at 2020-03-31 09:41:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:41:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:43.539999+00:00 (in 10.172425 seconds) 2020-03-31 09:41:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:33 UTC)" (scheduled at 2020-03-31 09:41:33.366791+00:00) 2020-03-31 09:41:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:41:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 33, 368622), True, datetime.datetime(2020, 3, 31, 9, 41, 33, 368622), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:41:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 33, 368622), True, datetime.datetime(2020, 3, 31, 9, 41, 33, 368622), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:41:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:41:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:41:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:33 UTC)" executed successfully 2020-03-31 09:41:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:41:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:44.323352+00:00 (in 9.999237 seconds) 2020-03-31 09:41:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:44 UTC)" (scheduled at 2020-03-31 09:41:34.323352+00:00) 2020-03-31 09:41:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:41:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:44.323352+00:00 (in 9.999237 seconds) 2020-03-31 09:41:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:44 UTC)" (scheduled at 2020-03-31 09:41:34.323352+00:00) 2020-03-31 09:41:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 34, 324817), True, datetime.datetime(2020, 3, 31, 9, 41, 34, 324817), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:41:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 34, 324817), True, datetime.datetime(2020, 3, 31, 9, 41, 34, 324817), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:41:34,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:41:34,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:41:34,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:34,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:44 UTC)" executed successfully 2020-03-31 09:41:34,445 [527] [DEBUG] [app] Starting request: urn:request:b154dfb1-8d96-4049-b899-96a46fb9eb1d (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:34,445 [527] [DEBUG] [app] Starting request: urn:request:b154dfb1-8d96-4049-b899-96a46fb9eb1d (/health/instance) 2020-03-31 09:41:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:34,453 [551] [DEBUG] [app] Starting request: urn:request:beb22527-2cfe-4a71-946b-050aa5a8757a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:34,453 [551] [DEBUG] [app] Starting request: urn:request:beb22527-2cfe-4a71-946b-050aa5a8757a (/v1/_internal_ping) 2020-03-31 09:41:34,454 [551] [DEBUG] [app] Ending request: urn:request:beb22527-2cfe-4a71-946b-050aa5a8757a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:34,454 [551] [DEBUG] [app] Ending request: urn:request:beb22527-2cfe-4a71-946b-050aa5a8757a (/v1/_internal_ping) 2020-03-31 09:41:34,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:41:34,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:41:34,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:34,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:41:34,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:41:34,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:34,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:34,459 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:34,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:34,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:34,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:34,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:34,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:34,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:34,485 [520] [DEBUG] [app] Starting request: urn:request:36fb606b-79c7-4b16-8103-2d42f80206fd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:34,485 [520] [DEBUG] [app] Starting request: urn:request:36fb606b-79c7-4b16-8103-2d42f80206fd (/_internal_ping) 2020-03-31 09:41:34,486 [520] [DEBUG] [app] Ending request: urn:request:36fb606b-79c7-4b16-8103-2d42f80206fd (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:34,486 [520] [DEBUG] [app] Ending request: urn:request:36fb606b-79c7-4b16-8103-2d42f80206fd (/_internal_ping) 2020-03-31 09:41:34,486 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:34,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:34,486 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:34,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:41:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:41:34,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:34,501 [539] [DEBUG] [app] Starting request: urn:request:dcf6b1bf-c8bc-4817-9086-5d8c83c78b23 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:34,501 [539] [DEBUG] [app] Starting request: urn:request:dcf6b1bf-c8bc-4817-9086-5d8c83c78b23 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:34,502 [539] [DEBUG] [app] Ending request: urn:request:dcf6b1bf-c8bc-4817-9086-5d8c83c78b23 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:34,502 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:41:34,502 [539] [DEBUG] [app] Ending request: urn:request:dcf6b1bf-c8bc-4817-9086-5d8c83c78b23 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:34,504 [527] [DEBUG] [app] Ending request: urn:request:b154dfb1-8d96-4049-b899-96a46fb9eb1d (/health/instance) 2020-03-31 09:41:34,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:41:34,502 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:34,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:41:34,504 [527] [DEBUG] [app] Ending request: urn:request:b154dfb1-8d96-4049-b899-96a46fb9eb1d (/health/instance) 2020-03-31 09:41:34,505 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:34,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:41:36,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:41:36,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:36,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:38.010934+00:00 (in 1.081914 seconds) 2020-03-31 09:41:36,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:46 UTC)" (scheduled at 2020-03-31 09:41:36.924497+00:00) 2020-03-31 09:41:36,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:41:36,929 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:38.010934+00:00 (in 1.081914 seconds) 2020-03-31 09:41:36,929 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:46 UTC)" (scheduled at 2020-03-31 09:41:36.924497+00:00) 2020-03-31 09:41:36,929 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:36,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 36, 929823), True, datetime.datetime(2020, 3, 31, 9, 41, 36, 929823), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:41:36,932 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 36, 929823), True, datetime.datetime(2020, 3, 31, 9, 41, 36, 929823), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:41:36,953 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:41:36,953 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:36,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:36,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:41:36,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:36,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:46 UTC)" executed successfully 2020-03-31 09:41:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:41:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:46.924497+00:00 (in 8.912801 seconds) 2020-03-31 09:41:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:38 UTC)" (scheduled at 2020-03-31 09:41:38.010934+00:00) 2020-03-31 09:41:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:41:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:46.924497+00:00 (in 8.912801 seconds) 2020-03-31 09:41:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:38 UTC)" (scheduled at 2020-03-31 09:41:38.010934+00:00) 2020-03-31 09:41:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:38 UTC)" executed successfully 2020-03-31 09:41:38,272 [518] [DEBUG] [app] Starting request: urn:request:33301113-0930-4d5a-900b-dd5453fbf81c (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:38,272 [518] [DEBUG] [app] Starting request: urn:request:33301113-0930-4d5a-900b-dd5453fbf81c (/health/instance) 2020-03-31 09:41:38,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:38,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:38,280 [538] [DEBUG] [app] Starting request: urn:request:badbdf5e-b495-40e9-b1de-d96bcb33587d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:38,280 [538] [DEBUG] [app] Starting request: urn:request:badbdf5e-b495-40e9-b1de-d96bcb33587d (/v1/_internal_ping) 2020-03-31 09:41:38,281 [538] [DEBUG] [app] Ending request: urn:request:badbdf5e-b495-40e9-b1de-d96bcb33587d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:38,281 [538] [DEBUG] [app] Ending request: urn:request:badbdf5e-b495-40e9-b1de-d96bcb33587d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:38,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:41:38,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:41:38,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:38,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:38,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:38,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:38,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:38,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:38,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:38,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:38,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:38,313 [518] [DEBUG] [app] Starting request: urn:request:a171ff38-157b-4c1e-8974-0279e5542e8d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:38,313 [518] [DEBUG] [app] Starting request: urn:request:a171ff38-157b-4c1e-8974-0279e5542e8d (/_internal_ping) 2020-03-31 09:41:38,314 [518] [DEBUG] [app] Ending request: urn:request:a171ff38-157b-4c1e-8974-0279e5542e8d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:38,314 [518] [DEBUG] [app] Ending request: urn:request:a171ff38-157b-4c1e-8974-0279e5542e8d (/_internal_ping) 2020-03-31 09:41:38,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:38,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:41:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:41:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:38,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:41:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:41:38,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:38,327 [535] [DEBUG] [app] Starting request: urn:request:45dc410f-6ad5-4532-95a0-e0eaeccf2b6d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:38,327 [535] [DEBUG] [app] Starting request: urn:request:45dc410f-6ad5-4532-95a0-e0eaeccf2b6d (/c1/_internal_ping) 2020-03-31 09:41:38,328 [535] [DEBUG] [app] Ending request: urn:request:45dc410f-6ad5-4532-95a0-e0eaeccf2b6d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:38,328 [535] [DEBUG] [app] Ending request: urn:request:45dc410f-6ad5-4532-95a0-e0eaeccf2b6d (/c1/_internal_ping) 2020-03-31 09:41:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:38,328 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:38,330 [518] [DEBUG] [app] Ending request: urn:request:33301113-0930-4d5a-900b-dd5453fbf81c (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:41:38,328 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:38,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:38,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:41:38,330 [518] [DEBUG] [app] Ending request: urn:request:33301113-0930-4d5a-900b-dd5453fbf81c (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:38,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:38,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:41:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:41:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:51.440707+00:00 (in 9.999157 seconds) 2020-03-31 09:41:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:51 UTC)" (scheduled at 2020-03-31 09:41:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:41:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:51.440707+00:00 (in 9.999157 seconds) 2020-03-31 09:41:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:51 UTC)" (scheduled at 2020-03-31 09:41:41.440707+00:00) 2020-03-31 09:41:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:41:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:41:41,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:41:41,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:41:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:41,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:41:41,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:41,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:51 UTC)" executed successfully 2020-03-31 09:41:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:41:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:33.366791+00:00 (in 49.825978 seconds) 2020-03-31 09:41:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:43 UTC)" (scheduled at 2020-03-31 09:41:43.539999+00:00) 2020-03-31 09:41:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:41:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:33.366791+00:00 (in 49.825978 seconds) 2020-03-31 09:41:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:43 UTC)" (scheduled at 2020-03-31 09:41:43.539999+00:00) 2020-03-31 09:41:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:41:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:43 UTC)" executed successfully 2020-03-31 09:41:44,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:41:44,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:54 UTC)" (scheduled at 2020-03-31 09:41:44.323352+00:00) 2020-03-31 09:41:44,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:41:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:54 UTC)" (scheduled at 2020-03-31 09:41:44.323352+00:00) 2020-03-31 09:41:44,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:44,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:49.498966+00:00 (in 5.172451 seconds) notificationworker stdout | 2020-03-31 09:41:44,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:49.498966+00:00 (in 5.172451 seconds) 2020-03-31 09:41:44,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 44, 327499), True, datetime.datetime(2020, 3, 31, 9, 41, 44, 327499), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:41:44,330 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 44, 327499), True, datetime.datetime(2020, 3, 31, 9, 41, 44, 327499), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:41:44,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:41:44,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:41:44,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:44,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:54 UTC)" executed successfully 2020-03-31 09:41:44,446 [518] [DEBUG] [app] Starting request: urn:request:d4901242-9e90-4c89-a024-e63ecb832d7a (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:44,446 [518] [DEBUG] [app] Starting request: urn:request:d4901242-9e90-4c89-a024-e63ecb832d7a (/health/instance) 2020-03-31 09:41:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:44,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:44,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:44,454 [538] [DEBUG] [app] Starting request: urn:request:16268c0c-2395-4397-ab08-3a369d217754 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:44,454 [538] [DEBUG] [app] Starting request: urn:request:16268c0c-2395-4397-ab08-3a369d217754 (/v1/_internal_ping) 2020-03-31 09:41:44,455 [538] [DEBUG] [app] Ending request: urn:request:16268c0c-2395-4397-ab08-3a369d217754 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:44,455 [538] [DEBUG] [app] Ending request: urn:request:16268c0c-2395-4397-ab08-3a369d217754 (/v1/_internal_ping) 2020-03-31 09:41:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:41:44,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:41:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:41:44,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:44,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:44,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:44,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:44,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:44,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:44,491 [526] [DEBUG] [app] Starting request: urn:request:10f79d6a-087a-48ba-931a-0046a4048aa3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:44,491 [526] [DEBUG] [app] Starting request: urn:request:10f79d6a-087a-48ba-931a-0046a4048aa3 (/_internal_ping) 2020-03-31 09:41:44,492 [526] [DEBUG] [app] Ending request: urn:request:10f79d6a-087a-48ba-931a-0046a4048aa3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:44,492 [526] [DEBUG] [app] Ending request: urn:request:10f79d6a-087a-48ba-931a-0046a4048aa3 (/_internal_ping) 2020-03-31 09:41:44,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:41:44,493 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:41:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:41:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:41:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:44,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:44,506 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:44,507 [535] [DEBUG] [app] Starting request: urn:request:67667132-78d4-42df-bf15-6c5ba5bf788d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:44,507 [535] [DEBUG] [app] Starting request: urn:request:67667132-78d4-42df-bf15-6c5ba5bf788d (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:44,508 [535] [DEBUG] [app] Ending request: urn:request:67667132-78d4-42df-bf15-6c5ba5bf788d (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:44,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:44,509 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:41:44,508 [535] [DEBUG] [app] Ending request: urn:request:67667132-78d4-42df-bf15-6c5ba5bf788d (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:44,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:41:44,509 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:44,511 [518] [DEBUG] [app] Ending request: urn:request:d4901242-9e90-4c89-a024-e63ecb832d7a (/health/instance) 2020-03-31 09:41:44,511 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:44,511 [518] [DEBUG] [app] Ending request: urn:request:d4901242-9e90-4c89-a024-e63ecb832d7a (/health/instance) 2020-03-31 09:41:44,511 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:41:44,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:41:44,512 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:41:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:41:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:15.064514+00:00 (in 29.999041 seconds) 2020-03-31 09:41:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:15 UTC)" (scheduled at 2020-03-31 09:41:45.064514+00:00) gcworker stdout | 2020-03-31 09:41:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:15.064514+00:00 (in 29.999041 seconds) 2020-03-31 09:41:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:15 UTC)" (scheduled at 2020-03-31 09:41:45.064514+00:00) 2020-03-31 09:41:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438105, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:41:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438105, 1209600, 500, 1, 0]) 2020-03-31 09:41:45,090 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:41:45,090 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:41:45,090 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:45,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:41:45,090 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:45,090 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:15 UTC)" executed successfully 2020-03-31 09:41:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:41:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:56.924497+00:00 (in 9.999187 seconds) 2020-03-31 09:41:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:56 UTC)" (scheduled at 2020-03-31 09:41:46.924497+00:00) 2020-03-31 09:41:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 46, 926178), True, datetime.datetime(2020, 3, 31, 9, 41, 46, 926178), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:41:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:56.924497+00:00 (in 9.999187 seconds) 2020-03-31 09:41:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:56 UTC)" (scheduled at 2020-03-31 09:41:46.924497+00:00) 2020-03-31 09:41:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:41:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 46, 926178), True, datetime.datetime(2020, 3, 31, 9, 41, 46, 926178), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:41:46,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:41:46,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:41:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:56 UTC)" executed successfully 2020-03-31 09:41:48,272 [518] [DEBUG] [app] Starting request: urn:request:8efcc97e-d000-4891-81e5-5319e366093d (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:48,272 [518] [DEBUG] [app] Starting request: urn:request:8efcc97e-d000-4891-81e5-5319e366093d (/health/instance) 2020-03-31 09:41:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:48,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:48,277 [553] [DEBUG] [app] Starting request: urn:request:3f8bde77-a1ec-4751-8663-a67b1fce6dd5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:48,277 [553] [DEBUG] [app] Starting request: urn:request:3f8bde77-a1ec-4751-8663-a67b1fce6dd5 (/v1/_internal_ping) 2020-03-31 09:41:48,278 [553] [DEBUG] [app] Ending request: urn:request:3f8bde77-a1ec-4751-8663-a67b1fce6dd5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:48,278 [553] [DEBUG] [app] Ending request: urn:request:3f8bde77-a1ec-4751-8663-a67b1fce6dd5 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:48,279 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:48,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:41:48,279 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:48,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:48,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:48,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:48,281 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:41:48,281 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:48,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:48,283 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:48,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:48,284 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:48,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:48,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:48,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:48,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:48,313 [527] [DEBUG] [app] Starting request: urn:request:9f88c3c8-86de-4098-a0f2-d911c59bd83f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:48,313 [527] [DEBUG] [app] Starting request: urn:request:9f88c3c8-86de-4098-a0f2-d911c59bd83f (/_internal_ping) 2020-03-31 09:41:48,314 [527] [DEBUG] [app] Ending request: urn:request:9f88c3c8-86de-4098-a0f2-d911c59bd83f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:48,314 [527] [DEBUG] [app] Ending request: urn:request:9f88c3c8-86de-4098-a0f2-d911c59bd83f (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:41:48,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:41:48,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:48,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:41:48,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:48,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:41:48,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:41:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:48,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:48,327 [539] [DEBUG] [app] Starting request: urn:request:1947de06-aafa-403d-b35d-5c070416c3b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:48,327 [539] [DEBUG] [app] Starting request: urn:request:1947de06-aafa-403d-b35d-5c070416c3b1 (/c1/_internal_ping) 2020-03-31 09:41:48,328 [539] [DEBUG] [app] Ending request: urn:request:1947de06-aafa-403d-b35d-5c070416c3b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:48,328 [539] [DEBUG] [app] Ending request: urn:request:1947de06-aafa-403d-b35d-5c070416c3b1 (/c1/_internal_ping) 2020-03-31 09:41:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:48,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:41:48,328 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:48,330 [518] [DEBUG] [app] Ending request: urn:request:8efcc97e-d000-4891-81e5-5319e366093d (/health/instance) 2020-03-31 09:41:48,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:48,330 [518] [DEBUG] [app] Ending request: urn:request:8efcc97e-d000-4891-81e5-5319e366093d (/health/instance) 2020-03-31 09:41:48,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:41:48,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:41:49,503 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:41:49,503 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:49,504 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:54.323352+00:00 (in 4.819433 seconds) 2020-03-31 09:41:49,504 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:49 UTC)" (scheduled at 2020-03-31 09:41:49.498966+00:00) 2020-03-31 09:41:49,504 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:49,504 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:41:49,504 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:54.323352+00:00 (in 4.819433 seconds) 2020-03-31 09:41:49,504 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:49 UTC)" (scheduled at 2020-03-31 09:41:49.498966+00:00) 2020-03-31 09:41:49,504 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:49,504 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:49 UTC)" executed successfully 2020-03-31 09:41:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:41:51,112 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:59.560326+00:00 (in 8.447704 seconds) 2020-03-31 09:41:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:51 UTC)" (scheduled at 2020-03-31 09:41:51.110610+00:00) 2020-03-31 09:41:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:41:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:41:59.560326+00:00 (in 8.447704 seconds) 2020-03-31 09:41:51,113 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:51 UTC)" (scheduled at 2020-03-31 09:41:51.110610+00:00) 2020-03-31 09:41:51,113 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 51, 113773), True, datetime.datetime(2020, 3, 31, 9, 41, 51, 113773), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:41:51,116 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 51, 113773), True, datetime.datetime(2020, 3, 31, 9, 41, 51, 113773), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:41:51,132 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:41:51,132 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:51,133 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:41:51,133 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:51,133 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:51 UTC)" executed successfully 2020-03-31 09:41:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:41:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:01.440707+00:00 (in 9.996010 seconds) 2020-03-31 09:41:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:01 UTC)" (scheduled at 2020-03-31 09:41:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:41:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:01.440707+00:00 (in 9.996010 seconds) 2020-03-31 09:41:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:01 UTC)" (scheduled at 2020-03-31 09:41:51.440707+00:00) 2020-03-31 09:41:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:41:51,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:41:51,467 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:41:51,467 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:41:51,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:51,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:41:51,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:51,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:01 UTC)" executed successfully 2020-03-31 09:41:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:41:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:22.113221+00:00 (in 29.999181 seconds) 2020-03-31 09:41:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:22 UTC)" (scheduled at 2020-03-31 09:41:52.113221+00:00) securityworker stdout | 2020-03-31 09:41:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:22.113221+00:00 (in 29.999181 seconds) 2020-03-31 09:41:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:22 UTC)" (scheduled at 2020-03-31 09:41:52.113221+00:00) 2020-03-31 09:41:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:41:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:41:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:41:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:41:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:41:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:41:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:41:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:22 UTC)" executed successfully 2020-03-31 09:41:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:41:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:04.323352+00:00 (in 9.998912 seconds) 2020-03-31 09:41:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:04 UTC)" (scheduled at 2020-03-31 09:41:54.323352+00:00) 2020-03-31 09:41:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:41:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:04.323352+00:00 (in 9.998912 seconds) 2020-03-31 09:41:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:04 UTC)" (scheduled at 2020-03-31 09:41:54.323352+00:00) 2020-03-31 09:41:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 54, 325526), True, datetime.datetime(2020, 3, 31, 9, 41, 54, 325526), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:41:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 54, 325526), True, datetime.datetime(2020, 3, 31, 9, 41, 54, 325526), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:41:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:41:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:41:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:04 UTC)" executed successfully 2020-03-31 09:41:54,446 [518] [DEBUG] [app] Starting request: urn:request:7ffad798-71aa-4c48-aeed-fe5bc4d8dfe0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:54,446 [518] [DEBUG] [app] Starting request: urn:request:7ffad798-71aa-4c48-aeed-fe5bc4d8dfe0 (/health/instance) 2020-03-31 09:41:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:54,455 [556] [DEBUG] [app] Starting request: urn:request:8cac4790-3ed1-4235-bb09-92159a358992 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:54,455 [556] [DEBUG] [app] Starting request: urn:request:8cac4790-3ed1-4235-bb09-92159a358992 (/v1/_internal_ping) 2020-03-31 09:41:54,456 [556] [DEBUG] [app] Ending request: urn:request:8cac4790-3ed1-4235-bb09-92159a358992 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:54,456 [556] [DEBUG] [app] Ending request: urn:request:8cac4790-3ed1-4235-bb09-92159a358992 (/v1/_internal_ping) 2020-03-31 09:41:54,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:41:54,457 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:41:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:54,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:41:54,460 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:54,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:54,461 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:54,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:54,462 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:54,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:54,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:54,485 [527] [DEBUG] [app] Starting request: urn:request:19e12d35-e59b-4867-9b7c-1330cab3918a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:54,485 [527] [DEBUG] [app] Starting request: urn:request:19e12d35-e59b-4867-9b7c-1330cab3918a (/_internal_ping) 2020-03-31 09:41:54,486 [527] [DEBUG] [app] Ending request: urn:request:19e12d35-e59b-4867-9b7c-1330cab3918a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:54,486 [527] [DEBUG] [app] Ending request: urn:request:19e12d35-e59b-4867-9b7c-1330cab3918a (/_internal_ping) 2020-03-31 09:41:54,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:41:54,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:41:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:41:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:41:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:41:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:41:54,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:54,500 [539] [DEBUG] [app] Starting request: urn:request:0e029acc-0aa6-4cb7-9e4c-f25bf7ca63fc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:54,500 [539] [DEBUG] [app] Starting request: urn:request:0e029acc-0aa6-4cb7-9e4c-f25bf7ca63fc (/c1/_internal_ping) 2020-03-31 09:41:54,501 [539] [DEBUG] [app] Ending request: urn:request:0e029acc-0aa6-4cb7-9e4c-f25bf7ca63fc (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:54,501 [539] [DEBUG] [app] Ending request: urn:request:0e029acc-0aa6-4cb7-9e4c-f25bf7ca63fc (/c1/_internal_ping) 2020-03-31 09:41:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:41:54,502 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:41:54,502 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:54,504 [518] [DEBUG] [app] Ending request: urn:request:7ffad798-71aa-4c48-aeed-fe5bc4d8dfe0 (/health/instance) 2020-03-31 09:41:54,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:41:54,504 [518] [DEBUG] [app] Ending request: urn:request:7ffad798-71aa-4c48-aeed-fe5bc4d8dfe0 (/health/instance) 2020-03-31 09:41:54,504 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:41:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:41:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:41:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:41:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:56 UTC)" (scheduled at 2020-03-31 09:41:56.924497+00:00) 2020-03-31 09:41:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:41:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:41:56 UTC)" (scheduled at 2020-03-31 09:41:56.924497+00:00) 2020-03-31 09:41:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:41:56,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:03.010318+00:00 (in 6.084786 seconds) storagereplication stdout | 2020-03-31 09:41:56,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:03.010318+00:00 (in 6.084786 seconds) 2020-03-31 09:41:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 56, 926263), True, datetime.datetime(2020, 3, 31, 9, 41, 56, 926263), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:41:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 41, 56, 926263), True, datetime.datetime(2020, 3, 31, 9, 41, 56, 926263), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:41:56,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:41:56,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:41:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:41:56,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:06 UTC)" executed successfully 2020-03-31 09:41:58,272 [518] [DEBUG] [app] Starting request: urn:request:b6ac0bcc-0a3d-4a4c-bab7-134d80ccb517 (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:58,272 [518] [DEBUG] [app] Starting request: urn:request:b6ac0bcc-0a3d-4a4c-bab7-134d80ccb517 (/health/instance) 2020-03-31 09:41:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:58,278 [538] [DEBUG] [app] Starting request: urn:request:7cfc5724-f431-4f29-896a-c60cb71194f3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:58,278 [538] [DEBUG] [app] Starting request: urn:request:7cfc5724-f431-4f29-896a-c60cb71194f3 (/v1/_internal_ping) 2020-03-31 09:41:58,279 [538] [DEBUG] [app] Ending request: urn:request:7cfc5724-f431-4f29-896a-c60cb71194f3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:41:58,279 [538] [DEBUG] [app] Ending request: urn:request:7cfc5724-f431-4f29-896a-c60cb71194f3 (/v1/_internal_ping) 2020-03-31 09:41:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:41:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:41:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:41:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:41:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:41:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:41:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:41:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:41:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:41:58,285 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:41:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:41:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:41:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:41:58,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:41:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:58,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:58,314 [527] [DEBUG] [app] Starting request: urn:request:3ca17ed6-b789-4bd3-ac2d-7856612f58a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:58,314 [527] [DEBUG] [app] Starting request: urn:request:3ca17ed6-b789-4bd3-ac2d-7856612f58a0 (/_internal_ping) 2020-03-31 09:41:58,315 [527] [DEBUG] [app] Ending request: urn:request:3ca17ed6-b789-4bd3-ac2d-7856612f58a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:41:58,315 [527] [DEBUG] [app] Ending request: urn:request:3ca17ed6-b789-4bd3-ac2d-7856612f58a0 (/_internal_ping) 2020-03-31 09:41:58,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:41:58,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:41:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:41:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:41:58,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:41:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:41:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:41:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:41:58,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:41:58,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:41:58,333 [535] [DEBUG] [app] Starting request: urn:request:4ca9e3d2-3742-41c5-aa7a-ea6deec8dc13 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:58,333 [535] [DEBUG] [app] Starting request: urn:request:4ca9e3d2-3742-41c5-aa7a-ea6deec8dc13 (/c1/_internal_ping) 2020-03-31 09:41:58,334 [535] [DEBUG] [app] Ending request: urn:request:4ca9e3d2-3742-41c5-aa7a-ea6deec8dc13 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:41:58,334 [535] [DEBUG] [app] Ending request: urn:request:4ca9e3d2-3742-41c5-aa7a-ea6deec8dc13 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:41:58,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:41:58,335 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:41:58,335 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:41:58,335 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:41:58,336 [518] [DEBUG] [app] Ending request: urn:request:b6ac0bcc-0a3d-4a4c-bab7-134d80ccb517 (/health/instance) gunicorn-web stdout | 2020-03-31 09:41:58,336 [518] [DEBUG] [app] Ending request: urn:request:b6ac0bcc-0a3d-4a4c-bab7-134d80ccb517 (/health/instance) 2020-03-31 09:41:58,336 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:41:58,336 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:41:58,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:41:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:41:58,337 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:41:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:41:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:41:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:41:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:51.110610+00:00 (in 51.549390 seconds) 2020-03-31 09:41:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:59 UTC)" (scheduled at 2020-03-31 09:41:59.560326+00:00) 2020-03-31 09:41:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:41:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:51.110610+00:00 (in 51.549390 seconds) 2020-03-31 09:41:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:59 UTC)" (scheduled at 2020-03-31 09:41:59.560326+00:00) 2020-03-31 09:41:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:41:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:59 UTC)" executed successfully 2020-03-31 09:42:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:42:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:30.337699+00:00 (in 29.998945 seconds) 2020-03-31 09:42:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:30 UTC)" (scheduled at 2020-03-31 09:42:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:42:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:30.337699+00:00 (in 29.998945 seconds) 2020-03-31 09:42:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:30 UTC)" (scheduled at 2020-03-31 09:42:00.337699+00:00) 2020-03-31 09:42:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 42, 0, 339500), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:42:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 42, 0, 339500), False, 50, 1, 0]) 2020-03-31 09:42:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:42:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:42:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:42:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:00,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:30 UTC)" executed successfully 2020-03-31 09:42:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:42:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:11.440707+00:00 (in 9.999156 seconds) 2020-03-31 09:42:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:11 UTC)" (scheduled at 2020-03-31 09:42:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:42:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:11.440707+00:00 (in 9.999156 seconds) 2020-03-31 09:42:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:11 UTC)" (scheduled at 2020-03-31 09:42:01.440707+00:00) 2020-03-31 09:42:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:42:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:42:01,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:42:01,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:42:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:01,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:42:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:01,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:11 UTC)" executed successfully 2020-03-31 09:42:03,012 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:03,012 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:03,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:06.924497+00:00 (in 3.911763 seconds) 2020-03-31 09:42:03,013 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:03 UTC)" (scheduled at 2020-03-31 09:42:03.010318+00:00) storagereplication stdout | 2020-03-31 09:42:03,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:06.924497+00:00 (in 3.911763 seconds) 2020-03-31 09:42:03,013 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:03 UTC)" (scheduled at 2020-03-31 09:42:03.010318+00:00) 2020-03-31 09:42:03,014 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:42:03,014 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), u'imagestoragereplication/%']) 2020-03-31 09:42:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), True, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:42:03,028 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), True, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), 0, u'imagestoragereplication/%']) 2020-03-31 09:42:03,032 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), True, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:42:03,032 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), True, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 42, 3, 13495), u'imagestoragereplication/%']) 2020-03-31 09:42:03,035 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:42:03,035 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:03,036 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:03 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:42:03,036 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:03 UTC)" executed successfully 2020-03-31 09:42:03,038 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:42:03,038 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:03,039 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:42:03,039 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 09:42:03,040 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:42:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:42:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:24.410927+00:00 (in 21.079699 seconds) 2020-03-31 09:42:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:33 UTC)" (scheduled at 2020-03-31 09:42:03.330112+00:00) 2020-03-31 09:42:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:42:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:24.410927+00:00 (in 21.079699 seconds) 2020-03-31 09:42:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:33 UTC)" (scheduled at 2020-03-31 09:42:03.330112+00:00) 2020-03-31 09:42:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 3, 332079), True, datetime.datetime(2020, 3, 31, 9, 42, 3, 332079), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:42:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 3, 332079), True, datetime.datetime(2020, 3, 31, 9, 42, 3, 332079), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:42:03,354 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:42:03,354 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:42:03,354 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:03,354 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:33 UTC)" executed successfully 2020-03-31 09:42:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:14.323352+00:00 (in 9.999002 seconds) 2020-03-31 09:42:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:14 UTC)" (scheduled at 2020-03-31 09:42:04.323352+00:00) 2020-03-31 09:42:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:42:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:14.323352+00:00 (in 9.999002 seconds) 2020-03-31 09:42:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:14 UTC)" (scheduled at 2020-03-31 09:42:04.323352+00:00) 2020-03-31 09:42:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 4, 325352), True, datetime.datetime(2020, 3, 31, 9, 42, 4, 325352), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:42:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 4, 325352), True, datetime.datetime(2020, 3, 31, 9, 42, 4, 325352), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:42:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:04,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:42:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:14 UTC)" executed successfully 2020-03-31 09:42:04,446 [527] [DEBUG] [app] Starting request: urn:request:08115344-223c-4248-8917-6d14c4b9ee3d (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:04,446 [527] [DEBUG] [app] Starting request: urn:request:08115344-223c-4248-8917-6d14c4b9ee3d (/health/instance) 2020-03-31 09:42:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:04,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:04,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:04,454 [540] [DEBUG] [app] Starting request: urn:request:ded26394-633d-4ea7-b0a6-b80e9be5103e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:04,454 [540] [DEBUG] [app] Starting request: urn:request:ded26394-633d-4ea7-b0a6-b80e9be5103e (/v1/_internal_ping) 2020-03-31 09:42:04,455 [540] [DEBUG] [app] Ending request: urn:request:ded26394-633d-4ea7-b0a6-b80e9be5103e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:04,455 [540] [DEBUG] [app] Ending request: urn:request:ded26394-633d-4ea7-b0a6-b80e9be5103e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:04,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:42:04,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:04,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:04,458 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:42:04,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:42:04,458 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:04,459 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:04,459 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:04,460 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:04,460 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:04,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:04,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:04,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:04,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:04,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:04,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:04,484 [527] [DEBUG] [app] Starting request: urn:request:a94c4be8-ea62-4c67-83c2-f382393ab2e7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:04,484 [527] [DEBUG] [app] Starting request: urn:request:a94c4be8-ea62-4c67-83c2-f382393ab2e7 (/_internal_ping) 2020-03-31 09:42:04,485 [527] [DEBUG] [app] Ending request: urn:request:a94c4be8-ea62-4c67-83c2-f382393ab2e7 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:04,485 [527] [DEBUG] [app] Ending request: urn:request:a94c4be8-ea62-4c67-83c2-f382393ab2e7 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:42:04,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:04,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:42:04,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:04,486 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:04,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:04,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:42:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:42:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:04,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:42:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:42:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:04,498 [535] [DEBUG] [app] Starting request: urn:request:0c717e5f-612f-42f2-800e-cb0bd62e7a2f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:04,498 [535] [DEBUG] [app] Starting request: urn:request:0c717e5f-612f-42f2-800e-cb0bd62e7a2f (/c1/_internal_ping) 2020-03-31 09:42:04,498 [535] [DEBUG] [app] Ending request: urn:request:0c717e5f-612f-42f2-800e-cb0bd62e7a2f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:04,498 [535] [DEBUG] [app] Ending request: urn:request:0c717e5f-612f-42f2-800e-cb0bd62e7a2f (/c1/_internal_ping) 2020-03-31 09:42:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:04,499 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:04,501 [527] [DEBUG] [app] Ending request: urn:request:08115344-223c-4248-8917-6d14c4b9ee3d (/health/instance) 2020-03-31 09:42:04,501 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:42:04,499 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:04,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:42:04,501 [527] [DEBUG] [app] Ending request: urn:request:08115344-223c-4248-8917-6d14c4b9ee3d (/health/instance) 2020-03-31 09:42:04,501 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:04,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:42:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:16.924497+00:00 (in 9.998909 seconds) 2020-03-31 09:42:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:16 UTC)" (scheduled at 2020-03-31 09:42:06.924497+00:00) 2020-03-31 09:42:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:42:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:16.924497+00:00 (in 9.998909 seconds) 2020-03-31 09:42:06,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:16 UTC)" (scheduled at 2020-03-31 09:42:06.924497+00:00) 2020-03-31 09:42:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 6, 926341), True, datetime.datetime(2020, 3, 31, 9, 42, 6, 926341), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:42:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 6, 926341), True, datetime.datetime(2020, 3, 31, 9, 42, 6, 926341), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:42:06,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:42:06,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:06,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:06,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:42:06,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:06,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:16 UTC)" executed successfully 2020-03-31 09:42:08,273 [526] [DEBUG] [app] Starting request: urn:request:df2479f6-853d-41df-8292-30c9b311a3be (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:08,273 [526] [DEBUG] [app] Starting request: urn:request:df2479f6-853d-41df-8292-30c9b311a3be (/health/instance) 2020-03-31 09:42:08,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:08,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:08,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:08,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:08,282 [550] [DEBUG] [app] Starting request: urn:request:d611f701-e51f-41d4-9646-fbcf075bc65f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:08,282 [550] [DEBUG] [app] Starting request: urn:request:d611f701-e51f-41d4-9646-fbcf075bc65f (/v1/_internal_ping) 2020-03-31 09:42:08,283 [550] [DEBUG] [app] Ending request: urn:request:d611f701-e51f-41d4-9646-fbcf075bc65f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:08,283 [550] [DEBUG] [app] Ending request: urn:request:d611f701-e51f-41d4-9646-fbcf075bc65f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:08,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:08,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:42:08,284 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:08,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:08,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:08,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:42:08,286 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:08,286 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:08,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:08,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:08,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:08,288 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:08,295 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:08,295 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:08,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:08,310 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:08,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:08,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:08,316 [518] [DEBUG] [app] Starting request: urn:request:88b963ba-76b2-42da-af81-1dc9c76a4ec0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:08,316 [518] [DEBUG] [app] Starting request: urn:request:88b963ba-76b2-42da-af81-1dc9c76a4ec0 (/_internal_ping) 2020-03-31 09:42:08,317 [518] [DEBUG] [app] Ending request: urn:request:88b963ba-76b2-42da-af81-1dc9c76a4ec0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:08,317 [518] [DEBUG] [app] Ending request: urn:request:88b963ba-76b2-42da-af81-1dc9c76a4ec0 (/_internal_ping) 2020-03-31 09:42:08,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:08,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:42:08,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:08,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:42:08,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:08,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:08,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:42:08,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:42:08,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:08,324 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:42:08,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:08,326 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:08,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:08,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:08,330 [539] [DEBUG] [app] Starting request: urn:request:e4aa9e6d-6c9f-41bc-be6a-52469ba64592 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:08,330 [539] [DEBUG] [app] Starting request: urn:request:e4aa9e6d-6c9f-41bc-be6a-52469ba64592 (/c1/_internal_ping) 2020-03-31 09:42:08,331 [539] [DEBUG] [app] Ending request: urn:request:e4aa9e6d-6c9f-41bc-be6a-52469ba64592 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:08,331 [539] [DEBUG] [app] Ending request: urn:request:e4aa9e6d-6c9f-41bc-be6a-52469ba64592 (/c1/_internal_ping) 2020-03-31 09:42:08,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:08,332 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:08,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:42:08,332 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:08,333 [526] [DEBUG] [app] Ending request: urn:request:df2479f6-853d-41df-8292-30c9b311a3be (/health/instance) 2020-03-31 09:42:08,334 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:08,333 [526] [DEBUG] [app] Ending request: urn:request:df2479f6-853d-41df-8292-30c9b311a3be (/health/instance) 2020-03-31 09:42:08,334 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:08,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:42:08,334 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:42:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:42:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:21 UTC)" (scheduled at 2020-03-31 09:42:11.440707+00:00) 2020-03-31 09:42:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:21.440707+00:00 (in 9.998882 seconds) repositoryactioncounter stdout | 2020-03-31 09:42:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:21 UTC)" (scheduled at 2020-03-31 09:42:11.440707+00:00) 2020-03-31 09:42:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:21.440707+00:00 (in 9.998882 seconds) 2020-03-31 09:42:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:42:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:42:11,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:42:11,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:42:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:42:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:21 UTC)" executed successfully 2020-03-31 09:42:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:24.323352+00:00 (in 9.998955 seconds) 2020-03-31 09:42:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:24 UTC)" (scheduled at 2020-03-31 09:42:14.323352+00:00) 2020-03-31 09:42:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:42:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:24.323352+00:00 (in 9.998955 seconds) 2020-03-31 09:42:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:24 UTC)" (scheduled at 2020-03-31 09:42:14.323352+00:00) 2020-03-31 09:42:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 14, 325274), True, datetime.datetime(2020, 3, 31, 9, 42, 14, 325274), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:42:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 14, 325274), True, datetime.datetime(2020, 3, 31, 9, 42, 14, 325274), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:42:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:42:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:24 UTC)" executed successfully 2020-03-31 09:42:14,446 [527] [DEBUG] [app] Starting request: urn:request:4283bd1b-4eee-4927-8a82-b95e73cbfe9f (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:14,446 [527] [DEBUG] [app] Starting request: urn:request:4283bd1b-4eee-4927-8a82-b95e73cbfe9f (/health/instance) 2020-03-31 09:42:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:14,454 [556] [DEBUG] [app] Starting request: urn:request:ceca198c-3920-47fe-be3a-e8de2e93d472 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:14,454 [556] [DEBUG] [app] Starting request: urn:request:ceca198c-3920-47fe-be3a-e8de2e93d472 (/v1/_internal_ping) 2020-03-31 09:42:14,455 [556] [DEBUG] [app] Ending request: urn:request:ceca198c-3920-47fe-be3a-e8de2e93d472 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:14,455 [556] [DEBUG] [app] Ending request: urn:request:ceca198c-3920-47fe-be3a-e8de2e93d472 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:14,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:42:14,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:14,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:42:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:42:14,458 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:14,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:14,459 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:14,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:14,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:14,481 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:14,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:14,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:14,486 [520] [DEBUG] [app] Starting request: urn:request:3c4e7254-a75f-4888-8c9c-46cc864f96e2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:14,486 [520] [DEBUG] [app] Starting request: urn:request:3c4e7254-a75f-4888-8c9c-46cc864f96e2 (/_internal_ping) 2020-03-31 09:42:14,487 [520] [DEBUG] [app] Ending request: urn:request:3c4e7254-a75f-4888-8c9c-46cc864f96e2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:14,487 [520] [DEBUG] [app] Ending request: urn:request:3c4e7254-a75f-4888-8c9c-46cc864f96e2 (/_internal_ping) 2020-03-31 09:42:14,488 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:14,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:42:14,488 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:14,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:42:14,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:14,492 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:42:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:14,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:14,503 [535] [DEBUG] [app] Starting request: urn:request:f00cdff6-0f9a-4e3b-aebe-9562bbb41e9e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:14,503 [535] [DEBUG] [app] Starting request: urn:request:f00cdff6-0f9a-4e3b-aebe-9562bbb41e9e (/c1/_internal_ping) 2020-03-31 09:42:14,503 [535] [DEBUG] [app] Ending request: urn:request:f00cdff6-0f9a-4e3b-aebe-9562bbb41e9e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:14,503 [535] [DEBUG] [app] Ending request: urn:request:f00cdff6-0f9a-4e3b-aebe-9562bbb41e9e (/c1/_internal_ping) 2020-03-31 09:42:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:14,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:14,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:42:14,504 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:14,505 [527] [DEBUG] [app] Ending request: urn:request:4283bd1b-4eee-4927-8a82-b95e73cbfe9f (/health/instance) 2020-03-31 09:42:14,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:14,505 [527] [DEBUG] [app] Ending request: urn:request:4283bd1b-4eee-4927-8a82-b95e73cbfe9f (/health/instance) 2020-03-31 09:42:14,506 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:14,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:42:14,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:42:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:42:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:45.064514+00:00 (in 29.999242 seconds) 2020-03-31 09:42:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:45 UTC)" (scheduled at 2020-03-31 09:42:15.064514+00:00) gcworker stdout | 2020-03-31 09:42:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:45.064514+00:00 (in 29.999242 seconds) 2020-03-31 09:42:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:45 UTC)" (scheduled at 2020-03-31 09:42:15.064514+00:00) 2020-03-31 09:42:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438135, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:42:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438135, 1209600, 500, 1, 0]) 2020-03-31 09:42:15,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:42:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:42:15,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:42:15,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:45 UTC)" executed successfully 2020-03-31 09:42:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:26.924497+00:00 (in 9.999240 seconds) 2020-03-31 09:42:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:26 UTC)" (scheduled at 2020-03-31 09:42:16.924497+00:00) 2020-03-31 09:42:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:42:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:26.924497+00:00 (in 9.999240 seconds) 2020-03-31 09:42:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:26 UTC)" (scheduled at 2020-03-31 09:42:16.924497+00:00) 2020-03-31 09:42:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 16, 925960), True, datetime.datetime(2020, 3, 31, 9, 42, 16, 925960), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:42:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 16, 925960), True, datetime.datetime(2020, 3, 31, 9, 42, 16, 925960), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:42:16,943 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 2, 16, 925960), 1, u'804e56f9-1a08-4b9c-baa2-01a2e8b99a78', 3, u'6a7f39c7-7513-442a-bec5-4d3b37ffac5c']) storagereplication stdout | 2020-03-31 09:42:16,943 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 2, 16, 925960), 1, u'804e56f9-1a08-4b9c-baa2-01a2e8b99a78', 3, u'6a7f39c7-7513-442a-bec5-4d3b37ffac5c']) 2020-03-31 09:42:16,949 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} storagereplication stdout | 2020-03-31 09:42:16,949 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:42:16,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:16,949 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 storagereplication stdout | 2020-03-31 09:42:16,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:16,949 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 2020-03-31 09:42:16,950 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:42:16,950 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:42:16,964 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:42:16,964 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:42:16,968 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) storagereplication stdout | 2020-03-31 09:42:16,968 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) 2020-03-31 09:42:16,973 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:42:16,973 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:42:16,973 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e8845b40-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:16 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:42:16,973 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8845b40-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8845b40-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ujw5eHlCIz3k/xP3H7hKG2Gh2tf/Jq38KQcjqZvUyx8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:16 GMT'}. storagereplication stdout | 2020-03-31 09:42:16,973 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e8845b40-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:16 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:42:16,973 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8845b40-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8845b40-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ujw5eHlCIz3k/xP3H7hKG2Gh2tf/Jq38KQcjqZvUyx8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:16 GMT'}. 2020-03-31 09:42:16,976 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:42:16,976 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:42:17,090 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:42:17,090 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:42:17,091 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8845b40-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=6e4dc53f-201e-0053-6140-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8845b40-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ujw5eHlCIz3k/xP3H7hKG2Gh2tf/Jq38KQcjqZvUyx8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:16 GMT'}. storagereplication stdout | 2020-03-31 09:42:17,091 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8845b40-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=6e4dc53f-201e-0053-6140-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8845b40-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ujw5eHlCIz3k/xP3H7hKG2Gh2tf/Jq38KQcjqZvUyx8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:16 GMT'}. 2020-03-31 09:42:17,092 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:42:17,092 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:42:17,092 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:42:17,092 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:17,093 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:42:17,093 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:e896a430-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=0oZ8BivWOKV6cGW12kZTKUjGcF3VaDRCNThTJM2UUxE%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A17Z x-ms-date:Tue, 31 Mar 2020 09:42:17 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:42:17,093 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e896a430-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=0oZ8BivWOKV6cGW12kZTKUjGcF3VaDRCNThTJM2UUxE%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FH/HaZ9XQ8mAExffbK7kJ6Ghp/qXq2GNf9dKuxZw9O0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e896a430-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. storagereplication stdout | 2020-03-31 09:42:17,093 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:42:17,093 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:e896a430-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=0oZ8BivWOKV6cGW12kZTKUjGcF3VaDRCNThTJM2UUxE%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A17Z x-ms-date:Tue, 31 Mar 2020 09:42:17 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:42:17,093 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e896a430-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=0oZ8BivWOKV6cGW12kZTKUjGcF3VaDRCNThTJM2UUxE%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FH/HaZ9XQ8mAExffbK7kJ6Ghp/qXq2GNf9dKuxZw9O0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e896a430-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. 2020-03-31 09:42:17,174 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:42:17,174 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 2020-03-31 09:42:17,175 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e896a430-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=6e4dc544-201e-0053-6340-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=0oZ8BivWOKV6cGW12kZTKUjGcF3VaDRCNThTJM2UUxE%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FH/HaZ9XQ8mAExffbK7kJ6Ghp/qXq2GNf9dKuxZw9O0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e896a430-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. storagereplication stdout | 2020-03-31 09:42:17,175 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e896a430-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=6e4dc544-201e-0053-6340-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=0oZ8BivWOKV6cGW12kZTKUjGcF3VaDRCNThTJM2UUxE%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A17Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:FH/HaZ9XQ8mAExffbK7kJ6Ghp/qXq2GNf9dKuxZw9O0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e896a430-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. 2020-03-31 09:42:17,175 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:42:17,176 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e8a34172-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:17 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:42:17,175 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:42:17,176 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e8a34172-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:17 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:42:17,176 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8a34172-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:JMCYpUyaX0zBE2QYl6N3cheoEd1ixHjncipGEby51Xc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. storagereplication stdout | 2020-03-31 09:42:17,176 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8a34172-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:JMCYpUyaX0zBE2QYl6N3cheoEd1ixHjncipGEby51Xc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. 2020-03-31 09:42:17,178 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:42:17,178 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:42:17,603 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:42:17,603 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:42:17,605 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=62ff4018-501e-0005-2d40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8a34172-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:JMCYpUyaX0zBE2QYl6N3cheoEd1ixHjncipGEby51Xc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. 2020-03-31 09:42:17,605 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=62ff4018-501e-0005-2d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:17,605 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=62ff4018-501e-0005-2d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:42:17,605 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=62ff4018-501e-0005-2d40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e8a34172-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:JMCYpUyaX0zBE2QYl6N3cheoEd1ixHjncipGEby51Xc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:17 GMT'}. 2020-03-31 09:42:17,605 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=62ff4018-501e-0005-2d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:17,605 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=e8a34172-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:16 GMT, Server-Request-ID=62ff4018-501e-0005-2d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:17,605 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:42:17,605 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:42:18,273 [518] [DEBUG] [app] Starting request: urn:request:b3ee1a49-3aae-4c36-ab78-f672b0a3007d (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:18,273 [518] [DEBUG] [app] Starting request: urn:request:b3ee1a49-3aae-4c36-ab78-f672b0a3007d (/health/instance) 2020-03-31 09:42:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:18,280 [555] [DEBUG] [app] Starting request: urn:request:27054fe0-e671-4d58-a8a6-f04d48dbee3b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:18,280 [555] [DEBUG] [app] Starting request: urn:request:27054fe0-e671-4d58-a8a6-f04d48dbee3b (/v1/_internal_ping) 2020-03-31 09:42:18,282 [555] [DEBUG] [app] Ending request: urn:request:27054fe0-e671-4d58-a8a6-f04d48dbee3b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:18,282 [555] [DEBUG] [app] Ending request: urn:request:27054fe0-e671-4d58-a8a6-f04d48dbee3b (/v1/_internal_ping) 2020-03-31 09:42:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:42:18,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:18,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:42:18,283 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:42:18,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:18,286 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:18,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:18,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:18,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:18,288 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:18,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:18,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:18,318 [518] [DEBUG] [app] Starting request: urn:request:69d2ebd1-49b5-4876-91c1-351d7395f73c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:18,318 [518] [DEBUG] [app] Starting request: urn:request:69d2ebd1-49b5-4876-91c1-351d7395f73c (/_internal_ping) 2020-03-31 09:42:18,319 [518] [DEBUG] [app] Ending request: urn:request:69d2ebd1-49b5-4876-91c1-351d7395f73c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:18,319 [518] [DEBUG] [app] Ending request: urn:request:69d2ebd1-49b5-4876-91c1-351d7395f73c (/_internal_ping) 2020-03-31 09:42:18,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:42:18,319 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:42:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:42:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:18,332 [541] [DEBUG] [app] Starting request: urn:request:88e785dc-26fc-4f49-9d59-9ac33dbff39b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:18,332 [541] [DEBUG] [app] Starting request: urn:request:88e785dc-26fc-4f49-9d59-9ac33dbff39b (/c1/_internal_ping) 2020-03-31 09:42:18,333 [541] [DEBUG] [app] Ending request: urn:request:88e785dc-26fc-4f49-9d59-9ac33dbff39b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:18,333 [541] [DEBUG] [app] Ending request: urn:request:88e785dc-26fc-4f49-9d59-9ac33dbff39b (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:18,333 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:42:18,333 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:18,335 [518] [DEBUG] [app] Ending request: urn:request:b3ee1a49-3aae-4c36-ab78-f672b0a3007d (/health/instance) 2020-03-31 09:42:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:42:18,335 [518] [DEBUG] [app] Ending request: urn:request:b3ee1a49-3aae-4c36-ab78-f672b0a3007d (/health/instance) 2020-03-31 09:42:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:42:19,607 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ea163dfc-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:19 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:42:19,607 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ea163dfc-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:19 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:42:19,607 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ea163dfc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4Q60rycolpvq5mbAdl4E01UtmEJerKdtS+KNV17PpAQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:19 GMT'}. storagereplication stdout | 2020-03-31 09:42:19,607 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ea163dfc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4Q60rycolpvq5mbAdl4E01UtmEJerKdtS+KNV17PpAQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:19 GMT'}. 2020-03-31 09:42:19,688 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:42:19,688 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:42:19,690 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:18 GMT, Server-Request-ID=62ff478c-501e-0005-0540-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ea163dfc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4Q60rycolpvq5mbAdl4E01UtmEJerKdtS+KNV17PpAQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:19 GMT'}. 2020-03-31 09:42:19,690 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:18 GMT, Server-Request-ID=62ff478c-501e-0005-0540-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:19,690 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:18 GMT, Server-Request-ID=62ff478c-501e-0005-0540-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:19,690 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:42:19,690 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:18 GMT, Server-Request-ID=62ff478c-501e-0005-0540-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ea163dfc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:4Q60rycolpvq5mbAdl4E01UtmEJerKdtS+KNV17PpAQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:19 GMT'}. 2020-03-31 09:42:19,690 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:18 GMT, Server-Request-ID=62ff478c-501e-0005-0540-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:19,690 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ea163dfc-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:18 GMT, Server-Request-ID=62ff478c-501e-0005-0540-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:19,690 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:42:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:42:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:31.440707+00:00 (in 9.998489 seconds) 2020-03-31 09:42:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:31 UTC)" (scheduled at 2020-03-31 09:42:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:42:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:31.440707+00:00 (in 9.998489 seconds) 2020-03-31 09:42:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:31 UTC)" (scheduled at 2020-03-31 09:42:21.440707+00:00) 2020-03-31 09:42:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:42:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:42:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:42:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:42:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:42:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:31 UTC)" executed successfully 2020-03-31 09:42:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:42:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:52.113221+00:00 (in 29.997255 seconds) 2020-03-31 09:42:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:52 UTC)" (scheduled at 2020-03-31 09:42:22.113221+00:00) securityworker stdout | 2020-03-31 09:42:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:52.113221+00:00 (in 29.997255 seconds) 2020-03-31 09:42:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:52 UTC)" (scheduled at 2020-03-31 09:42:22.113221+00:00) 2020-03-31 09:42:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:42:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:42:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:42:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:42:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:42:22,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:42:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:42:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:42:52 UTC)" executed successfully 2020-03-31 09:42:23,694 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ec85e02e-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:23 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:42:23,694 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ec85e02e-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:23 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:42:23,694 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ec85e02e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Sgue7PAxuK5Vhh/j1YoeaR/itwrQgInLVcSSnq2Oqls=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:23 GMT'}. storagereplication stdout | 2020-03-31 09:42:23,694 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ec85e02e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Sgue7PAxuK5Vhh/j1YoeaR/itwrQgInLVcSSnq2Oqls=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:23 GMT'}. 2020-03-31 09:42:23,775 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:42:23,775 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:42:23,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:23 GMT, Server-Request-ID=62ff55f3-501e-0005-5140-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ec85e02e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Sgue7PAxuK5Vhh/j1YoeaR/itwrQgInLVcSSnq2Oqls=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:23 GMT'}. 2020-03-31 09:42:23,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:23 GMT, Server-Request-ID=62ff55f3-501e-0005-5140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:42:23,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:23 GMT, Server-Request-ID=62ff55f3-501e-0005-5140-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ec85e02e-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Sgue7PAxuK5Vhh/j1YoeaR/itwrQgInLVcSSnq2Oqls=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:23 GMT'}. 2020-03-31 09:42:23,776 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:23 GMT, Server-Request-ID=62ff55f3-501e-0005-5140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:23,776 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:23 GMT, Server-Request-ID=62ff55f3-501e-0005-5140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:23,776 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:42:23,776 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ec85e02e-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:23 GMT, Server-Request-ID=62ff55f3-501e-0005-5140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:23,776 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:42:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:31.498397+00:00 (in 7.174223 seconds) 2020-03-31 09:42:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:34 UTC)" (scheduled at 2020-03-31 09:42:24.323352+00:00) 2020-03-31 09:42:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 24, 324984), True, datetime.datetime(2020, 3, 31, 9, 42, 24, 324984), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:42:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:31.498397+00:00 (in 7.174223 seconds) 2020-03-31 09:42:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:34 UTC)" (scheduled at 2020-03-31 09:42:24.323352+00:00) 2020-03-31 09:42:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:42:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 24, 324984), True, datetime.datetime(2020, 3, 31, 9, 42, 24, 324984), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:42:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:24,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:42:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:34 UTC)" executed successfully 2020-03-31 09:42:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:42:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:33.330112+00:00 (in 8.918393 seconds) 2020-03-31 09:42:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:24 UTC)" (scheduled at 2020-03-31 09:42:24.410927+00:00) 2020-03-31 09:42:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:42:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:33.330112+00:00 (in 8.918393 seconds) 2020-03-31 09:42:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:24 UTC)" (scheduled at 2020-03-31 09:42:24.410927+00:00) 2020-03-31 09:42:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:42:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:42:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:24 UTC)" executed successfully 2020-03-31 09:42:24,456 [556] [DEBUG] [app] Starting request: urn:request:a343ffc8-1741-4c52-b06d-e9b689ddb17b (/v1/_internal_ping) 2020-03-31 09:42:24,447 [518] [DEBUG] [app] Starting request: urn:request:9085fe0c-d3fc-4a9c-b606-c0eead3f256f (/health/instance) 2020-03-31 09:42:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-registry stdout | 2020-03-31 09:42:24,456 [556] [DEBUG] [app] Starting request: urn:request:a343ffc8-1741-4c52-b06d-e9b689ddb17b (/v1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:24,447 [518] [DEBUG] [app] Starting request: urn:request:9085fe0c-d3fc-4a9c-b606-c0eead3f256f (/health/instance) 2020-03-31 09:42:24,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:24,460 [556] [DEBUG] [app] Ending request: urn:request:a343ffc8-1741-4c52-b06d-e9b689ddb17b (/v1/_internal_ping) 2020-03-31 09:42:24,461 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.007 162 0.007) 2020-03-31 09:42:24,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:24,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:42:24,460 [556] [DEBUG] [app] Ending request: urn:request:a343ffc8-1741-4c52-b06d-e9b689ddb17b (/v1/_internal_ping) 2020-03-31 09:42:24,464 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:24,461 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.007 162 0.007) 2020-03-31 09:42:24,465 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:24,466 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:42:24,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:42:24,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:42:24,464 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:24,465 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:24,466 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:24,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:24,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:24,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:24,489 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:24,495 [527] [DEBUG] [app] Starting request: urn:request:f6310d7b-bc5a-4abb-8ee3-bb8d8f8a731d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:24,495 [527] [DEBUG] [app] Starting request: urn:request:f6310d7b-bc5a-4abb-8ee3-bb8d8f8a731d (/_internal_ping) 2020-03-31 09:42:24,497 [527] [DEBUG] [app] Ending request: urn:request:f6310d7b-bc5a-4abb-8ee3-bb8d8f8a731d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:24,497 [527] [DEBUG] [app] Ending request: urn:request:f6310d7b-bc5a-4abb-8ee3-bb8d8f8a731d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:42:24,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:42:24,497 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:24,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:24,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:24,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:24,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:42:24,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:42:24,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:24,510 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:24,512 [535] [DEBUG] [app] Starting request: urn:request:79f1f186-4645-4058-bb7c-2ee30dfac41c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:24,512 [535] [DEBUG] [app] Starting request: urn:request:79f1f186-4645-4058-bb7c-2ee30dfac41c (/c1/_internal_ping) 2020-03-31 09:42:24,513 [535] [DEBUG] [app] Ending request: urn:request:79f1f186-4645-4058-bb7c-2ee30dfac41c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:24,513 [535] [DEBUG] [app] Ending request: urn:request:79f1f186-4645-4058-bb7c-2ee30dfac41c (/c1/_internal_ping) 2020-03-31 09:42:24,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:24,516 [518] [DEBUG] [app] Ending request: urn:request:9085fe0c-d3fc-4a9c-b606-c0eead3f256f (/health/instance) 2020-03-31 09:42:24,516 [518] [DEBUG] [data.database] Disconnecting from database. 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:42:24,514 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:24,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:24,516 [518] [DEBUG] [app] Ending request: urn:request:9085fe0c-d3fc-4a9c-b606-c0eead3f256f (/health/instance) 2020-03-31 09:42:24,516 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:24,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 10.131.0.1 () - - [31/Mar/2020:09:42:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-verbs stdout | 2020-03-31 09:42:24,514 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:24,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:42:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:36.924497+00:00 (in 9.999286 seconds) storagereplication stdout | 2020-03-31 09:42:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:36.924497+00:00 (in 9.999286 seconds) 2020-03-31 09:42:28,273 [526] [DEBUG] [app] Starting request: urn:request:357c79a9-6a9e-428b-91bc-6beb42a238ba (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:28,273 [526] [DEBUG] [app] Starting request: urn:request:357c79a9-6a9e-428b-91bc-6beb42a238ba (/health/instance) 2020-03-31 09:42:28,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:28,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:28,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:28,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:28,283 [538] [DEBUG] [app] Starting request: urn:request:5364e422-90b2-425a-b39b-b731835bc2ce (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:28,283 [538] [DEBUG] [app] Starting request: urn:request:5364e422-90b2-425a-b39b-b731835bc2ce (/v1/_internal_ping) 2020-03-31 09:42:28,285 [538] [DEBUG] [app] Ending request: urn:request:5364e422-90b2-425a-b39b-b731835bc2ce (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:28,285 [538] [DEBUG] [app] Ending request: urn:request:5364e422-90b2-425a-b39b-b731835bc2ce (/v1/_internal_ping) 2020-03-31 09:42:28,285 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:42:28,286 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:42:28,285 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:42:28,287 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:28,286 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:28,288 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:42:28,287 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:42:28,288 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:28,290 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:28,290 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:28,291 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:28,291 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:28,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:28,298 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:28,314 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:28,314 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:28,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:28,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:28,320 [518] [DEBUG] [app] Starting request: urn:request:4de3d364-a5b9-4884-9bc4-e8d95cc904fe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:28,320 [518] [DEBUG] [app] Starting request: urn:request:4de3d364-a5b9-4884-9bc4-e8d95cc904fe (/_internal_ping) 2020-03-31 09:42:28,321 [518] [DEBUG] [app] Ending request: urn:request:4de3d364-a5b9-4884-9bc4-e8d95cc904fe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:28,321 [518] [DEBUG] [app] Ending request: urn:request:4de3d364-a5b9-4884-9bc4-e8d95cc904fe (/_internal_ping) 2020-03-31 09:42:28,322 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:42:28,322 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:28,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:42:28,322 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:28,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:28,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:28,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:42:28,327 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:42:28,331 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:28,331 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:42:28,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:28,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:28,335 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:28,335 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:28,337 [541] [DEBUG] [app] Starting request: urn:request:11046b66-6354-48d9-9589-a080a582d0d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:28,337 [541] [DEBUG] [app] Starting request: urn:request:11046b66-6354-48d9-9589-a080a582d0d6 (/c1/_internal_ping) 2020-03-31 09:42:28,338 [541] [DEBUG] [app] Ending request: urn:request:11046b66-6354-48d9-9589-a080a582d0d6 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:28,338 [541] [DEBUG] [app] Ending request: urn:request:11046b66-6354-48d9-9589-a080a582d0d6 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:28,339 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:28,340 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:42:28,339 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:28,340 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:28,342 [526] [DEBUG] [app] Ending request: urn:request:357c79a9-6a9e-428b-91bc-6beb42a238ba (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:28,342 [526] [DEBUG] [app] Ending request: urn:request:357c79a9-6a9e-428b-91bc-6beb42a238ba (/health/instance) 2020-03-31 09:42:28,343 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:28,344 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 09:42:28,343 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:28,344 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:42:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:42:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:00.337699+00:00 (in 29.999310 seconds) 2020-03-31 09:42:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:00 UTC)" (scheduled at 2020-03-31 09:42:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:42:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:00.337699+00:00 (in 29.999310 seconds) 2020-03-31 09:42:30,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:00 UTC)" (scheduled at 2020-03-31 09:42:30.337699+00:00) 2020-03-31 09:42:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 42, 30, 339106), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:42:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 42, 30, 339106), False, 50, 1, 0]) 2020-03-31 09:42:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:42:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:42:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:42:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:00 UTC)" executed successfully 2020-03-31 09:42:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:42:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:41.440707+00:00 (in 9.998969 seconds) 2020-03-31 09:42:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:41 UTC)" (scheduled at 2020-03-31 09:42:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:42:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:41.440707+00:00 (in 9.998969 seconds) 2020-03-31 09:42:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:41 UTC)" (scheduled at 2020-03-31 09:42:31.440707+00:00) 2020-03-31 09:42:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:42:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:42:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:42:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:42:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:42:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:41 UTC)" executed successfully 2020-03-31 09:42:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:34.323352+00:00 (in 2.824185 seconds) 2020-03-31 09:42:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:31 UTC)" (scheduled at 2020-03-31 09:42:31.498397+00:00) notificationworker stdout | 2020-03-31 09:42:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:34.323352+00:00 (in 2.824185 seconds) 2020-03-31 09:42:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:31 UTC)" (scheduled at 2020-03-31 09:42:31.498397+00:00) 2020-03-31 09:42:31,500 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), u'notification/%']) notificationworker stdout | 2020-03-31 09:42:31,500 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), u'notification/%']) 2020-03-31 09:42:31,518 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), True, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:42:31,518 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), True, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), 0, u'notification/%']) 2020-03-31 09:42:31,523 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), True, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), u'notification/%']) notificationworker stdout | 2020-03-31 09:42:31,523 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), True, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 42, 31, 499900), u'notification/%']) 2020-03-31 09:42:31,528 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:42:31,528 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:31,529 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:31,529 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:47:31 UTC)" executed successfully 2020-03-31 09:42:31,530 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:42:31,530 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:31,531 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:42:31,531 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:31,532 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics notificationworker stdout | 2020-03-31 09:42:31,532 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:31,533 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:42:31,533 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:31,534 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:42:31,534 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:31,535 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics notificationworker stdout | 2020-03-31 09:42:31,535 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:42:31,781 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:f157e4da-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:31 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:42:31,781 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:f157e4da-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:31 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:42:31,782 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f157e4da-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bpOgC5ycf9h01vRnBQTq7ZDpDVPK3BNJZ383pCv2uCE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:31 GMT'}. storagereplication stdout | 2020-03-31 09:42:31,782 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f157e4da-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bpOgC5ycf9h01vRnBQTq7ZDpDVPK3BNJZ383pCv2uCE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:31 GMT'}. 2020-03-31 09:42:31,862 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:42:31,862 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:42:31,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:31 GMT, Server-Request-ID=62ff72e8-501e-0005-6f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f157e4da-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bpOgC5ycf9h01vRnBQTq7ZDpDVPK3BNJZ383pCv2uCE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:31 GMT'}. 2020-03-31 09:42:31,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:31 GMT, Server-Request-ID=62ff72e8-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:31,863 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:31 GMT, Server-Request-ID=62ff72e8-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:42:31,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:31 GMT, Server-Request-ID=62ff72e8-501e-0005-6f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f157e4da-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:bpOgC5ycf9h01vRnBQTq7ZDpDVPK3BNJZ383pCv2uCE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:31 GMT'}. 2020-03-31 09:42:31,863 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:31 GMT, Server-Request-ID=62ff72e8-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:31,863 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=f157e4da-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:31 GMT, Server-Request-ID=62ff72e8-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:31,863 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:42:31,863 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:42:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:42:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:03.330112+00:00 (in 29.998967 seconds) 2020-03-31 09:42:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:03 UTC)" (scheduled at 2020-03-31 09:42:33.330112+00:00) 2020-03-31 09:42:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:42:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:03.330112+00:00 (in 29.998967 seconds) 2020-03-31 09:42:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:03 UTC)" (scheduled at 2020-03-31 09:42:33.330112+00:00) 2020-03-31 09:42:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 33, 332508), True, datetime.datetime(2020, 3, 31, 9, 42, 33, 332508), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:42:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 33, 332508), True, datetime.datetime(2020, 3, 31, 9, 42, 33, 332508), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:42:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:42:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:42:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:03 UTC)" executed successfully 2020-03-31 09:42:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:42:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:33 UTC)" (scheduled at 2020-03-31 09:42:33.366791+00:00) 2020-03-31 09:42:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:42:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:42:33 UTC)" (scheduled at 2020-03-31 09:42:33.366791+00:00) 2020-03-31 09:42:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:33,372 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 33, 368593), True, datetime.datetime(2020, 3, 31, 9, 42, 33, 368593), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:42:33,372 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 33, 368593), True, datetime.datetime(2020, 3, 31, 9, 42, 33, 368593), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:42:33,372 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:43.539999+00:00 (in 10.172112 seconds) exportactionlogsworker stdout | 2020-03-31 09:42:33,372 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:43.539999+00:00 (in 10.172112 seconds) 2020-03-31 09:42:33,387 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:42:33,387 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:33,388 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:33,388 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:42:33,388 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:33,388 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:33 UTC)" executed successfully 2020-03-31 09:42:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:44.323352+00:00 (in 9.999082 seconds) 2020-03-31 09:42:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:44 UTC)" (scheduled at 2020-03-31 09:42:34.323352+00:00) 2020-03-31 09:42:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:42:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:44.323352+00:00 (in 9.999082 seconds) 2020-03-31 09:42:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:44 UTC)" (scheduled at 2020-03-31 09:42:34.323352+00:00) 2020-03-31 09:42:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 34, 325171), True, datetime.datetime(2020, 3, 31, 9, 42, 34, 325171), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:42:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 34, 325171), True, datetime.datetime(2020, 3, 31, 9, 42, 34, 325171), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:42:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:42:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:44 UTC)" executed successfully 2020-03-31 09:42:34,446 [518] [DEBUG] [app] Starting request: urn:request:dda084f4-27f8-4ba4-822a-afd4153b9aae (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:34,446 [518] [DEBUG] [app] Starting request: urn:request:dda084f4-27f8-4ba4-822a-afd4153b9aae (/health/instance) 2020-03-31 09:42:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:34,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:34,454 [555] [DEBUG] [app] Starting request: urn:request:316da239-8928-4e61-a8a1-33f65b599b53 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:34,454 [555] [DEBUG] [app] Starting request: urn:request:316da239-8928-4e61-a8a1-33f65b599b53 (/v1/_internal_ping) 2020-03-31 09:42:34,456 [555] [DEBUG] [app] Ending request: urn:request:316da239-8928-4e61-a8a1-33f65b599b53 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:34,456 [555] [DEBUG] [app] Ending request: urn:request:316da239-8928-4e61-a8a1-33f65b599b53 (/v1/_internal_ping) 2020-03-31 09:42:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:42:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:42:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:42:34,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:34,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:34,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:34,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:34,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:34,462 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:34,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:34,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:34,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:34,487 [525] [DEBUG] [app] Starting request: urn:request:2a1b2ca7-961b-4f37-9edf-58c2ff83c87d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:34,487 [525] [DEBUG] [app] Starting request: urn:request:2a1b2ca7-961b-4f37-9edf-58c2ff83c87d (/_internal_ping) 2020-03-31 09:42:34,488 [525] [DEBUG] [app] Ending request: urn:request:2a1b2ca7-961b-4f37-9edf-58c2ff83c87d (/_internal_ping) 2020-03-31 09:42:34,489 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:42:34,488 [525] [DEBUG] [app] Ending request: urn:request:2a1b2ca7-961b-4f37-9edf-58c2ff83c87d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:34,489 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:42:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:42:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:42:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:42:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:34,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:34,502 [535] [DEBUG] [app] Starting request: urn:request:4c401956-b724-414b-98bb-8ec4f45ad370 (/c1/_internal_ping) 2020-03-31 09:42:34,503 [535] [DEBUG] [app] Ending request: urn:request:4c401956-b724-414b-98bb-8ec4f45ad370 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:34,503 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:34,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:42:34,502 [535] [DEBUG] [app] Starting request: urn:request:4c401956-b724-414b-98bb-8ec4f45ad370 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:34,503 [535] [DEBUG] [app] Ending request: urn:request:4c401956-b724-414b-98bb-8ec4f45ad370 (/c1/_internal_ping) 2020-03-31 09:42:34,505 [518] [DEBUG] [app] Ending request: urn:request:dda084f4-27f8-4ba4-822a-afd4153b9aae (/health/instance) 2020-03-31 09:42:34,505 [518] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:42:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:42:34,503 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:34,505 [518] [DEBUG] [app] Ending request: urn:request:dda084f4-27f8-4ba4-822a-afd4153b9aae (/health/instance) 2020-03-31 09:42:34,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:42:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:38.010934+00:00 (in 1.085672 seconds) storagereplication stdout | 2020-03-31 09:42:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:38.010934+00:00 (in 1.085672 seconds) 2020-03-31 09:42:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:46.924497+00:00 (in 8.912846 seconds) 2020-03-31 09:42:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:38 UTC)" (scheduled at 2020-03-31 09:42:38.010934+00:00) 2020-03-31 09:42:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:42:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:46.924497+00:00 (in 8.912846 seconds) 2020-03-31 09:42:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:38 UTC)" (scheduled at 2020-03-31 09:42:38.010934+00:00) 2020-03-31 09:42:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:42:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:42:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:38 UTC)" executed successfully 2020-03-31 09:42:38,273 [518] [DEBUG] [app] Starting request: urn:request:05cce831-35f1-406d-807b-9ae08f0e63e6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:38,273 [518] [DEBUG] [app] Starting request: urn:request:05cce831-35f1-406d-807b-9ae08f0e63e6 (/health/instance) 2020-03-31 09:42:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:38,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:38,278 [538] [DEBUG] [app] Starting request: urn:request:07f19f95-13ce-4759-8d75-95cc1682439a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:38,278 [538] [DEBUG] [app] Starting request: urn:request:07f19f95-13ce-4759-8d75-95cc1682439a (/v1/_internal_ping) 2020-03-31 09:42:38,279 [538] [DEBUG] [app] Ending request: urn:request:07f19f95-13ce-4759-8d75-95cc1682439a (/v1/_internal_ping) 2020-03-31 09:42:38,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:42:38,279 [538] [DEBUG] [app] Ending request: urn:request:07f19f95-13ce-4759-8d75-95cc1682439a (/v1/_internal_ping) 2020-03-31 09:42:38,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:38,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:42:38,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:42:38,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:38,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:38,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:38,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:38,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:38,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:38,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:38,312 [518] [DEBUG] [app] Starting request: urn:request:a05d6705-0656-496b-919d-b31ab9fe4f4a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:38,312 [518] [DEBUG] [app] Starting request: urn:request:a05d6705-0656-496b-919d-b31ab9fe4f4a (/_internal_ping) 2020-03-31 09:42:38,313 [518] [DEBUG] [app] Ending request: urn:request:a05d6705-0656-496b-919d-b31ab9fe4f4a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:38,313 [518] [DEBUG] [app] Ending request: urn:request:a05d6705-0656-496b-919d-b31ab9fe4f4a (/_internal_ping) 2020-03-31 09:42:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:42:38,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:38,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:42:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:38,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:38,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:38,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:38,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:42:38,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:38,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:38,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:38,325 [537] [DEBUG] [app] Starting request: urn:request:454648ab-687b-44a4-896d-c6f957919ba0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:38,325 [537] [DEBUG] [app] Starting request: urn:request:454648ab-687b-44a4-896d-c6f957919ba0 (/c1/_internal_ping) 2020-03-31 09:42:38,326 [537] [DEBUG] [app] Ending request: urn:request:454648ab-687b-44a4-896d-c6f957919ba0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:38,326 [537] [DEBUG] [app] Ending request: urn:request:454648ab-687b-44a4-896d-c6f957919ba0 (/c1/_internal_ping) 2020-03-31 09:42:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:38,327 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:38,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:42:38,327 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:38,328 [518] [DEBUG] [app] Ending request: urn:request:05cce831-35f1-406d-807b-9ae08f0e63e6 (/health/instance) 2020-03-31 09:42:38,329 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:38,328 [518] [DEBUG] [app] Ending request: urn:request:05cce831-35f1-406d-807b-9ae08f0e63e6 (/health/instance) 2020-03-31 09:42:38,329 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:38,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:42:38,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:42:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:42:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:51.440707+00:00 (in 9.998872 seconds) 2020-03-31 09:42:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:51 UTC)" (scheduled at 2020-03-31 09:42:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:42:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:51.440707+00:00 (in 9.998872 seconds) 2020-03-31 09:42:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:51 UTC)" (scheduled at 2020-03-31 09:42:41.440707+00:00) 2020-03-31 09:42:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:42:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:42:41,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:42:41,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:42:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:42:41,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:41,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:51 UTC)" executed successfully 2020-03-31 09:42:43,545 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:42:43,545 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:43,545 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:43 UTC)" (scheduled at 2020-03-31 09:42:43.539999+00:00) 2020-03-31 09:42:43,545 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:42:43,545 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:43 UTC)" (scheduled at 2020-03-31 09:42:43.539999+00:00) 2020-03-31 09:42:43,545 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:42:43,546 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:33.366791+00:00 (in 49.821516 seconds) 2020-03-31 09:42:43,546 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:42:43,546 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:33.366791+00:00 (in 49.821516 seconds) 2020-03-31 09:42:43,546 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:43 UTC)" executed successfully 2020-03-31 09:42:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:54 UTC)" (scheduled at 2020-03-31 09:42:44.323352+00:00) 2020-03-31 09:42:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:49.498966+00:00 (in 5.174547 seconds) 2020-03-31 09:42:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:42:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:54 UTC)" (scheduled at 2020-03-31 09:42:44.323352+00:00) 2020-03-31 09:42:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:49.498966+00:00 (in 5.174547 seconds) 2020-03-31 09:42:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 44, 325403), True, datetime.datetime(2020, 3, 31, 9, 42, 44, 325403), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:42:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 44, 325403), True, datetime.datetime(2020, 3, 31, 9, 42, 44, 325403), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:42:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:42:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:54 UTC)" executed successfully 2020-03-31 09:42:44,446 [527] [DEBUG] [app] Starting request: urn:request:d71b33e8-01ae-4e19-8465-638aa1c29f45 (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:44,446 [527] [DEBUG] [app] Starting request: urn:request:d71b33e8-01ae-4e19-8465-638aa1c29f45 (/health/instance) 2020-03-31 09:42:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:44,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:44,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:44,453 [540] [DEBUG] [app] Starting request: urn:request:0ae1776f-4bd3-40ef-a103-7c91ed598240 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:44,453 [540] [DEBUG] [app] Starting request: urn:request:0ae1776f-4bd3-40ef-a103-7c91ed598240 (/v1/_internal_ping) 2020-03-31 09:42:44,454 [540] [DEBUG] [app] Ending request: urn:request:0ae1776f-4bd3-40ef-a103-7c91ed598240 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:44,454 [540] [DEBUG] [app] Ending request: urn:request:0ae1776f-4bd3-40ef-a103-7c91ed598240 (/v1/_internal_ping) 2020-03-31 09:42:44,454 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:42:44,454 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:42:44,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:44,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:42:44,456 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:44,456 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:44,458 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:44,458 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:44,459 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:44,459 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:44,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:44,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:44,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:44,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:44,485 [527] [DEBUG] [app] Starting request: urn:request:7dfec3ce-5bc1-42ee-bc2b-3d41de062840 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:44,485 [527] [DEBUG] [app] Starting request: urn:request:7dfec3ce-5bc1-42ee-bc2b-3d41de062840 (/_internal_ping) 2020-03-31 09:42:44,486 [527] [DEBUG] [app] Ending request: urn:request:7dfec3ce-5bc1-42ee-bc2b-3d41de062840 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:44,486 [527] [DEBUG] [app] Ending request: urn:request:7dfec3ce-5bc1-42ee-bc2b-3d41de062840 (/_internal_ping) 2020-03-31 09:42:44,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:42:44,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:42:44,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:44,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:44,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:44,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:44,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:42:44,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:44,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:44,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:44,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:44,501 [535] [DEBUG] [app] Starting request: urn:request:38e2ca50-c5aa-416a-83dd-9be0d48f4f01 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:44,501 [535] [DEBUG] [app] Starting request: urn:request:38e2ca50-c5aa-416a-83dd-9be0d48f4f01 (/c1/_internal_ping) 2020-03-31 09:42:44,502 [535] [DEBUG] [app] Ending request: urn:request:38e2ca50-c5aa-416a-83dd-9be0d48f4f01 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:44,502 [535] [DEBUG] [app] Ending request: urn:request:38e2ca50-c5aa-416a-83dd-9be0d48f4f01 (/c1/_internal_ping) 2020-03-31 09:42:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:44,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:42:44,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:44,504 [527] [DEBUG] [app] Ending request: urn:request:d71b33e8-01ae-4e19-8465-638aa1c29f45 (/health/instance) 2020-03-31 09:42:44,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:44,504 [527] [DEBUG] [app] Ending request: urn:request:d71b33e8-01ae-4e19-8465-638aa1c29f45 (/health/instance) 2020-03-31 09:42:44,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:42:44,505 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:42:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:42:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:15.064514+00:00 (in 29.999179 seconds) 2020-03-31 09:42:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:15 UTC)" (scheduled at 2020-03-31 09:42:45.064514+00:00) gcworker stdout | 2020-03-31 09:42:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:15.064514+00:00 (in 29.999179 seconds) 2020-03-31 09:42:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:15 UTC)" (scheduled at 2020-03-31 09:42:45.064514+00:00) 2020-03-31 09:42:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438165, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:42:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438165, 1209600, 500, 1, 0]) 2020-03-31 09:42:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:42:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:42:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:42:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:15 UTC)" executed successfully 2020-03-31 09:42:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:56.924497+00:00 (in 9.999200 seconds) storagereplication stdout | 2020-03-31 09:42:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:56.924497+00:00 (in 9.999200 seconds) 2020-03-31 09:42:47,879 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:42:47,879 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy 2020-03-31 09:42:47,879 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:42:47,879 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:42:47,879 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:42:47,879 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:42:47,889 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) storagereplication stdout | 2020-03-31 09:42:47,889 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) 2020-03-31 09:42:47,894 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 47, 47, 888667), True, u'300d19df-aa55-45a0-bfa0-0120629460e0', 3]) storagereplication stdout | 2020-03-31 09:42:47,894 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 47, 47, 888667), True, u'300d19df-aa55-45a0-bfa0-0120629460e0', 3]) 2020-03-31 09:42:47,903 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 47, 902142), True, datetime.datetime(2020, 3, 31, 9, 42, 47, 902142), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:42:47,903 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 47, 902142), True, datetime.datetime(2020, 3, 31, 9, 42, 47, 902142), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:42:47,908 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 2, 47, 902142), 1, u'fdef917e-8bed-417e-b9a3-da47c6e76eae', 4, u'2239fc24-e783-4a19-a59b-554d003f4d3c']) storagereplication stdout | 2020-03-31 09:42:47,908 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 2, 47, 902142), 1, u'fdef917e-8bed-417e-b9a3-da47c6e76eae', 4, u'2239fc24-e783-4a19-a59b-554d003f4d3c']) 2020-03-31 09:42:47,914 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} storagereplication stdout | 2020-03-31 09:42:47,914 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:42:47,914 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:47,914 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 storagereplication stdout | 2020-03-31 09:42:47,914 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:47,914 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 2020-03-31 09:42:47,916 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:42:47,916 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:42:47,935 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:42:47,935 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:42:47,939 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) storagereplication stdout | 2020-03-31 09:42:47,939 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) 2020-03-31 09:42:47,943 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:42:47,943 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:42:47,943 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:fafa0716-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:42:47,944 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafa0716-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fafa0716-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ce2JFPBfpmL+aV7Nask8Vo4ZF59WXkrrszGyKzayVjk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. storagereplication stdout | 2020-03-31 09:42:47,943 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:fafa0716-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:42:47,944 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafa0716-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fafa0716-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ce2JFPBfpmL+aV7Nask8Vo4ZF59WXkrrszGyKzayVjk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. 2020-03-31 09:42:47,952 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:42:47,952 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:42:47,954 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafa0716-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:48 GMT, Server-Request-ID=6e4dd0d9-201e-0053-6f40-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fafa0716-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ce2JFPBfpmL+aV7Nask8Vo4ZF59WXkrrszGyKzayVjk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. storagereplication stdout | 2020-03-31 09:42:47,954 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafa0716-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:48 GMT, Server-Request-ID=6e4dd0d9-201e-0053-6f40-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fafa0716-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:ce2JFPBfpmL+aV7Nask8Vo4ZF59WXkrrszGyKzayVjk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. 2020-03-31 09:42:47,955 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us storagereplication stdout | 2020-03-31 09:42:47,955 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us 2020-03-31 09:42:47,955 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:47,955 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:42:47,955 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:47,955 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:42:47,956 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:fafbdeb0-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=V161uXGH8NAhYYkNyt1Vg1X7f6470XQi3KU%2BZOXt7aU%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A47Z x-ms-date:Tue, 31 Mar 2020 09:42:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:42:47,956 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafbdeb0-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=V161uXGH8NAhYYkNyt1Vg1X7f6470XQi3KU%2BZOXt7aU%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3pButkZff8JBYGjKYM5PONqVqMdcN048K+Iro+WdACI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'fafbdeb0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. storagereplication stdout | 2020-03-31 09:42:47,956 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:fafbdeb0-7333-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=V161uXGH8NAhYYkNyt1Vg1X7f6470XQi3KU%2BZOXt7aU%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A47Z x-ms-date:Tue, 31 Mar 2020 09:42:47 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:42:47,956 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafbdeb0-7333-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=V161uXGH8NAhYYkNyt1Vg1X7f6470XQi3KU%2BZOXt7aU%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3pButkZff8JBYGjKYM5PONqVqMdcN048K+Iro+WdACI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'fafbdeb0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. 2020-03-31 09:42:48,002 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:42:48,002 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 2020-03-31 09:42:48,003 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafbdeb0-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:48 GMT, Server-Request-ID=6e4dd0de-201e-0053-7240-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=V161uXGH8NAhYYkNyt1Vg1X7f6470XQi3KU%2BZOXt7aU%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3pButkZff8JBYGjKYM5PONqVqMdcN048K+Iro+WdACI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'fafbdeb0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. storagereplication stdout | 2020-03-31 09:42:48,003 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fafbdeb0-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:48 GMT, Server-Request-ID=6e4dd0de-201e-0053-7240-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=V161uXGH8NAhYYkNyt1Vg1X7f6470XQi3KU%2BZOXt7aU%3D&sv=2017-07-29&se=2020-03-31T09%3A43%3A47Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3pButkZff8JBYGjKYM5PONqVqMdcN048K+Iro+WdACI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'fafbdeb0-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:42:47 GMT'}. 2020-03-31 09:42:48,004 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:42:48,004 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:42:48,004 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:fb034cb8-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:48 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:42:48,004 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fb034cb8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tlw98EQW4poqadBaam34rMoQQ8DHiVH/ur/0htibhJc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:48 GMT'}. storagereplication stdout | 2020-03-31 09:42:48,004 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:fb034cb8-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:48 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:42:48,004 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fb034cb8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tlw98EQW4poqadBaam34rMoQQ8DHiVH/ur/0htibhJc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:48 GMT'}. 2020-03-31 09:42:48,088 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:42:48,088 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:42:48,089 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:47 GMT, Server-Request-ID=62ffb0e7-501e-0005-6e40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fb034cb8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tlw98EQW4poqadBaam34rMoQQ8DHiVH/ur/0htibhJc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:48 GMT'}. 2020-03-31 09:42:48,089 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:47 GMT, Server-Request-ID=62ffb0e7-501e-0005-6e40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:48,089 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:47 GMT, Server-Request-ID=62ffb0e7-501e-0005-6e40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:48,089 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:42:48,089 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:47 GMT, Server-Request-ID=62ffb0e7-501e-0005-6e40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fb034cb8-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:tlw98EQW4poqadBaam34rMoQQ8DHiVH/ur/0htibhJc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:48 GMT'}. 2020-03-31 09:42:48,089 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:47 GMT, Server-Request-ID=62ffb0e7-501e-0005-6e40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:48,089 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=fb034cb8-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:47 GMT, Server-Request-ID=62ffb0e7-501e-0005-6e40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:48,089 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:42:48,273 [526] [DEBUG] [app] Starting request: urn:request:ac945c83-55ef-4409-add7-e5eef462377e (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:48,273 [526] [DEBUG] [app] Starting request: urn:request:ac945c83-55ef-4409-add7-e5eef462377e (/health/instance) 2020-03-31 09:42:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:48,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:48,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:48,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:48,280 [546] [DEBUG] [app] Starting request: urn:request:d0509849-9e38-45c4-bcb2-fd4e75209a3d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:48,280 [546] [DEBUG] [app] Starting request: urn:request:d0509849-9e38-45c4-bcb2-fd4e75209a3d (/v1/_internal_ping) 2020-03-31 09:42:48,281 [546] [DEBUG] [app] Ending request: urn:request:d0509849-9e38-45c4-bcb2-fd4e75209a3d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:48,281 [546] [DEBUG] [app] Ending request: urn:request:d0509849-9e38-45c4-bcb2-fd4e75209a3d (/v1/_internal_ping) 2020-03-31 09:42:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:42:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:42:48,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:48,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:42:48,286 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:42:48,286 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:48,288 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:48,288 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:48,289 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:48,289 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:48,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:48,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:48,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:48,307 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:48,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:48,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:48,312 [518] [DEBUG] [app] Starting request: urn:request:b52188c3-5a68-4a07-b64b-80b810604b66 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:48,312 [518] [DEBUG] [app] Starting request: urn:request:b52188c3-5a68-4a07-b64b-80b810604b66 (/_internal_ping) 2020-03-31 09:42:48,313 [518] [DEBUG] [app] Ending request: urn:request:b52188c3-5a68-4a07-b64b-80b810604b66 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:48,313 [518] [DEBUG] [app] Ending request: urn:request:b52188c3-5a68-4a07-b64b-80b810604b66 (/_internal_ping) 2020-03-31 09:42:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:48,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:42:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:48,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:42:48,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:48,318 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:48,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:48,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:48,322 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:42:48,323 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:42:48,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:48,328 [535] [DEBUG] [app] Starting request: urn:request:6d5f8336-d16a-4768-a894-ce3f7a5f78ae (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:48,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:42:48,328 [535] [DEBUG] [app] Starting request: urn:request:6d5f8336-d16a-4768-a894-ce3f7a5f78ae (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:48,329 [535] [DEBUG] [app] Ending request: urn:request:6d5f8336-d16a-4768-a894-ce3f7a5f78ae (/c1/_internal_ping) 2020-03-31 09:42:48,329 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:48,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:42:48,329 [535] [DEBUG] [app] Ending request: urn:request:6d5f8336-d16a-4768-a894-ce3f7a5f78ae (/c1/_internal_ping) 2020-03-31 09:42:48,329 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:48,329 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:48,331 [526] [DEBUG] [app] Ending request: urn:request:ac945c83-55ef-4409-add7-e5eef462377e (/health/instance) 2020-03-31 09:42:48,331 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:48,331 [526] [DEBUG] [app] Ending request: urn:request:ac945c83-55ef-4409-add7-e5eef462377e (/health/instance) 2020-03-31 09:42:48,331 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:48,332 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:42:48,332 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:42:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:54.323352+00:00 (in 4.823088 seconds) 2020-03-31 09:42:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:49 UTC)" (scheduled at 2020-03-31 09:42:49.498966+00:00) 2020-03-31 09:42:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:42:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:54.323352+00:00 (in 4.823088 seconds) 2020-03-31 09:42:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:49 UTC)" (scheduled at 2020-03-31 09:42:49.498966+00:00) 2020-03-31 09:42:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:42:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:49 UTC)" executed successfully 2020-03-31 09:42:50,092 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:fc41d5cc-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:50 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:42:50,092 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:fc41d5cc-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:50 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:42:50,092 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fc41d5cc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:G7VEd9Sl8jqNuchBVL3zqLIt0gtmpNcwRBzJe+Uv0e8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:50 GMT'}. storagereplication stdout | 2020-03-31 09:42:50,092 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fc41d5cc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:G7VEd9Sl8jqNuchBVL3zqLIt0gtmpNcwRBzJe+Uv0e8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:50 GMT'}. 2020-03-31 09:42:50,173 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:42:50,173 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:42:50,174 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:49 GMT, Server-Request-ID=62ffb8eb-501e-0005-5940-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fc41d5cc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:G7VEd9Sl8jqNuchBVL3zqLIt0gtmpNcwRBzJe+Uv0e8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:50 GMT'}. 2020-03-31 09:42:50,174 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:49 GMT, Server-Request-ID=62ffb8eb-501e-0005-5940-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:50,175 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:49 GMT, Server-Request-ID=62ffb8eb-501e-0005-5940-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:42:50,174 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:49 GMT, Server-Request-ID=62ffb8eb-501e-0005-5940-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'fc41d5cc-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:G7VEd9Sl8jqNuchBVL3zqLIt0gtmpNcwRBzJe+Uv0e8=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:50 GMT'}. 2020-03-31 09:42:50,174 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:49 GMT, Server-Request-ID=62ffb8eb-501e-0005-5940-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:50,175 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=fc41d5cc-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:49 GMT, Server-Request-ID=62ffb8eb-501e-0005-5940-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:50,175 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:42:50,175 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:42:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:42:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:59.560326+00:00 (in 8.448597 seconds) 2020-03-31 09:42:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:51 UTC)" (scheduled at 2020-03-31 09:42:51.110610+00:00) 2020-03-31 09:42:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:42:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:42:59.560326+00:00 (in 8.448597 seconds) 2020-03-31 09:42:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:51 UTC)" (scheduled at 2020-03-31 09:42:51.110610+00:00) 2020-03-31 09:42:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 51, 112742), True, datetime.datetime(2020, 3, 31, 9, 42, 51, 112742), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:42:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 51, 112742), True, datetime.datetime(2020, 3, 31, 9, 42, 51, 112742), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:42:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:42:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:42:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:51 UTC)" executed successfully 2020-03-31 09:42:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:42:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:01.440707+00:00 (in 9.999156 seconds) 2020-03-31 09:42:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:01 UTC)" (scheduled at 2020-03-31 09:42:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:42:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:01.440707+00:00 (in 9.999156 seconds) 2020-03-31 09:42:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:01 UTC)" (scheduled at 2020-03-31 09:42:51.440707+00:00) 2020-03-31 09:42:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:42:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:42:51,551 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:42:51,551 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:42:51,551 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:42:51,551 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:51,551 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:42:51,551 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:01 UTC)" executed successfully 2020-03-31 09:42:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:42:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:22.113221+00:00 (in 29.999053 seconds) 2020-03-31 09:42:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:22 UTC)" (scheduled at 2020-03-31 09:42:52.113221+00:00) securityworker stdout | 2020-03-31 09:42:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:22.113221+00:00 (in 29.999053 seconds) 2020-03-31 09:42:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:22 UTC)" (scheduled at 2020-03-31 09:42:52.113221+00:00) 2020-03-31 09:42:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:42:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:42:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:42:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:42:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:42:52,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:42:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:42:52,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:22 UTC)" executed successfully 2020-03-31 09:42:54,179 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:feb184e2-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:54 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:42:54,179 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'feb184e2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:E2uFYb5UEh5hRFa/Savey9PJ54W6SjT76erXg8CoAO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:54 GMT'}. storagereplication stdout | 2020-03-31 09:42:54,179 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:feb184e2-7333-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:42:54 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:42:54,179 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'feb184e2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:E2uFYb5UEh5hRFa/Savey9PJ54W6SjT76erXg8CoAO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:54 GMT'}. 2020-03-31 09:42:54,263 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:42:54,263 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:42:54,264 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:53 GMT, Server-Request-ID=62ffc7e7-501e-0005-4d40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'feb184e2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:E2uFYb5UEh5hRFa/Savey9PJ54W6SjT76erXg8CoAO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:54 GMT'}. 2020-03-31 09:42:54,264 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:53 GMT, Server-Request-ID=62ffc7e7-501e-0005-4d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:54,264 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:53 GMT, Server-Request-ID=62ffc7e7-501e-0005-4d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:54,264 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:42:54,264 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:42:53 GMT, Server-Request-ID=62ffc7e7-501e-0005-4d40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'feb184e2-7333-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:E2uFYb5UEh5hRFa/Savey9PJ54W6SjT76erXg8CoAO0=', 'x-ms-date': 'Tue, 31 Mar 2020 09:42:54 GMT'}. 2020-03-31 09:42:54,264 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:42:53 GMT, Server-Request-ID=62ffc7e7-501e-0005-4d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:54,264 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=feb184e2-7333-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:42:53 GMT, Server-Request-ID=62ffc7e7-501e-0005-4d40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:42:54,264 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:42:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:42:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:04.323352+00:00 (in 9.998966 seconds) 2020-03-31 09:42:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:04 UTC)" (scheduled at 2020-03-31 09:42:54.323352+00:00) 2020-03-31 09:42:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:42:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:04.323352+00:00 (in 9.998966 seconds) 2020-03-31 09:42:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:04 UTC)" (scheduled at 2020-03-31 09:42:54.323352+00:00) 2020-03-31 09:42:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:42:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 54, 325318), True, datetime.datetime(2020, 3, 31, 9, 42, 54, 325318), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:42:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 42, 54, 325318), True, datetime.datetime(2020, 3, 31, 9, 42, 54, 325318), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:42:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:42:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:42:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:42:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:04 UTC)" executed successfully 2020-03-31 09:42:54,446 [527] [DEBUG] [app] Starting request: urn:request:2ac7a8d7-0a84-4b3f-b447-7adb7a10fd31 (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:54,446 [527] [DEBUG] [app] Starting request: urn:request:2ac7a8d7-0a84-4b3f-b447-7adb7a10fd31 (/health/instance) 2020-03-31 09:42:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:54,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:54,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:54,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:54,455 [538] [DEBUG] [app] Starting request: urn:request:71e8c392-c5da-4619-9db4-3708b31baf0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:54,455 [538] [DEBUG] [app] Starting request: urn:request:71e8c392-c5da-4619-9db4-3708b31baf0e (/v1/_internal_ping) 2020-03-31 09:42:54,457 [538] [DEBUG] [app] Ending request: urn:request:71e8c392-c5da-4619-9db4-3708b31baf0e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:54,457 [538] [DEBUG] [app] Ending request: urn:request:71e8c392-c5da-4619-9db4-3708b31baf0e (/v1/_internal_ping) 2020-03-31 09:42:54,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:54,458 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:42:54,457 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:54,459 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:42:54,458 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:54,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:42:54,459 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:42:54,460 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:54,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:54,462 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:54,464 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:54,464 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:54,476 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:54,476 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:54,491 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:54,491 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:54,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:54,497 [527] [DEBUG] [app] Starting request: urn:request:3e6b4c0f-8031-4384-b31e-1e5f241bccbe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:54,497 [527] [DEBUG] [app] Starting request: urn:request:3e6b4c0f-8031-4384-b31e-1e5f241bccbe (/_internal_ping) 2020-03-31 09:42:54,498 [527] [DEBUG] [app] Ending request: urn:request:3e6b4c0f-8031-4384-b31e-1e5f241bccbe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:54,498 [527] [DEBUG] [app] Ending request: urn:request:3e6b4c0f-8031-4384-b31e-1e5f241bccbe (/_internal_ping) 2020-03-31 09:42:54,499 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:42:54,499 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:42:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:54,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:54,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:54,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:54,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:54,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:42:54,508 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:54,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:42:54,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:54,513 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:54,513 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:54,515 [535] [DEBUG] [app] Starting request: urn:request:cf1c4c4e-f754-42d4-99eb-4b8d4751588c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:54,515 [535] [DEBUG] [app] Starting request: urn:request:cf1c4c4e-f754-42d4-99eb-4b8d4751588c (/c1/_internal_ping) 2020-03-31 09:42:54,516 [535] [DEBUG] [app] Ending request: urn:request:cf1c4c4e-f754-42d4-99eb-4b8d4751588c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:54,516 [535] [DEBUG] [app] Ending request: urn:request:cf1c4c4e-f754-42d4-99eb-4b8d4751588c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:54,517 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:54,517 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:42:54,517 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:54,517 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:54,519 [527] [DEBUG] [app] Ending request: urn:request:2ac7a8d7-0a84-4b3f-b447-7adb7a10fd31 (/health/instance) 2020-03-31 09:42:54,519 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:54,519 [527] [DEBUG] [app] Ending request: urn:request:2ac7a8d7-0a84-4b3f-b447-7adb7a10fd31 (/health/instance) 2020-03-31 09:42:54,519 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:54,520 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) gunicorn-web stdout | 2020-03-31 09:42:54,520 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) 2020-03-31 09:42:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:42:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:06.924497+00:00 (in 9.999295 seconds) storagereplication stdout | 2020-03-31 09:42:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:42:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:42:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:06.924497+00:00 (in 9.999295 seconds) 2020-03-31 09:42:58,273 [518] [DEBUG] [app] Starting request: urn:request:9b5537b3-67c4-484c-8f30-1fd158fee54f (/health/instance) gunicorn-web stdout | 2020-03-31 09:42:58,273 [518] [DEBUG] [app] Starting request: urn:request:9b5537b3-67c4-484c-8f30-1fd158fee54f (/health/instance) 2020-03-31 09:42:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:58,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:58,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:58,281 [553] [DEBUG] [app] Starting request: urn:request:b7a5e515-da9a-4ff7-9bb5-f0945bdab1e5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:58,281 [553] [DEBUG] [app] Starting request: urn:request:b7a5e515-da9a-4ff7-9bb5-f0945bdab1e5 (/v1/_internal_ping) 2020-03-31 09:42:58,282 [553] [DEBUG] [app] Ending request: urn:request:b7a5e515-da9a-4ff7-9bb5-f0945bdab1e5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:42:58,282 [553] [DEBUG] [app] Ending request: urn:request:b7a5e515-da9a-4ff7-9bb5-f0945bdab1e5 (/v1/_internal_ping) 2020-03-31 09:42:58,283 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:42:58,283 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:42:58,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:42:58,286 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:42:58,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:42:58,286 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:42:58,288 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:42:58,288 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:42:58,289 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:42:58,289 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:42:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:42:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:42:58,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:42:58,311 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:42:58,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:58,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:58,316 [518] [DEBUG] [app] Starting request: urn:request:8776d851-fff4-4d51-bce6-fedf2729a01b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:58,316 [518] [DEBUG] [app] Starting request: urn:request:8776d851-fff4-4d51-bce6-fedf2729a01b (/_internal_ping) 2020-03-31 09:42:58,317 [518] [DEBUG] [app] Ending request: urn:request:8776d851-fff4-4d51-bce6-fedf2729a01b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:42:58,317 [518] [DEBUG] [app] Ending request: urn:request:8776d851-fff4-4d51-bce6-fedf2729a01b (/_internal_ping) 2020-03-31 09:42:58,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:42:58,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:42:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:42:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:58,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:42:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:42:58,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:58,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:42:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:42:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:42:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:42:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:42:58,330 [535] [DEBUG] [app] Starting request: urn:request:23725020-7b8a-4ec9-a9af-577db3f81210 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:58,330 [535] [DEBUG] [app] Starting request: urn:request:23725020-7b8a-4ec9-a9af-577db3f81210 (/c1/_internal_ping) 2020-03-31 09:42:58,330 [535] [DEBUG] [app] Ending request: urn:request:23725020-7b8a-4ec9-a9af-577db3f81210 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:42:58,330 [535] [DEBUG] [app] Ending request: urn:request:23725020-7b8a-4ec9-a9af-577db3f81210 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:58,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:42:58,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:42:58,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:42:58,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:42:58,333 [518] [DEBUG] [app] Ending request: urn:request:9b5537b3-67c4-484c-8f30-1fd158fee54f (/health/instance) 2020-03-31 09:42:58,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:42:58,333 [518] [DEBUG] [app] Ending request: urn:request:9b5537b3-67c4-484c-8f30-1fd158fee54f (/health/instance) 2020-03-31 09:42:58,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:42:58,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:42:58,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:42:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:42:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:42:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:42:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:42:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:51.110610+00:00 (in 51.549345 seconds) 2020-03-31 09:42:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:59 UTC)" (scheduled at 2020-03-31 09:42:59.560326+00:00) 2020-03-31 09:42:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:42:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:42:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:51.110610+00:00 (in 51.549345 seconds) 2020-03-31 09:42:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:59 UTC)" (scheduled at 2020-03-31 09:42:59.560326+00:00) 2020-03-31 09:42:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:42:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:43:59 UTC)" executed successfully 2020-03-31 09:43:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:43:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:30.337699+00:00 (in 29.999225 seconds) 2020-03-31 09:43:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:30 UTC)" (scheduled at 2020-03-31 09:43:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:43:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:30.337699+00:00 (in 29.999225 seconds) 2020-03-31 09:43:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:30 UTC)" (scheduled at 2020-03-31 09:43:00.337699+00:00) 2020-03-31 09:43:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 43, 0, 339321), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:43:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 43, 0, 339321), False, 50, 1, 0]) 2020-03-31 09:43:00,354 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:43:00,354 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:43:00,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:43:00,354 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:30 UTC)" executed successfully 2020-03-31 09:43:01,450 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:01,450 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:11.440707+00:00 (in 9.990498 seconds) 2020-03-31 09:43:01,450 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:11 UTC)" (scheduled at 2020-03-31 09:43:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:43:01,450 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:43:01,450 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:11.440707+00:00 (in 9.990498 seconds) 2020-03-31 09:43:01,450 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:11 UTC)" (scheduled at 2020-03-31 09:43:01.440707+00:00) 2020-03-31 09:43:01,452 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:43:01,452 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:43:01,469 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:43:01,469 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:43:01,469 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:43:01,469 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:01,470 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:43:01,470 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:11 UTC)" executed successfully 2020-03-31 09:43:02,267 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:03839d20-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:02 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:43:02,267 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:03839d20-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:02 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:43:02,267 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '03839d20-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:VpPWiIql6x/kGb7N/ydlNcKcMfJ/v8Qop/tPC9oC80k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:02 GMT'}. storagereplication stdout | 2020-03-31 09:43:02,267 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '03839d20-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:VpPWiIql6x/kGb7N/ydlNcKcMfJ/v8Qop/tPC9oC80k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:02 GMT'}. 2020-03-31 09:43:02,348 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:43:02,348 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:02,349 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:01 GMT, Server-Request-ID=62ffe77e-501e-0005-2840-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '03839d20-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:VpPWiIql6x/kGb7N/ydlNcKcMfJ/v8Qop/tPC9oC80k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:02 GMT'}. 2020-03-31 09:43:02,349 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:01 GMT, Server-Request-ID=62ffe77e-501e-0005-2840-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:02,349 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:01 GMT, Server-Request-ID=62ffe77e-501e-0005-2840-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:43:02,349 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:01 GMT, Server-Request-ID=62ffe77e-501e-0005-2840-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '03839d20-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:VpPWiIql6x/kGb7N/ydlNcKcMfJ/v8Qop/tPC9oC80k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:02 GMT'}. 2020-03-31 09:43:02,349 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:01 GMT, Server-Request-ID=62ffe77e-501e-0005-2840-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:02,349 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=03839d20-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:01 GMT, Server-Request-ID=62ffe77e-501e-0005-2840-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:02,349 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:43:02,349 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:43:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:43:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:24.410927+00:00 (in 21.079528 seconds) 2020-03-31 09:43:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:33 UTC)" (scheduled at 2020-03-31 09:43:03.330112+00:00) 2020-03-31 09:43:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:43:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:24.410927+00:00 (in 21.079528 seconds) 2020-03-31 09:43:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:33 UTC)" (scheduled at 2020-03-31 09:43:03.330112+00:00) 2020-03-31 09:43:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 3, 332414), True, datetime.datetime(2020, 3, 31, 9, 43, 3, 332414), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:43:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 3, 332414), True, datetime.datetime(2020, 3, 31, 9, 43, 3, 332414), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:43:03,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:43:03,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:43:03,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:03,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:33 UTC)" executed successfully 2020-03-31 09:43:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:43:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:14 UTC)" (scheduled at 2020-03-31 09:43:04.323352+00:00) 2020-03-31 09:43:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:43:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:14 UTC)" (scheduled at 2020-03-31 09:43:04.323352+00:00) 2020-03-31 09:43:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:04,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:14.323352+00:00 (in 9.999024 seconds) notificationworker stdout | 2020-03-31 09:43:04,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:14.323352+00:00 (in 9.999024 seconds) 2020-03-31 09:43:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 4, 325038), True, datetime.datetime(2020, 3, 31, 9, 43, 4, 325038), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:43:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 4, 325038), True, datetime.datetime(2020, 3, 31, 9, 43, 4, 325038), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:43:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:04,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:43:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:43:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:14 UTC)" executed successfully 2020-03-31 09:43:04,446 [518] [DEBUG] [app] Starting request: urn:request:d6df9b01-4874-438a-b6d8-e5ae7de02dda (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:04,446 [518] [DEBUG] [app] Starting request: urn:request:d6df9b01-4874-438a-b6d8-e5ae7de02dda (/health/instance) 2020-03-31 09:43:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:04,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:04,454 [555] [DEBUG] [app] Starting request: urn:request:d7232db7-ae7d-4c32-8c2e-7f039e7ad1b9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:04,454 [555] [DEBUG] [app] Starting request: urn:request:d7232db7-ae7d-4c32-8c2e-7f039e7ad1b9 (/v1/_internal_ping) 2020-03-31 09:43:04,455 [555] [DEBUG] [app] Ending request: urn:request:d7232db7-ae7d-4c32-8c2e-7f039e7ad1b9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:04,455 [555] [DEBUG] [app] Ending request: urn:request:d7232db7-ae7d-4c32-8c2e-7f039e7ad1b9 (/v1/_internal_ping) 2020-03-31 09:43:04,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:43:04,455 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:43:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:04,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:43:04,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:04,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:04,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:04,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:04,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:04,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:04,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:04,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:04,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:04,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:04,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:04,485 [518] [DEBUG] [app] Starting request: urn:request:4e2f6dda-0f04-49fe-9a7a-3d8448a9482e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:04,485 [518] [DEBUG] [app] Starting request: urn:request:4e2f6dda-0f04-49fe-9a7a-3d8448a9482e (/_internal_ping) 2020-03-31 09:43:04,485 [518] [DEBUG] [app] Ending request: urn:request:4e2f6dda-0f04-49fe-9a7a-3d8448a9482e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:04,485 [518] [DEBUG] [app] Ending request: urn:request:4e2f6dda-0f04-49fe-9a7a-3d8448a9482e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:43:04,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:04,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:43:04,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:04,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:04,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:04,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:43:04,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:43:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:04,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:43:04,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:43:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:04,499 [535] [DEBUG] [app] Starting request: urn:request:b2f3f998-6f1c-46ad-b4d0-e48c0dc4f158 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:04,499 [535] [DEBUG] [app] Starting request: urn:request:b2f3f998-6f1c-46ad-b4d0-e48c0dc4f158 (/c1/_internal_ping) 2020-03-31 09:43:04,499 [535] [DEBUG] [app] Ending request: urn:request:b2f3f998-6f1c-46ad-b4d0-e48c0dc4f158 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:04,499 [535] [DEBUG] [app] Ending request: urn:request:b2f3f998-6f1c-46ad-b4d0-e48c0dc4f158 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:04,500 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:43:04,500 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:04,501 [518] [DEBUG] [app] Ending request: urn:request:d6df9b01-4874-438a-b6d8-e5ae7de02dda (/health/instance) 2020-03-31 09:43:04,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:04,501 [518] [DEBUG] [app] Ending request: urn:request:d6df9b01-4874-438a-b6d8-e5ae7de02dda (/health/instance) 2020-03-31 09:43:04,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:04,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:43:04,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:43:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:43:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:16.924497+00:00 (in 9.999230 seconds) storagereplication stdout | 2020-03-31 09:43:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:16.924497+00:00 (in 9.999230 seconds) 2020-03-31 09:43:08,273 [527] [DEBUG] [app] Starting request: urn:request:d35e5352-7e60-4f4b-853a-05c8311d306c (/health/instance) 2020-03-31 09:43:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:08,273 [527] [DEBUG] [app] Starting request: urn:request:d35e5352-7e60-4f4b-853a-05c8311d306c (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:08,288 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:08,288 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:08,290 [550] [DEBUG] [app] Starting request: urn:request:ae2de124-c467-49aa-9855-85c09b3a6482 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:08,290 [550] [DEBUG] [app] Starting request: urn:request:ae2de124-c467-49aa-9855-85c09b3a6482 (/v1/_internal_ping) 2020-03-31 09:43:08,291 [550] [DEBUG] [app] Ending request: urn:request:ae2de124-c467-49aa-9855-85c09b3a6482 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:08,291 [550] [DEBUG] [app] Ending request: urn:request:ae2de124-c467-49aa-9855-85c09b3a6482 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:08,292 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:08,292 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:43:08,292 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:08,292 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:08,293 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:08,293 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:08,294 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:43:08,294 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:08,296 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:08,296 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:08,296 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:08,296 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:08,305 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:08,305 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:08,318 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:08,318 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:08,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:08,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:08,323 [527] [DEBUG] [app] Starting request: urn:request:4e579f00-1900-4be5-bb34-6b24aac7390a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:08,323 [527] [DEBUG] [app] Starting request: urn:request:4e579f00-1900-4be5-bb34-6b24aac7390a (/_internal_ping) 2020-03-31 09:43:08,324 [527] [DEBUG] [app] Ending request: urn:request:4e579f00-1900-4be5-bb34-6b24aac7390a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:08,324 [527] [DEBUG] [app] Ending request: urn:request:4e579f00-1900-4be5-bb34-6b24aac7390a (/_internal_ping) 2020-03-31 09:43:08,325 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:43:08,325 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:08,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:43:08,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:08,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:08,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:08,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:43:08,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:43:08,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:08,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:08,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:43:08,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:43:08,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:08,336 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:08,338 [535] [DEBUG] [app] Starting request: urn:request:8ff07d0c-e8b9-4251-9dbd-349ac9c1c8e5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:08,338 [535] [DEBUG] [app] Starting request: urn:request:8ff07d0c-e8b9-4251-9dbd-349ac9c1c8e5 (/c1/_internal_ping) 2020-03-31 09:43:08,338 [535] [DEBUG] [app] Ending request: urn:request:8ff07d0c-e8b9-4251-9dbd-349ac9c1c8e5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:08,338 [535] [DEBUG] [app] Ending request: urn:request:8ff07d0c-e8b9-4251-9dbd-349ac9c1c8e5 (/c1/_internal_ping) 2020-03-31 09:43:08,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:08,339 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:08,339 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:08,341 [527] [DEBUG] [app] Ending request: urn:request:d35e5352-7e60-4f4b-853a-05c8311d306c (/health/instance) 2020-03-31 09:43:08,341 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:43:08,339 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:08,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:43:08,341 [527] [DEBUG] [app] Ending request: urn:request:d35e5352-7e60-4f4b-853a-05c8311d306c (/health/instance) 2020-03-31 09:43:08,341 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:08,342 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:43:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:43:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:21.440707+00:00 (in 9.999157 seconds) 2020-03-31 09:43:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:21 UTC)" (scheduled at 2020-03-31 09:43:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:43:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:21.440707+00:00 (in 9.999157 seconds) 2020-03-31 09:43:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:21 UTC)" (scheduled at 2020-03-31 09:43:11.440707+00:00) 2020-03-31 09:43:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:43:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:43:11,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:43:11,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:43:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:43:11,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:11,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:21 UTC)" executed successfully 2020-03-31 09:43:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:43:14,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:24 UTC)" (scheduled at 2020-03-31 09:43:14.323352+00:00) 2020-03-31 09:43:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:24.323352+00:00 (in 9.995342 seconds) 2020-03-31 09:43:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:43:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:24 UTC)" (scheduled at 2020-03-31 09:43:14.323352+00:00) 2020-03-31 09:43:14,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:24.323352+00:00 (in 9.995342 seconds) 2020-03-31 09:43:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 14, 328984), True, datetime.datetime(2020, 3, 31, 9, 43, 14, 328984), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:43:14,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 14, 328984), True, datetime.datetime(2020, 3, 31, 9, 43, 14, 328984), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:43:14,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:43:14,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:43:14,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:14,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:24 UTC)" executed successfully 2020-03-31 09:43:14,446 [518] [DEBUG] [app] Starting request: urn:request:8d23aaa9-92a2-435c-950a-719d2ad8f002 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:14,446 [518] [DEBUG] [app] Starting request: urn:request:8d23aaa9-92a2-435c-950a-719d2ad8f002 (/health/instance) 2020-03-31 09:43:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:14,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:14,455 [556] [DEBUG] [app] Starting request: urn:request:fda23b9f-b3ad-485d-9652-aac8f629338b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:14,455 [556] [DEBUG] [app] Starting request: urn:request:fda23b9f-b3ad-485d-9652-aac8f629338b (/v1/_internal_ping) 2020-03-31 09:43:14,456 [556] [DEBUG] [app] Ending request: urn:request:fda23b9f-b3ad-485d-9652-aac8f629338b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:14,456 [556] [DEBUG] [app] Ending request: urn:request:fda23b9f-b3ad-485d-9652-aac8f629338b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:14,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:43:14,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:14,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:14,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:14,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:43:14,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-registry stdout | 2020-03-31 09:43:14,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:14,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:14,460 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:43:14,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:14,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:14,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:14,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:14,500 [518] [DEBUG] [app] Starting request: urn:request:d37263d6-b9c7-45a1-875c-bebe50ae1004 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:14,500 [518] [DEBUG] [app] Starting request: urn:request:d37263d6-b9c7-45a1-875c-bebe50ae1004 (/_internal_ping) 2020-03-31 09:43:14,501 [518] [DEBUG] [app] Ending request: urn:request:d37263d6-b9c7-45a1-875c-bebe50ae1004 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:14,501 [518] [DEBUG] [app] Ending request: urn:request:d37263d6-b9c7-45a1-875c-bebe50ae1004 (/_internal_ping) 2020-03-31 09:43:14,501 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:14,501 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:14,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:14,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:43:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:43:14,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:14,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:14,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:43:14,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:43:14,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:14,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:14,514 [535] [DEBUG] [app] Starting request: urn:request:9c05d223-28e2-4d29-be06-94604fe2070b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:14,514 [535] [DEBUG] [app] Starting request: urn:request:9c05d223-28e2-4d29-be06-94604fe2070b (/c1/_internal_ping) 2020-03-31 09:43:14,515 [535] [DEBUG] [app] Ending request: urn:request:9c05d223-28e2-4d29-be06-94604fe2070b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:14,515 [535] [DEBUG] [app] Ending request: urn:request:9c05d223-28e2-4d29-be06-94604fe2070b (/c1/_internal_ping) 2020-03-31 09:43:14,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:14,516 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:14,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:43:14,516 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:14,517 [518] [DEBUG] [app] Ending request: urn:request:8d23aaa9-92a2-435c-950a-719d2ad8f002 (/health/instance) 2020-03-31 09:43:14,517 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:14,517 [518] [DEBUG] [app] Ending request: urn:request:8d23aaa9-92a2-435c-950a-719d2ad8f002 (/health/instance) 2020-03-31 09:43:14,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:14,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:43:14,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:43:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:43:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:45.064514+00:00 (in 29.999233 seconds) 2020-03-31 09:43:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:45 UTC)" (scheduled at 2020-03-31 09:43:15.064514+00:00) gcworker stdout | 2020-03-31 09:43:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:45.064514+00:00 (in 29.999233 seconds) 2020-03-31 09:43:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:45 UTC)" (scheduled at 2020-03-31 09:43:15.064514+00:00) 2020-03-31 09:43:15,066 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2020-03-31 09:43:15,066 [174] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) 2020-03-31 09:43:15,080 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438195, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:43:15,080 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438195, 1209600, 500, 1, 0]) 2020-03-31 09:43:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:43:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:43:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:43:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:45 UTC)" executed successfully 2020-03-31 09:43:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:43:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:26.924497+00:00 (in 9.999219 seconds) storagereplication stdout | 2020-03-31 09:43:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:26.924497+00:00 (in 9.999219 seconds) 2020-03-31 09:43:18,272 [518] [DEBUG] [app] Starting request: urn:request:999fa64c-504d-4f79-b00e-3a5c1418edb4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:18,272 [518] [DEBUG] [app] Starting request: urn:request:999fa64c-504d-4f79-b00e-3a5c1418edb4 (/health/instance) 2020-03-31 09:43:18,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:18,275 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:18,277 [546] [DEBUG] [app] Starting request: urn:request:4cc90a07-f030-44b6-8408-42784c497c5c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:18,277 [546] [DEBUG] [app] Starting request: urn:request:4cc90a07-f030-44b6-8408-42784c497c5c (/v1/_internal_ping) 2020-03-31 09:43:18,278 [546] [DEBUG] [app] Ending request: urn:request:4cc90a07-f030-44b6-8408-42784c497c5c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:18,278 [546] [DEBUG] [app] Ending request: urn:request:4cc90a07-f030-44b6-8408-42784c497c5c (/v1/_internal_ping) 2020-03-31 09:43:18,279 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:43:18,279 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:18,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:18,281 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:43:18,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:43:18,281 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:18,282 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:18,282 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:18,283 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:18,283 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:18,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:18,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:18,311 [518] [DEBUG] [app] Starting request: urn:request:e3e9e048-39f9-45da-8f12-dc2873f1665c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:18,311 [518] [DEBUG] [app] Starting request: urn:request:e3e9e048-39f9-45da-8f12-dc2873f1665c (/_internal_ping) 2020-03-31 09:43:18,312 [518] [DEBUG] [app] Ending request: urn:request:e3e9e048-39f9-45da-8f12-dc2873f1665c (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:18,312 [518] [DEBUG] [app] Ending request: urn:request:e3e9e048-39f9-45da-8f12-dc2873f1665c (/_internal_ping) 2020-03-31 09:43:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:43:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:43:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:43:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:43:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:18,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:18,328 [535] [DEBUG] [app] Starting request: urn:request:fbac47e1-2691-47a5-9e6e-28f94c7141a3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:18,328 [535] [DEBUG] [app] Starting request: urn:request:fbac47e1-2691-47a5-9e6e-28f94c7141a3 (/c1/_internal_ping) 2020-03-31 09:43:18,329 [535] [DEBUG] [app] Ending request: urn:request:fbac47e1-2691-47a5-9e6e-28f94c7141a3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:18,329 [535] [DEBUG] [app] Ending request: urn:request:fbac47e1-2691-47a5-9e6e-28f94c7141a3 (/c1/_internal_ping) 2020-03-31 09:43:18,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:18,330 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:18,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:43:18,330 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:18,332 [518] [DEBUG] [app] Ending request: urn:request:999fa64c-504d-4f79-b00e-3a5c1418edb4 (/health/instance) 2020-03-31 09:43:18,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:18,332 [518] [DEBUG] [app] Ending request: urn:request:999fa64c-504d-4f79-b00e-3a5c1418edb4 (/health/instance) 2020-03-31 09:43:18,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:18,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:43:18,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:43:18,365 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:43:18,365 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy 2020-03-31 09:43:18,365 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:43:18,365 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:43:18,365 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:43:18,365 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:43:18,375 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) storagereplication stdout | 2020-03-31 09:43:18,375 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) 2020-03-31 09:43:18,379 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 48, 18, 374724), True, u'1ce6d63c-596b-4a6b-b342-736d881feb7c', 4]) storagereplication stdout | 2020-03-31 09:43:18,379 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 48, 18, 374724), True, u'1ce6d63c-596b-4a6b-b342-736d881feb7c', 4]) 2020-03-31 09:43:18,385 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 18, 384342), True, datetime.datetime(2020, 3, 31, 9, 43, 18, 384342), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:43:18,385 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 18, 384342), True, datetime.datetime(2020, 3, 31, 9, 43, 18, 384342), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:43:18,389 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 3, 18, 384342), 1, u'aa34abb3-5ba6-49b4-a1a5-093aadd79315', 2, u'bae61519-03c4-42e9-b0e3-e9d3ef876e69']) storagereplication stdout | 2020-03-31 09:43:18,389 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 3, 18, 384342), 1, u'aa34abb3-5ba6-49b4-a1a5-093aadd79315', 2, u'bae61519-03c4-42e9-b0e3-e9d3ef876e69']) 2020-03-31 09:43:18,398 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} storagereplication stdout | 2020-03-31 09:43:18,398 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:43:18,398 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:18,398 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 storagereplication stdout | 2020-03-31 09:43:18,398 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:18,398 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 2020-03-31 09:43:18,399 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:43:18,399 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:43:18,414 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:43:18,414 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:43:18,419 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) storagereplication stdout | 2020-03-31 09:43:18,419 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) 2020-03-31 09:43:18,423 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:43:18,423 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:43:18,424 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:0d24eb68-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:43:18,424 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d24eb68-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d24eb68-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/1EoHUX16Rn/JnrB2GQNkkTlx708WhD/+AITmXU19mc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. storagereplication stdout | 2020-03-31 09:43:18,424 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:0d24eb68-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:43:18,424 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d24eb68-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d24eb68-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/1EoHUX16Rn/JnrB2GQNkkTlx708WhD/+AITmXU19mc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. 2020-03-31 09:43:18,432 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:43:18,432 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:43:18,433 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d24eb68-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:18 GMT, Server-Request-ID=6e4ddb7d-201e-0053-2140-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d24eb68-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/1EoHUX16Rn/JnrB2GQNkkTlx708WhD/+AITmXU19mc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. storagereplication stdout | 2020-03-31 09:43:18,433 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d24eb68-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:18 GMT, Server-Request-ID=6e4ddb7d-201e-0053-2140-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d24eb68-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:/1EoHUX16Rn/JnrB2GQNkkTlx708WhD/+AITmXU19mc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. 2020-03-31 09:43:18,435 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us storagereplication stdout | 2020-03-31 09:43:18,435 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us 2020-03-31 09:43:18,435 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:18,435 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:43:18,435 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:18,435 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:43:18,436 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:0d26cfd2-7334-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=c/OQLbHe4inyngvN209TwxJGDre9gbS2fjFEC84AB5o%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A18Z x-ms-date:Tue, 31 Mar 2020 09:43:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:43:18,436 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d26cfd2-7334-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=c/OQLbHe4inyngvN209TwxJGDre9gbS2fjFEC84AB5o%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3t/tJoSqk2YvtTe15xt7azhNhWlEsfSxSFATSSJ+w0Q=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '0d26cfd2-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. storagereplication stdout | 2020-03-31 09:43:18,436 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:0d26cfd2-7334-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=c/OQLbHe4inyngvN209TwxJGDre9gbS2fjFEC84AB5o%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A18Z x-ms-date:Tue, 31 Mar 2020 09:43:18 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:43:18,436 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d26cfd2-7334-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=c/OQLbHe4inyngvN209TwxJGDre9gbS2fjFEC84AB5o%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3t/tJoSqk2YvtTe15xt7azhNhWlEsfSxSFATSSJ+w0Q=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '0d26cfd2-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. 2020-03-31 09:43:18,451 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:43:18,451 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 2020-03-31 09:43:18,452 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d26cfd2-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:18 GMT, Server-Request-ID=6e4ddb82-201e-0053-2440-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=c/OQLbHe4inyngvN209TwxJGDre9gbS2fjFEC84AB5o%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3t/tJoSqk2YvtTe15xt7azhNhWlEsfSxSFATSSJ+w0Q=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '0d26cfd2-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. storagereplication stdout | 2020-03-31 09:43:18,452 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d26cfd2-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:18 GMT, Server-Request-ID=6e4ddb82-201e-0053-2440-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=c/OQLbHe4inyngvN209TwxJGDre9gbS2fjFEC84AB5o%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A18Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:3t/tJoSqk2YvtTe15xt7azhNhWlEsfSxSFATSSJ+w0Q=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '0d26cfd2-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. 2020-03-31 09:43:18,452 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:43:18,453 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:0d296760-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:18 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:43:18,453 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d296760-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ZGYnTymoXdw/UAOSsJfW3aPCEUk/vc6KbR/eXikt1fo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. storagereplication stdout | 2020-03-31 09:43:18,452 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:43:18,453 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:0d296760-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:18 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:43:18,453 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d296760-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ZGYnTymoXdw/UAOSsJfW3aPCEUk/vc6KbR/eXikt1fo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. 2020-03-31 09:43:18,538 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:43:18,538 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:18,539 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:17 GMT, Server-Request-ID=630021f7-501e-0005-2740-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d296760-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ZGYnTymoXdw/UAOSsJfW3aPCEUk/vc6KbR/eXikt1fo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. 2020-03-31 09:43:18,539 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:17 GMT, Server-Request-ID=630021f7-501e-0005-2740-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:18,539 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:17 GMT, Server-Request-ID=630021f7-501e-0005-2740-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:43:18,539 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:17 GMT, Server-Request-ID=630021f7-501e-0005-2740-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0d296760-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:ZGYnTymoXdw/UAOSsJfW3aPCEUk/vc6KbR/eXikt1fo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:18 GMT'}. 2020-03-31 09:43:18,539 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:17 GMT, Server-Request-ID=630021f7-501e-0005-2740-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:18,539 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=0d296760-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:17 GMT, Server-Request-ID=630021f7-501e-0005-2740-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:18,539 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:43:18,539 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:43:20,542 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:0e682184-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:20 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:43:20,542 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:0e682184-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:20 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:43:20,542 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0e682184-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:MWH/gsi/KDdFwrVDDuuIUenP5WcZ95CAujdVZdw6cSc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:20 GMT'}. storagereplication stdout | 2020-03-31 09:43:20,542 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0e682184-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:MWH/gsi/KDdFwrVDDuuIUenP5WcZ95CAujdVZdw6cSc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:20 GMT'}. 2020-03-31 09:43:20,622 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:43:20,622 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:20,623 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:19 GMT, Server-Request-ID=63002b1f-501e-0005-2140-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0e682184-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:MWH/gsi/KDdFwrVDDuuIUenP5WcZ95CAujdVZdw6cSc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:20 GMT'}. 2020-03-31 09:43:20,624 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:19 GMT, Server-Request-ID=63002b1f-501e-0005-2140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:43:20,623 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:19 GMT, Server-Request-ID=63002b1f-501e-0005-2140-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '0e682184-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:MWH/gsi/KDdFwrVDDuuIUenP5WcZ95CAujdVZdw6cSc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:20 GMT'}. 2020-03-31 09:43:20,624 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:19 GMT, Server-Request-ID=63002b1f-501e-0005-2140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:20,624 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:19 GMT, Server-Request-ID=63002b1f-501e-0005-2140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:20,624 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:43:20,624 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=0e682184-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:19 GMT, Server-Request-ID=63002b1f-501e-0005-2140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:20,624 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:43:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:43:21,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:31.440707+00:00 (in 9.998022 seconds) 2020-03-31 09:43:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:31 UTC)" (scheduled at 2020-03-31 09:43:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:43:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:31.440707+00:00 (in 9.998022 seconds) 2020-03-31 09:43:21,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:31 UTC)" (scheduled at 2020-03-31 09:43:21.440707+00:00) 2020-03-31 09:43:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:43:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:43:21,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:43:21,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:43:21,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:43:21,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:31 UTC)" executed successfully 2020-03-31 09:43:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:43:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:52 UTC)" (scheduled at 2020-03-31 09:43:22.113221+00:00) securityworker stdout | 2020-03-31 09:43:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:52 UTC)" (scheduled at 2020-03-31 09:43:22.113221+00:00) 2020-03-31 09:43:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:52.113221+00:00 (in 29.999245 seconds) 2020-03-31 09:43:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:43:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:52.113221+00:00 (in 29.999245 seconds) 2020-03-31 09:43:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:43:22,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:43:22,132 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:43:22,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:43:22,136 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:43:22,136 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:22,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:43:22,136 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:22,136 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:43:52 UTC)" executed successfully 2020-03-31 09:43:24,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:43:24,326 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:34.323352+00:00 (in 9.996486 seconds) 2020-03-31 09:43:24,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:34 UTC)" (scheduled at 2020-03-31 09:43:24.323352+00:00) 2020-03-31 09:43:24,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:43:24,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:34.323352+00:00 (in 9.996486 seconds) 2020-03-31 09:43:24,327 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:34 UTC)" (scheduled at 2020-03-31 09:43:24.323352+00:00) 2020-03-31 09:43:24,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:24,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 24, 327690), True, datetime.datetime(2020, 3, 31, 9, 43, 24, 327690), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:43:24,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 24, 327690), True, datetime.datetime(2020, 3, 31, 9, 43, 24, 327690), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:43:24,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:43:24,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:43:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:34 UTC)" executed successfully 2020-03-31 09:43:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:43:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:33.330112+00:00 (in 8.918208 seconds) 2020-03-31 09:43:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:24 UTC)" (scheduled at 2020-03-31 09:43:24.410927+00:00) 2020-03-31 09:43:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:43:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:33.330112+00:00 (in 8.918208 seconds) 2020-03-31 09:43:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:24 UTC)" (scheduled at 2020-03-31 09:43:24.410927+00:00) 2020-03-31 09:43:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:24 UTC)" executed successfully 2020-03-31 09:43:24,445 [518] [DEBUG] [app] Starting request: urn:request:2d0d07a6-c3ee-4c8f-a1d9-153c37b716f5 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:24,445 [518] [DEBUG] [app] Starting request: urn:request:2d0d07a6-c3ee-4c8f-a1d9-153c37b716f5 (/health/instance) 2020-03-31 09:43:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:24,453 [549] [DEBUG] [app] Starting request: urn:request:9712d90f-4d83-4863-89cc-1875edd81766 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:24,453 [549] [DEBUG] [app] Starting request: urn:request:9712d90f-4d83-4863-89cc-1875edd81766 (/v1/_internal_ping) 2020-03-31 09:43:24,454 [549] [DEBUG] [app] Ending request: urn:request:9712d90f-4d83-4863-89cc-1875edd81766 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:24,454 [549] [DEBUG] [app] Ending request: urn:request:9712d90f-4d83-4863-89cc-1875edd81766 (/v1/_internal_ping) 2020-03-31 09:43:24,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:43:24,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:24,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:43:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:43:24,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:24,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:24,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:24,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:24,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:24,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:24,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:24,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:24,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:24,486 [518] [DEBUG] [app] Starting request: urn:request:187b74d6-11e1-4b90-8ca6-779bef421705 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:24,486 [518] [DEBUG] [app] Starting request: urn:request:187b74d6-11e1-4b90-8ca6-779bef421705 (/_internal_ping) 2020-03-31 09:43:24,486 [518] [DEBUG] [app] Ending request: urn:request:187b74d6-11e1-4b90-8ca6-779bef421705 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:24,486 [518] [DEBUG] [app] Ending request: urn:request:187b74d6-11e1-4b90-8ca6-779bef421705 (/_internal_ping) 2020-03-31 09:43:24,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:24,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:43:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:43:24,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:43:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:24,499 [535] [DEBUG] [app] Starting request: urn:request:09f7d0fc-b476-4bc2-ae29-49ec53bbb3b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:24,499 [535] [DEBUG] [app] Starting request: urn:request:09f7d0fc-b476-4bc2-ae29-49ec53bbb3b1 (/c1/_internal_ping) 2020-03-31 09:43:24,500 [535] [DEBUG] [app] Ending request: urn:request:09f7d0fc-b476-4bc2-ae29-49ec53bbb3b1 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:24,500 [535] [DEBUG] [app] Ending request: urn:request:09f7d0fc-b476-4bc2-ae29-49ec53bbb3b1 (/c1/_internal_ping) 2020-03-31 09:43:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:24,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:43:24,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:24,502 [518] [DEBUG] [app] Ending request: urn:request:2d0d07a6-c3ee-4c8f-a1d9-153c37b716f5 (/health/instance) 2020-03-31 09:43:24,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:24,502 [518] [DEBUG] [app] Ending request: urn:request:2d0d07a6-c3ee-4c8f-a1d9-153c37b716f5 (/health/instance) 2020-03-31 09:43:24,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:24,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:43:24,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:43:24,629 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:10d7b632-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:24 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:43:24,629 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:10d7b632-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:24 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:43:24,629 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '10d7b632-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:beWqMIGRB3L5wf3Y2j4iGdCbW95My6oW49DW5b5LS0w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:24 GMT'}. storagereplication stdout | 2020-03-31 09:43:24,629 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '10d7b632-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:beWqMIGRB3L5wf3Y2j4iGdCbW95My6oW49DW5b5LS0w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:24 GMT'}. 2020-03-31 09:43:24,710 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:24,711 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:23 GMT, Server-Request-ID=63003b35-501e-0005-6f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '10d7b632-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:beWqMIGRB3L5wf3Y2j4iGdCbW95My6oW49DW5b5LS0w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:24 GMT'}. 2020-03-31 09:43:24,711 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:23 GMT, Server-Request-ID=63003b35-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:43:24,710 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:24,711 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:23 GMT, Server-Request-ID=63003b35-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:24,711 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:43:24,711 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:23 GMT, Server-Request-ID=63003b35-501e-0005-6f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '10d7b632-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:beWqMIGRB3L5wf3Y2j4iGdCbW95My6oW49DW5b5LS0w=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:24 GMT'}. 2020-03-31 09:43:24,711 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:23 GMT, Server-Request-ID=63003b35-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:43:24,711 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=10d7b632-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:23 GMT, Server-Request-ID=63003b35-501e-0005-6f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:24,711 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:43:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:43:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:36.924497+00:00 (in 9.999157 seconds) storagereplication stdout | 2020-03-31 09:43:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:36.924497+00:00 (in 9.999157 seconds) 2020-03-31 09:43:28,274 [526] [DEBUG] [app] Starting request: urn:request:1b5aa0d9-0822-499d-b905-d6292ef2a076 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:28,274 [526] [DEBUG] [app] Starting request: urn:request:1b5aa0d9-0822-499d-b905-d6292ef2a076 (/health/instance) 2020-03-31 09:43:28,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:28,276 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:28,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:28,281 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:28,282 [538] [DEBUG] [app] Starting request: urn:request:f33149c0-8053-41ae-a356-336a01c91603 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:28,282 [538] [DEBUG] [app] Starting request: urn:request:f33149c0-8053-41ae-a356-336a01c91603 (/v1/_internal_ping) 2020-03-31 09:43:28,283 [538] [DEBUG] [app] Ending request: urn:request:f33149c0-8053-41ae-a356-336a01c91603 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:28,283 [538] [DEBUG] [app] Ending request: urn:request:f33149c0-8053-41ae-a356-336a01c91603 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:28,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:43:28,284 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:28,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:28,284 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:28,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:43:28,285 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:28,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:28,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:43:28,288 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:28,289 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:28,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:28,294 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:28,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:28,309 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:28,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:28,312 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:28,314 [526] [DEBUG] [app] Starting request: urn:request:04b04552-58ff-4e02-9817-924e47a83ed8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:28,314 [526] [DEBUG] [app] Starting request: urn:request:04b04552-58ff-4e02-9817-924e47a83ed8 (/_internal_ping) 2020-03-31 09:43:28,315 [526] [DEBUG] [app] Ending request: urn:request:04b04552-58ff-4e02-9817-924e47a83ed8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:28,315 [526] [DEBUG] [app] Ending request: urn:request:04b04552-58ff-4e02-9817-924e47a83ed8 (/_internal_ping) 2020-03-31 09:43:28,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:43:28,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:28,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:43:28,316 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:28,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:28,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:28,320 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:28,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:28,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:43:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:43:28,325 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:43:28,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:28,329 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:28,331 [535] [DEBUG] [app] Starting request: urn:request:9595c0b0-6ef1-4066-b4b6-b12a6e56af44 (/c1/_internal_ping) 2020-03-31 09:43:28,332 [535] [DEBUG] [app] Ending request: urn:request:9595c0b0-6ef1-4066-b4b6-b12a6e56af44 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:28,331 [535] [DEBUG] [app] Starting request: urn:request:9595c0b0-6ef1-4066-b4b6-b12a6e56af44 (/c1/_internal_ping) 2020-03-31 09:43:28,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:28,332 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:43:28,332 [535] [DEBUG] [app] Ending request: urn:request:9595c0b0-6ef1-4066-b4b6-b12a6e56af44 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:28,332 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:28,334 [526] [DEBUG] [app] Ending request: urn:request:1b5aa0d9-0822-499d-b905-d6292ef2a076 (/health/instance) 2020-03-31 09:43:28,334 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:43:28,332 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:28,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:43:28,334 [526] [DEBUG] [app] Ending request: urn:request:1b5aa0d9-0822-499d-b905-d6292ef2a076 (/health/instance) 2020-03-31 09:43:28,334 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:28,335 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:43:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:43:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:00.337699+00:00 (in 29.999183 seconds) 2020-03-31 09:43:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:00 UTC)" (scheduled at 2020-03-31 09:43:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:43:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:00.337699+00:00 (in 29.999183 seconds) 2020-03-31 09:43:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:00 UTC)" (scheduled at 2020-03-31 09:43:30.337699+00:00) 2020-03-31 09:43:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 43, 30, 339282), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:43:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 43, 30, 339282), False, 50, 1, 0]) 2020-03-31 09:43:30,358 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:43:30,358 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:43:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:43:30,359 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:30,359 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:00 UTC)" executed successfully 2020-03-31 09:43:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:43:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:41.440707+00:00 (in 9.999066 seconds) 2020-03-31 09:43:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:41 UTC)" (scheduled at 2020-03-31 09:43:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:43:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:41.440707+00:00 (in 9.999066 seconds) 2020-03-31 09:43:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:41 UTC)" (scheduled at 2020-03-31 09:43:31.440707+00:00) 2020-03-31 09:43:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:43:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:43:31,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:43:31,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:43:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:43:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:41 UTC)" executed successfully 2020-03-31 09:43:32,719 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:15aa3f0e-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:32 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:43:32,719 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:15aa3f0e-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:32 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:43:32,719 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '15aa3f0e-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DxzaCCR474C9yFGbImSMuhqjPU8B09DKmIid9O2s18k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:32 GMT'}. storagereplication stdout | 2020-03-31 09:43:32,719 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '15aa3f0e-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DxzaCCR474C9yFGbImSMuhqjPU8B09DKmIid9O2s18k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:32 GMT'}. 2020-03-31 09:43:32,800 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:43:32,800 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:32,801 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:32 GMT, Server-Request-ID=6300576d-501e-0005-4f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '15aa3f0e-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DxzaCCR474C9yFGbImSMuhqjPU8B09DKmIid9O2s18k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:32 GMT'}. 2020-03-31 09:43:32,801 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:32 GMT, Server-Request-ID=6300576d-501e-0005-4f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:32,801 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:32 GMT, Server-Request-ID=6300576d-501e-0005-4f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:32,801 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:43:32,801 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:32 GMT, Server-Request-ID=6300576d-501e-0005-4f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '15aa3f0e-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:DxzaCCR474C9yFGbImSMuhqjPU8B09DKmIid9O2s18k=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:32 GMT'}. 2020-03-31 09:43:32,801 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:32 GMT, Server-Request-ID=6300576d-501e-0005-4f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:32,801 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=15aa3f0e-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:32 GMT, Server-Request-ID=6300576d-501e-0005-4f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:32,801 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:43:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:43:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:03.330112+00:00 (in 29.999209 seconds) 2020-03-31 09:43:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:03 UTC)" (scheduled at 2020-03-31 09:43:33.330112+00:00) 2020-03-31 09:43:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:43:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:03.330112+00:00 (in 29.999209 seconds) 2020-03-31 09:43:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:03 UTC)" (scheduled at 2020-03-31 09:43:33.330112+00:00) 2020-03-31 09:43:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 33, 331886), True, datetime.datetime(2020, 3, 31, 9, 43, 33, 331886), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:43:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 33, 331886), True, datetime.datetime(2020, 3, 31, 9, 43, 33, 331886), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:43:33,353 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:43:33,353 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:33,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:43:33,353 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:33,353 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:03 UTC)" executed successfully 2020-03-31 09:43:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:43:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:43.539999+00:00 (in 10.171707 seconds) 2020-03-31 09:43:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:33 UTC)" (scheduled at 2020-03-31 09:43:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:43:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:43.539999+00:00 (in 10.171707 seconds) 2020-03-31 09:43:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:33 UTC)" (scheduled at 2020-03-31 09:43:33.366791+00:00) 2020-03-31 09:43:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:43:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 33, 369101), True, datetime.datetime(2020, 3, 31, 9, 43, 33, 369101), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:43:33,371 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 33, 369101), True, datetime.datetime(2020, 3, 31, 9, 43, 33, 369101), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:43:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:43:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:43:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:33 UTC)" executed successfully 2020-03-31 09:43:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:43:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:44.323352+00:00 (in 9.999172 seconds) 2020-03-31 09:43:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:44 UTC)" (scheduled at 2020-03-31 09:43:34.323352+00:00) notificationworker stdout | 2020-03-31 09:43:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:44.323352+00:00 (in 9.999172 seconds) 2020-03-31 09:43:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:44 UTC)" (scheduled at 2020-03-31 09:43:34.323352+00:00) 2020-03-31 09:43:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:43:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 34, 325101), True, datetime.datetime(2020, 3, 31, 9, 43, 34, 325101), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:43:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 34, 325101), True, datetime.datetime(2020, 3, 31, 9, 43, 34, 325101), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:43:34,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:43:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:43:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:44 UTC)" executed successfully 2020-03-31 09:43:34,447 [518] [DEBUG] [app] Starting request: urn:request:2ac315b5-e77c-47dc-96fe-8ef20085a029 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:34,447 [518] [DEBUG] [app] Starting request: urn:request:2ac315b5-e77c-47dc-96fe-8ef20085a029 (/health/instance) 2020-03-31 09:43:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:34,454 [555] [DEBUG] [app] Starting request: urn:request:2e226584-15a0-4ce8-aea1-21ee715b2750 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:34,454 [555] [DEBUG] [app] Starting request: urn:request:2e226584-15a0-4ce8-aea1-21ee715b2750 (/v1/_internal_ping) 2020-03-31 09:43:34,456 [555] [DEBUG] [app] Ending request: urn:request:2e226584-15a0-4ce8-aea1-21ee715b2750 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:34,456 [555] [DEBUG] [app] Ending request: urn:request:2e226584-15a0-4ce8-aea1-21ee715b2750 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:34,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:43:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:34,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:34,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:43:34,459 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:34,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:34,461 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:34,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:34,461 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:34,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:34,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:34,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:34,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:34,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:34,487 [518] [DEBUG] [app] Starting request: urn:request:43c54418-7128-4947-9f0b-656709b9df66 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:34,487 [518] [DEBUG] [app] Starting request: urn:request:43c54418-7128-4947-9f0b-656709b9df66 (/_internal_ping) 2020-03-31 09:43:34,488 [518] [DEBUG] [app] Ending request: urn:request:43c54418-7128-4947-9f0b-656709b9df66 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:34,488 [518] [DEBUG] [app] Ending request: urn:request:43c54418-7128-4947-9f0b-656709b9df66 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:43:34,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:43:34,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:34,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:43:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:43:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:34,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:43:34,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:43:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:34,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:34,502 [534] [DEBUG] [app] Starting request: urn:request:ff45a9e6-7e6e-4664-944a-2cc602d4d4a0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:34,502 [534] [DEBUG] [app] Starting request: urn:request:ff45a9e6-7e6e-4664-944a-2cc602d4d4a0 (/c1/_internal_ping) 2020-03-31 09:43:34,503 [534] [DEBUG] [app] Ending request: urn:request:ff45a9e6-7e6e-4664-944a-2cc602d4d4a0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:34,503 [534] [DEBUG] [app] Ending request: urn:request:ff45a9e6-7e6e-4664-944a-2cc602d4d4a0 (/c1/_internal_ping) 2020-03-31 09:43:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:34,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:34,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:34,505 [518] [DEBUG] [app] Ending request: urn:request:2ac315b5-e77c-47dc-96fe-8ef20085a029 (/health/instance) 2020-03-31 09:43:34,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:43:34,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:43:34,505 [518] [DEBUG] [app] Ending request: urn:request:2ac315b5-e77c-47dc-96fe-8ef20085a029 (/health/instance) 2020-03-31 09:43:34,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:34,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:43:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:43:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:38.010934+00:00 (in 1.085568 seconds) storagereplication stdout | 2020-03-31 09:43:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:38.010934+00:00 (in 1.085568 seconds) 2020-03-31 09:43:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:43:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:46.924497+00:00 (in 8.912691 seconds) 2020-03-31 09:43:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:38 UTC)" (scheduled at 2020-03-31 09:43:38.010934+00:00) 2020-03-31 09:43:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:43:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:46.924497+00:00 (in 8.912691 seconds) 2020-03-31 09:43:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:38 UTC)" (scheduled at 2020-03-31 09:43:38.010934+00:00) 2020-03-31 09:43:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:38 UTC)" executed successfully 2020-03-31 09:43:38,272 [527] [DEBUG] [app] Starting request: urn:request:97859292-5683-4fdb-b2eb-65428d40acec (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:38,272 [527] [DEBUG] [app] Starting request: urn:request:97859292-5683-4fdb-b2eb-65428d40acec (/health/instance) 2020-03-31 09:43:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:38,280 [550] [DEBUG] [app] Starting request: urn:request:e6eb1e63-8012-4665-8fd1-8a85236d9941 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:38,280 [550] [DEBUG] [app] Starting request: urn:request:e6eb1e63-8012-4665-8fd1-8a85236d9941 (/v1/_internal_ping) 2020-03-31 09:43:38,282 [550] [DEBUG] [app] Ending request: urn:request:e6eb1e63-8012-4665-8fd1-8a85236d9941 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:38,282 [550] [DEBUG] [app] Ending request: urn:request:e6eb1e63-8012-4665-8fd1-8a85236d9941 (/v1/_internal_ping) 2020-03-31 09:43:38,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:43:38,282 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:43:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:38,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:43:38,285 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:38,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:38,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:38,287 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:38,287 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:38,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:38,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:38,310 [527] [DEBUG] [app] Starting request: urn:request:1f767a85-9064-4a35-9e67-b047af8bb3ba (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:38,310 [527] [DEBUG] [app] Starting request: urn:request:1f767a85-9064-4a35-9e67-b047af8bb3ba (/_internal_ping) 2020-03-31 09:43:38,311 [527] [DEBUG] [app] Ending request: urn:request:1f767a85-9064-4a35-9e67-b047af8bb3ba (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:38,311 [527] [DEBUG] [app] Ending request: urn:request:1f767a85-9064-4a35-9e67-b047af8bb3ba (/_internal_ping) 2020-03-31 09:43:38,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:43:38,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:38,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:43:38,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:43:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:43:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:43:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:38,323 [535] [DEBUG] [app] Starting request: urn:request:bec6e299-bf95-40a6-baa8-ebacb08ac231 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:38,323 [535] [DEBUG] [app] Starting request: urn:request:bec6e299-bf95-40a6-baa8-ebacb08ac231 (/c1/_internal_ping) 2020-03-31 09:43:38,324 [535] [DEBUG] [app] Ending request: urn:request:bec6e299-bf95-40a6-baa8-ebacb08ac231 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:38,324 [535] [DEBUG] [app] Ending request: urn:request:bec6e299-bf95-40a6-baa8-ebacb08ac231 (/c1/_internal_ping) 2020-03-31 09:43:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:38,324 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:38,326 [527] [DEBUG] [app] Ending request: urn:request:97859292-5683-4fdb-b2eb-65428d40acec (/health/instance) 2020-03-31 09:43:38,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:43:38,324 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:38,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:43:38,326 [527] [DEBUG] [app] Ending request: urn:request:97859292-5683-4fdb-b2eb-65428d40acec (/health/instance) 2020-03-31 09:43:38,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:38,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:43:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:43:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:51.440707+00:00 (in 9.999149 seconds) 2020-03-31 09:43:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:51 UTC)" (scheduled at 2020-03-31 09:43:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:43:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:51.440707+00:00 (in 9.999149 seconds) 2020-03-31 09:43:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:51 UTC)" (scheduled at 2020-03-31 09:43:41.440707+00:00) 2020-03-31 09:43:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:43:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:43:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:43:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:43:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:43:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:51 UTC)" executed successfully 2020-03-31 09:43:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:43:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:43,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:33.366791+00:00 (in 49.826091 seconds) 2020-03-31 09:43:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:43 UTC)" (scheduled at 2020-03-31 09:43:43.539999+00:00) 2020-03-31 09:43:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:43:43,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:33.366791+00:00 (in 49.826091 seconds) 2020-03-31 09:43:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:43 UTC)" (scheduled at 2020-03-31 09:43:43.539999+00:00) 2020-03-31 09:43:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:43 UTC)" executed successfully 2020-03-31 09:43:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:43:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:49.498966+00:00 (in 5.174745 seconds) 2020-03-31 09:43:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:54 UTC)" (scheduled at 2020-03-31 09:43:44.323352+00:00) 2020-03-31 09:43:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:43:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:49.498966+00:00 (in 5.174745 seconds) 2020-03-31 09:43:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:54 UTC)" (scheduled at 2020-03-31 09:43:44.323352+00:00) 2020-03-31 09:43:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 44, 325107), True, datetime.datetime(2020, 3, 31, 9, 43, 44, 325107), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:43:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 44, 325107), True, datetime.datetime(2020, 3, 31, 9, 43, 44, 325107), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:43:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:43:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:43:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:54 UTC)" executed successfully 2020-03-31 09:43:44,446 [518] [DEBUG] [app] Starting request: urn:request:6a0182bc-3b18-4530-844f-952b681817d2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:44,446 [518] [DEBUG] [app] Starting request: urn:request:6a0182bc-3b18-4530-844f-952b681817d2 (/health/instance) 2020-03-31 09:43:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:44,454 [549] [DEBUG] [app] Starting request: urn:request:0139839e-cc03-4f1c-acd8-ce5d4e35823a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:44,454 [549] [DEBUG] [app] Starting request: urn:request:0139839e-cc03-4f1c-acd8-ce5d4e35823a (/v1/_internal_ping) 2020-03-31 09:43:44,455 [549] [DEBUG] [app] Ending request: urn:request:0139839e-cc03-4f1c-acd8-ce5d4e35823a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:44,455 [549] [DEBUG] [app] Ending request: urn:request:0139839e-cc03-4f1c-acd8-ce5d4e35823a (/v1/_internal_ping) 2020-03-31 09:43:44,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:43:44,455 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:44,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:43:44,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:44,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:43:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:44,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:44,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:44,459 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:44,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:44,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:44,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:44,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:44,487 [518] [DEBUG] [app] Starting request: urn:request:d4d73460-7f37-4dcd-8a83-50701f4781be (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:44,487 [518] [DEBUG] [app] Starting request: urn:request:d4d73460-7f37-4dcd-8a83-50701f4781be (/_internal_ping) 2020-03-31 09:43:44,488 [518] [DEBUG] [app] Ending request: urn:request:d4d73460-7f37-4dcd-8a83-50701f4781be (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:44,488 [518] [DEBUG] [app] Ending request: urn:request:d4d73460-7f37-4dcd-8a83-50701f4781be (/_internal_ping) 2020-03-31 09:43:44,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:44,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:44,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:44,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:43:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:43:44,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:43:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:44,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:44,501 [534] [DEBUG] [app] Starting request: urn:request:d763763f-fa07-4f26-a4ed-9d6f9b7af1c0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:44,501 [534] [DEBUG] [app] Starting request: urn:request:d763763f-fa07-4f26-a4ed-9d6f9b7af1c0 (/c1/_internal_ping) 2020-03-31 09:43:44,502 [534] [DEBUG] [app] Ending request: urn:request:d763763f-fa07-4f26-a4ed-9d6f9b7af1c0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:44,502 [534] [DEBUG] [app] Ending request: urn:request:d763763f-fa07-4f26-a4ed-9d6f9b7af1c0 (/c1/_internal_ping) 2020-03-31 09:43:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:44,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:44,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:44,504 [518] [DEBUG] [app] Ending request: urn:request:6a0182bc-3b18-4530-844f-952b681817d2 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:43:44,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:44,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:44,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:43:44,504 [518] [DEBUG] [app] Ending request: urn:request:6a0182bc-3b18-4530-844f-952b681817d2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:44,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:44,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:43:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:43:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:15.064514+00:00 (in 29.998941 seconds) 2020-03-31 09:43:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:15 UTC)" (scheduled at 2020-03-31 09:43:45.064514+00:00) gcworker stdout | 2020-03-31 09:43:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:15.064514+00:00 (in 29.998941 seconds) 2020-03-31 09:43:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:15 UTC)" (scheduled at 2020-03-31 09:43:45.064514+00:00) 2020-03-31 09:43:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438225, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:43:45,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438225, 1209600, 500, 1, 0]) 2020-03-31 09:43:45,088 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:43:45,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:43:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:43:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:15 UTC)" executed successfully 2020-03-31 09:43:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:43:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:56.924497+00:00 (in 9.999001 seconds) storagereplication stdout | 2020-03-31 09:43:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:56.924497+00:00 (in 9.999001 seconds) 2020-03-31 09:43:48,273 [527] [DEBUG] [app] Starting request: urn:request:b4752d29-91fa-4ee1-803a-3cfcb317b3a9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:48,273 [527] [DEBUG] [app] Starting request: urn:request:b4752d29-91fa-4ee1-803a-3cfcb317b3a9 (/health/instance) 2020-03-31 09:43:48,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:48,276 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:48,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:48,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:48,281 [545] [DEBUG] [app] Starting request: urn:request:8dfea9c1-10c5-4387-a61b-1ef378df908e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:48,281 [545] [DEBUG] [app] Starting request: urn:request:8dfea9c1-10c5-4387-a61b-1ef378df908e (/v1/_internal_ping) 2020-03-31 09:43:48,282 [545] [DEBUG] [app] Ending request: urn:request:8dfea9c1-10c5-4387-a61b-1ef378df908e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:48,282 [545] [DEBUG] [app] Ending request: urn:request:8dfea9c1-10c5-4387-a61b-1ef378df908e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:48,283 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:43:48,283 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:48,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:48,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:43:48,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:43:48,285 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:48,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:48,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:48,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:48,287 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:48,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:48,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:48,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:48,313 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:48,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:48,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:48,319 [527] [DEBUG] [app] Starting request: urn:request:12fd75aa-8efd-4b28-aa19-7de269daf3fb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:48,319 [527] [DEBUG] [app] Starting request: urn:request:12fd75aa-8efd-4b28-aa19-7de269daf3fb (/_internal_ping) 2020-03-31 09:43:48,319 [527] [DEBUG] [app] Ending request: urn:request:12fd75aa-8efd-4b28-aa19-7de269daf3fb (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:48,319 [527] [DEBUG] [app] Ending request: urn:request:12fd75aa-8efd-4b28-aa19-7de269daf3fb (/_internal_ping) 2020-03-31 09:43:48,320 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:48,320 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:48,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:43:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:48,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:48,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:48,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:43:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:43:48,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:43:48,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:48,332 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:48,333 [534] [DEBUG] [app] Starting request: urn:request:517a25e7-7a82-4f15-9166-a911c326a5a4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:48,333 [534] [DEBUG] [app] Starting request: urn:request:517a25e7-7a82-4f15-9166-a911c326a5a4 (/c1/_internal_ping) 2020-03-31 09:43:48,334 [534] [DEBUG] [app] Ending request: urn:request:517a25e7-7a82-4f15-9166-a911c326a5a4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:48,334 [534] [DEBUG] [app] Ending request: urn:request:517a25e7-7a82-4f15-9166-a911c326a5a4 (/c1/_internal_ping) 2020-03-31 09:43:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:48,335 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:48,335 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:43:48,335 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:48,336 [527] [DEBUG] [app] Ending request: urn:request:b4752d29-91fa-4ee1-803a-3cfcb317b3a9 (/health/instance) 2020-03-31 09:43:48,337 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:48,336 [527] [DEBUG] [app] Ending request: urn:request:b4752d29-91fa-4ee1-803a-3cfcb317b3a9 (/health/instance) 2020-03-31 09:43:48,337 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:48,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:43:48,337 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:43:48,816 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:43:48,816 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy 2020-03-31 09:43:48,816 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:43:48,816 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:43:48,816 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:43:48,816 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:43:48,826 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) storagereplication stdout | 2020-03-31 09:43:48,826 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) 2020-03-31 09:43:48,830 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 48, 48, 825894), True, u'93e98dd4-693f-4354-83aa-01ff072767fa', 2]) storagereplication stdout | 2020-03-31 09:43:48,830 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 48, 48, 825894), True, u'93e98dd4-693f-4354-83aa-01ff072767fa', 2]) 2020-03-31 09:43:48,837 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 48, 835894), True, datetime.datetime(2020, 3, 31, 9, 43, 48, 835894), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:43:48,837 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 48, 835894), True, datetime.datetime(2020, 3, 31, 9, 43, 48, 835894), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:43:48,841 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 3, 48, 835894), 1, u'cd3aa7fd-a325-4cee-9271-371c0f41d8c5', 1, u'45d05454-9350-48bb-9e2a-fead2a46ef3d']) storagereplication stdout | 2020-03-31 09:43:48,841 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 3, 48, 835894), 1, u'cd3aa7fd-a325-4cee-9271-371c0f41d8c5', 1, u'45d05454-9350-48bb-9e2a-fead2a46ef3d']) 2020-03-31 09:43:48,847 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} storagereplication stdout | 2020-03-31 09:43:48,847 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:43:48,847 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:48,847 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 storagereplication stdout | 2020-03-31 09:43:48,847 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:48,847 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 2020-03-31 09:43:48,849 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:43:48,849 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:43:48,861 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:43:48,861 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:43:48,866 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) storagereplication stdout | 2020-03-31 09:43:48,866 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:43:48,869 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:43:48,869 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:43:48,870 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1f4aae22-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:43:48,870 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4aae22-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f4aae22-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Yx3B8Esv5taHbgti3s+xZH5bJi+XoMGTooHGVYOiK9g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. storagereplication stdout | 2020-03-31 09:43:48,870 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1f4aae22-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:43:48,870 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4aae22-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f4aae22-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Yx3B8Esv5taHbgti3s+xZH5bJi+XoMGTooHGVYOiK9g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. 2020-03-31 09:43:48,879 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:43:48,879 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:43:48,880 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4aae22-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=6e4de5d5-201e-0053-7840-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f4aae22-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Yx3B8Esv5taHbgti3s+xZH5bJi+XoMGTooHGVYOiK9g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. storagereplication stdout | 2020-03-31 09:43:48,880 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4aae22-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=6e4de5d5-201e-0053-7840-07fc5b000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f4aae22-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Yx3B8Esv5taHbgti3s+xZH5bJi+XoMGTooHGVYOiK9g=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. 2020-03-31 09:43:48,881 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:43:48,881 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:43:48,881 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:43:48,881 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:48,881 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:43:48,881 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:43:48,882 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:1f4c7a40-7334-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=c0mfSvs4PTx2Qq14BGF5D3OEszEOYKCtMG665YDLhbs%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A48Z x-ms-date:Tue, 31 Mar 2020 09:43:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:43:48,882 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4c7a40-7334-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=c0mfSvs4PTx2Qq14BGF5D3OEszEOYKCtMG665YDLhbs%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uwT9HayMI9cNDjZ+EZ7JwPcM54jBNU8Qh24vAuIBxMg=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1f4c7a40-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. storagereplication stdout | 2020-03-31 09:43:48,882 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:1f4c7a40-7334-11ea-b427-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=c0mfSvs4PTx2Qq14BGF5D3OEszEOYKCtMG665YDLhbs%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A48Z x-ms-date:Tue, 31 Mar 2020 09:43:48 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:43:48,882 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4c7a40-7334-11ea-b427-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=c0mfSvs4PTx2Qq14BGF5D3OEszEOYKCtMG665YDLhbs%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uwT9HayMI9cNDjZ+EZ7JwPcM54jBNU8Qh24vAuIBxMg=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1f4c7a40-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. 2020-03-31 09:43:48,929 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:43:48,929 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 2020-03-31 09:43:48,929 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4c7a40-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=6e4de5d8-201e-0053-7940-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=c0mfSvs4PTx2Qq14BGF5D3OEszEOYKCtMG665YDLhbs%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uwT9HayMI9cNDjZ+EZ7JwPcM54jBNU8Qh24vAuIBxMg=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1f4c7a40-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. storagereplication stdout | 2020-03-31 09:43:48,929 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f4c7a40-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=6e4de5d8-201e-0053-7940-07fc5b000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=c0mfSvs4PTx2Qq14BGF5D3OEszEOYKCtMG665YDLhbs%3D&sv=2017-07-29&se=2020-03-31T09%3A44%3A48Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:uwT9HayMI9cNDjZ+EZ7JwPcM54jBNU8Qh24vAuIBxMg=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': '1f4c7a40-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. 2020-03-31 09:43:48,930 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:43:48,930 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:43:48,930 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1f53e924-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:48 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:43:48,931 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f53e924-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:OD1nna/ZOUpV9dpbpXtRpPESEpZ2kwvoS1MMcT+WQSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. storagereplication stdout | 2020-03-31 09:43:48,930 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:1f53e924-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:48 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:43:48,931 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f53e924-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:OD1nna/ZOUpV9dpbpXtRpPESEpZ2kwvoS1MMcT+WQSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. 2020-03-31 09:43:49,013 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:43:49,013 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:49,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=630094a1-501e-0005-0140-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f53e924-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:OD1nna/ZOUpV9dpbpXtRpPESEpZ2kwvoS1MMcT+WQSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. 2020-03-31 09:43:49,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=630094a1-501e-0005-0140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:49,014 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=630094a1-501e-0005-0140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:49,014 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:43:49,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=630094a1-501e-0005-0140-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '1f53e924-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:OD1nna/ZOUpV9dpbpXtRpPESEpZ2kwvoS1MMcT+WQSA=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:48 GMT'}. 2020-03-31 09:43:49,014 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=630094a1-501e-0005-0140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:49,014 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=1f53e924-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:48 GMT, Server-Request-ID=630094a1-501e-0005-0140-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:49,014 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:43:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:43:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:54.323352+00:00 (in 4.823399 seconds) 2020-03-31 09:43:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:49 UTC)" (scheduled at 2020-03-31 09:43:49.498966+00:00) 2020-03-31 09:43:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:43:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:54.323352+00:00 (in 4.823399 seconds) 2020-03-31 09:43:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:49 UTC)" (scheduled at 2020-03-31 09:43:49.498966+00:00) 2020-03-31 09:43:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:43:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:49 UTC)" executed successfully 2020-03-31 09:43:51,017 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:20924132-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:51 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:43:51,017 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:20924132-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:51 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:43:51,017 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '20924132-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AV75rTxcKcvzqYLNPZAwQkX7ZP/NQi+/OY+Yj7hx7oM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:51 GMT'}. storagereplication stdout | 2020-03-31 09:43:51,017 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '20924132-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AV75rTxcKcvzqYLNPZAwQkX7ZP/NQi+/OY+Yj7hx7oM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:51 GMT'}. 2020-03-31 09:43:51,097 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:43:51,097 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:51,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:50 GMT, Server-Request-ID=63009cdf-501e-0005-3f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '20924132-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AV75rTxcKcvzqYLNPZAwQkX7ZP/NQi+/OY+Yj7hx7oM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:51 GMT'}. 2020-03-31 09:43:51,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:50 GMT, Server-Request-ID=63009cdf-501e-0005-3f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:51,098 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:50 GMT, Server-Request-ID=63009cdf-501e-0005-3f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:51,098 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:43:51,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:50 GMT, Server-Request-ID=63009cdf-501e-0005-3f40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '20924132-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AV75rTxcKcvzqYLNPZAwQkX7ZP/NQi+/OY+Yj7hx7oM=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:51 GMT'}. 2020-03-31 09:43:51,098 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:50 GMT, Server-Request-ID=63009cdf-501e-0005-3f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:51,098 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=20924132-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:50 GMT, Server-Request-ID=63009cdf-501e-0005-3f40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:51,098 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:43:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:43:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:59.560326+00:00 (in 8.448649 seconds) 2020-03-31 09:43:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:51 UTC)" (scheduled at 2020-03-31 09:43:51.110610+00:00) 2020-03-31 09:43:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:43:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:43:59.560326+00:00 (in 8.448649 seconds) 2020-03-31 09:43:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:51 UTC)" (scheduled at 2020-03-31 09:43:51.110610+00:00) 2020-03-31 09:43:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 51, 112653), True, datetime.datetime(2020, 3, 31, 9, 43, 51, 112653), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:43:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 51, 112653), True, datetime.datetime(2020, 3, 31, 9, 43, 51, 112653), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:43:51,128 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:43:51,128 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:51,128 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:51,128 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:43:51,128 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:51,128 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:51 UTC)" executed successfully 2020-03-31 09:43:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:43:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:01.440707+00:00 (in 9.999299 seconds) 2020-03-31 09:43:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:01 UTC)" (scheduled at 2020-03-31 09:43:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:43:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:01.440707+00:00 (in 9.999299 seconds) 2020-03-31 09:43:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:01 UTC)" (scheduled at 2020-03-31 09:43:51.440707+00:00) 2020-03-31 09:43:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:43:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:43:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:43:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:43:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:43:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:01 UTC)" executed successfully 2020-03-31 09:43:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:43:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:22.113221+00:00 (in 29.997831 seconds) 2020-03-31 09:43:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:22 UTC)" (scheduled at 2020-03-31 09:43:52.113221+00:00) securityworker stdout | 2020-03-31 09:43:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:22.113221+00:00 (in 29.997831 seconds) 2020-03-31 09:43:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:22 UTC)" (scheduled at 2020-03-31 09:43:52.113221+00:00) 2020-03-31 09:43:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:43:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:43:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:43:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:43:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:43:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:43:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:43:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:22 UTC)" executed successfully 2020-03-31 09:43:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:43:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:04.323352+00:00 (in 9.998775 seconds) 2020-03-31 09:43:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:04 UTC)" (scheduled at 2020-03-31 09:43:54.323352+00:00) 2020-03-31 09:43:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:43:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:04.323352+00:00 (in 9.998775 seconds) 2020-03-31 09:43:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:04 UTC)" (scheduled at 2020-03-31 09:43:54.323352+00:00) 2020-03-31 09:43:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:43:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 54, 325429), True, datetime.datetime(2020, 3, 31, 9, 43, 54, 325429), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:43:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 43, 54, 325429), True, datetime.datetime(2020, 3, 31, 9, 43, 54, 325429), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:43:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:43:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:43:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:43:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:04 UTC)" executed successfully 2020-03-31 09:43:54,446 [518] [DEBUG] [app] Starting request: urn:request:261fe3c4-f3b6-484c-becb-21628ecae010 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:54,446 [518] [DEBUG] [app] Starting request: urn:request:261fe3c4-f3b6-484c-becb-21628ecae010 (/health/instance) 2020-03-31 09:43:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:54,453 [538] [DEBUG] [app] Starting request: urn:request:22b92f99-4ea9-40af-a8a8-e1a899efef6e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:54,453 [538] [DEBUG] [app] Starting request: urn:request:22b92f99-4ea9-40af-a8a8-e1a899efef6e (/v1/_internal_ping) 2020-03-31 09:43:54,455 [538] [DEBUG] [app] Ending request: urn:request:22b92f99-4ea9-40af-a8a8-e1a899efef6e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:54,455 [538] [DEBUG] [app] Ending request: urn:request:22b92f99-4ea9-40af-a8a8-e1a899efef6e (/v1/_internal_ping) 2020-03-31 09:43:54,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:43:54,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:43:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:43:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:43:54,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:54,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:54,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:54,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:54,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:54,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:54,485 [527] [DEBUG] [app] Starting request: urn:request:e57b954b-2a88-4313-935a-f374202450a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:54,485 [527] [DEBUG] [app] Starting request: urn:request:e57b954b-2a88-4313-935a-f374202450a0 (/_internal_ping) 2020-03-31 09:43:54,486 [527] [DEBUG] [app] Ending request: urn:request:e57b954b-2a88-4313-935a-f374202450a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:54,486 [527] [DEBUG] [app] Ending request: urn:request:e57b954b-2a88-4313-935a-f374202450a0 (/_internal_ping) 2020-03-31 09:43:54,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:43:54,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:54,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:43:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:54,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:43:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:43:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:43:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:54,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:54,501 [535] [DEBUG] [app] Starting request: urn:request:857dbb70-558e-47cc-86cf-08a0fb4a42cd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:54,501 [535] [DEBUG] [app] Starting request: urn:request:857dbb70-558e-47cc-86cf-08a0fb4a42cd (/c1/_internal_ping) 2020-03-31 09:43:54,501 [535] [DEBUG] [app] Ending request: urn:request:857dbb70-558e-47cc-86cf-08a0fb4a42cd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:54,501 [535] [DEBUG] [app] Ending request: urn:request:857dbb70-558e-47cc-86cf-08a0fb4a42cd (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:54,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:43:54,502 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:54,503 [518] [DEBUG] [app] Ending request: urn:request:261fe3c4-f3b6-484c-becb-21628ecae010 (/health/instance) 2020-03-31 09:43:54,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:54,503 [518] [DEBUG] [app] Ending request: urn:request:261fe3c4-f3b6-484c-becb-21628ecae010 (/health/instance) 2020-03-31 09:43:54,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:43:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:43:55,102 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:23019ecc-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:55 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:43:55,102 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:23019ecc-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:43:55 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:43:55,102 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '23019ecc-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Lj5yzGf++vV+lzGxeWrfBrcpL9Y0BDuav5WJo27lSeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:55 GMT'}. storagereplication stdout | 2020-03-31 09:43:55,102 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '23019ecc-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Lj5yzGf++vV+lzGxeWrfBrcpL9Y0BDuav5WJo27lSeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:55 GMT'}. 2020-03-31 09:43:55,183 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:43:55,183 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:43:55,184 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:54 GMT, Server-Request-ID=6300ab01-501e-0005-0b40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '23019ecc-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Lj5yzGf++vV+lzGxeWrfBrcpL9Y0BDuav5WJo27lSeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:55 GMT'}. 2020-03-31 09:43:55,184 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:54 GMT, Server-Request-ID=6300ab01-501e-0005-0b40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:55,184 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:54 GMT, Server-Request-ID=6300ab01-501e-0005-0b40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:43:55,184 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:43:54 GMT, Server-Request-ID=6300ab01-501e-0005-0b40-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '23019ecc-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Lj5yzGf++vV+lzGxeWrfBrcpL9Y0BDuav5WJo27lSeI=', 'x-ms-date': 'Tue, 31 Mar 2020 09:43:55 GMT'}. 2020-03-31 09:43:55,184 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:43:54 GMT, Server-Request-ID=6300ab01-501e-0005-0b40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:55,184 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=23019ecc-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:43:54 GMT, Server-Request-ID=6300ab01-501e-0005-0b40-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:43:55,184 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:43:55,184 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:43:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:43:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:06.924497+00:00 (in 9.999254 seconds) storagereplication stdout | 2020-03-31 09:43:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:43:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:43:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:06.924497+00:00 (in 9.999254 seconds) 2020-03-31 09:43:58,273 [518] [DEBUG] [app] Starting request: urn:request:c5875f2f-d7bf-4767-a546-41841094c2b6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:43:58,273 [518] [DEBUG] [app] Starting request: urn:request:c5875f2f-d7bf-4767-a546-41841094c2b6 (/health/instance) 2020-03-31 09:43:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:58,278 [555] [DEBUG] [app] Starting request: urn:request:e45d49e1-76af-4abb-9a29-e477112dc05f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:58,278 [555] [DEBUG] [app] Starting request: urn:request:e45d49e1-76af-4abb-9a29-e477112dc05f (/v1/_internal_ping) 2020-03-31 09:43:58,279 [555] [DEBUG] [app] Ending request: urn:request:e45d49e1-76af-4abb-9a29-e477112dc05f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:43:58,279 [555] [DEBUG] [app] Ending request: urn:request:e45d49e1-76af-4abb-9a29-e477112dc05f (/v1/_internal_ping) 2020-03-31 09:43:58,279 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:43:58,279 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:43:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:43:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:58,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:43:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:43:58,282 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:43:58,283 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:43:58,283 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:43:58,284 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:43:58,284 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:43:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:43:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:43:58,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:43:58,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:43:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:58,311 [527] [DEBUG] [app] Starting request: urn:request:af0d986e-946b-4b43-a2d4-ac44fcab6b5a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:58,311 [527] [DEBUG] [app] Starting request: urn:request:af0d986e-946b-4b43-a2d4-ac44fcab6b5a (/_internal_ping) 2020-03-31 09:43:58,312 [527] [DEBUG] [app] Ending request: urn:request:af0d986e-946b-4b43-a2d4-ac44fcab6b5a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:43:58,312 [527] [DEBUG] [app] Ending request: urn:request:af0d986e-946b-4b43-a2d4-ac44fcab6b5a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:43:58,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:43:58,312 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:58,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:43:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:43:58,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:43:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:43:58,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:43:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:43:58,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:43:58,330 [537] [DEBUG] [app] Starting request: urn:request:90874ea9-1193-4df7-934b-54d5b88b898d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:58,330 [537] [DEBUG] [app] Starting request: urn:request:90874ea9-1193-4df7-934b-54d5b88b898d (/c1/_internal_ping) 2020-03-31 09:43:58,331 [537] [DEBUG] [app] Ending request: urn:request:90874ea9-1193-4df7-934b-54d5b88b898d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:43:58,331 [537] [DEBUG] [app] Ending request: urn:request:90874ea9-1193-4df7-934b-54d5b88b898d (/c1/_internal_ping) 2020-03-31 09:43:58,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:43:58,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:43:58,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:43:58,332 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:43:58,334 [518] [DEBUG] [app] Ending request: urn:request:c5875f2f-d7bf-4767-a546-41841094c2b6 (/health/instance) 2020-03-31 09:43:58,334 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:43:58,334 [518] [DEBUG] [app] Ending request: urn:request:c5875f2f-d7bf-4767-a546-41841094c2b6 (/health/instance) 2020-03-31 09:43:58,334 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:43:58,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:43:58,335 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:43:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:43:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:43:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:43:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:43:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:51.110610+00:00 (in 51.549140 seconds) 2020-03-31 09:43:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:59 UTC)" (scheduled at 2020-03-31 09:43:59.560326+00:00) 2020-03-31 09:43:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:43:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:51.110610+00:00 (in 51.549140 seconds) 2020-03-31 09:43:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:59 UTC)" (scheduled at 2020-03-31 09:43:59.560326+00:00) 2020-03-31 09:43:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:43:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:59 UTC)" executed successfully 2020-03-31 09:44:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:44:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:30.337699+00:00 (in 29.999016 seconds) 2020-03-31 09:44:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:30 UTC)" (scheduled at 2020-03-31 09:44:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:44:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:30.337699+00:00 (in 29.999016 seconds) 2020-03-31 09:44:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:30 UTC)" (scheduled at 2020-03-31 09:44:00.337699+00:00) 2020-03-31 09:44:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 44, 0, 339594), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:44:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 44, 0, 339594), False, 50, 1, 0]) 2020-03-31 09:44:00,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:44:00,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:44:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:44:00,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:00,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:30 UTC)" executed successfully 2020-03-31 09:44:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:44:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:01 UTC)" (scheduled at 2020-03-31 09:44:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:44:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:01 UTC)" (scheduled at 2020-03-31 09:44:01.440707+00:00) 2020-03-31 09:44:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:11.440707+00:00 (in 9.998994 seconds) repositoryactioncounter stdout | 2020-03-31 09:44:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:11.440707+00:00 (in 9.998994 seconds) 2020-03-31 09:44:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:44:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:44:01,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:44:01,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:44:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:44:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:11 UTC)" executed successfully 2020-03-31 09:44:03,189 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:27d3a3a0-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:44:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:44:03,189 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:27d3a3a0-7334-11ea-b427-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:44:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:44:03,190 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '27d3a3a0-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hbbgO0LATIjTLbK/Su5Fh3Xw6NHHqIeVACAmM3YJAWs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:44:03 GMT'}. storagereplication stdout | 2020-03-31 09:44:03,190 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '27d3a3a0-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hbbgO0LATIjTLbK/Su5Fh3Xw6NHHqIeVACAmM3YJAWs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:44:03 GMT'}. 2020-03-31 09:44:03,270 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:44:03,270 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:44:03,271 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:44:02 GMT, Server-Request-ID=6300c74f-501e-0005-2640-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '27d3a3a0-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hbbgO0LATIjTLbK/Su5Fh3Xw6NHHqIeVACAmM3YJAWs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:44:03 GMT'}. 2020-03-31 09:44:03,271 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:44:02 GMT, Server-Request-ID=6300c74f-501e-0005-2640-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:44:03,271 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:44:02 GMT, Server-Request-ID=6300c74f-501e-0005-2640-075c3a000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': '27d3a3a0-7334-11ea-b427-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hbbgO0LATIjTLbK/Su5Fh3Xw6NHHqIeVACAmM3YJAWs=', 'x-ms-date': 'Tue, 31 Mar 2020 09:44:03 GMT'}. 2020-03-31 09:44:03,271 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:44:02 GMT, Server-Request-ID=6300c74f-501e-0005-2640-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:44:03,271 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:44:02 GMT, Server-Request-ID=6300c74f-501e-0005-2640-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:44:03,271 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:44:03,271 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=27d3a3a0-7334-11ea-b427-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:44:02 GMT, Server-Request-ID=6300c74f-501e-0005-2640-075c3a000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:44:03,271 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:44:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:44:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:23.410371+00:00 (in 20.079288 seconds) 2020-03-31 09:44:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:33 UTC)" (scheduled at 2020-03-31 09:44:03.330112+00:00) 2020-03-31 09:44:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:44:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:23.410371+00:00 (in 20.079288 seconds) 2020-03-31 09:44:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:33 UTC)" (scheduled at 2020-03-31 09:44:03.330112+00:00) 2020-03-31 09:44:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 3, 332174), True, datetime.datetime(2020, 3, 31, 9, 44, 3, 332174), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:44:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 3, 332174), True, datetime.datetime(2020, 3, 31, 9, 44, 3, 332174), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:44:03,355 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:44:03,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:44:03,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:03,355 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:33 UTC)" executed successfully 2020-03-31 09:44:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:44:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:14.323352+00:00 (in 9.998837 seconds) 2020-03-31 09:44:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:14 UTC)" (scheduled at 2020-03-31 09:44:04.323352+00:00) 2020-03-31 09:44:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:44:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:14.323352+00:00 (in 9.998837 seconds) 2020-03-31 09:44:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:14 UTC)" (scheduled at 2020-03-31 09:44:04.323352+00:00) 2020-03-31 09:44:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 4, 325391), True, datetime.datetime(2020, 3, 31, 9, 44, 4, 325391), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:44:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 4, 325391), True, datetime.datetime(2020, 3, 31, 9, 44, 4, 325391), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:44:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:44:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:44:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:14 UTC)" executed successfully 2020-03-31 09:44:04,445 [518] [DEBUG] [app] Starting request: urn:request:632f39ac-2a7c-4046-b862-3c84c8a902fb (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:04,445 [518] [DEBUG] [app] Starting request: urn:request:632f39ac-2a7c-4046-b862-3c84c8a902fb (/health/instance) 2020-03-31 09:44:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:04,453 [551] [DEBUG] [app] Starting request: urn:request:232dc598-d3ba-4661-811a-be7372e83375 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:04,453 [551] [DEBUG] [app] Starting request: urn:request:232dc598-d3ba-4661-811a-be7372e83375 (/v1/_internal_ping) 2020-03-31 09:44:04,454 [551] [DEBUG] [app] Ending request: urn:request:232dc598-d3ba-4661-811a-be7372e83375 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:04,454 [551] [DEBUG] [app] Ending request: urn:request:232dc598-d3ba-4661-811a-be7372e83375 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:04,454 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:44:04,454 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:04,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:04,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:44:04,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:44:04,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:04,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:04,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:04,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:04,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:04,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:04,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:04,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:04,486 [521] [DEBUG] [app] Starting request: urn:request:0503b1dc-944d-4de7-bab0-271b001a3b46 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:04,486 [521] [DEBUG] [app] Starting request: urn:request:0503b1dc-944d-4de7-bab0-271b001a3b46 (/_internal_ping) 2020-03-31 09:44:04,487 [521] [DEBUG] [app] Ending request: urn:request:0503b1dc-944d-4de7-bab0-271b001a3b46 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:04,487 [521] [DEBUG] [app] Ending request: urn:request:0503b1dc-944d-4de7-bab0-271b001a3b46 (/_internal_ping) 2020-03-31 09:44:04,488 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:44:04,488 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:04,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:44:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:04,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:04,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:44:04,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:04,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:04,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:04,502 [534] [DEBUG] [app] Starting request: urn:request:2ed2c0e6-4f7d-4dc9-9180-581dc0194e66 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:04,502 [534] [DEBUG] [app] Starting request: urn:request:2ed2c0e6-4f7d-4dc9-9180-581dc0194e66 (/c1/_internal_ping) 2020-03-31 09:44:04,503 [534] [DEBUG] [app] Ending request: urn:request:2ed2c0e6-4f7d-4dc9-9180-581dc0194e66 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:04,503 [534] [DEBUG] [app] Ending request: urn:request:2ed2c0e6-4f7d-4dc9-9180-581dc0194e66 (/c1/_internal_ping) 2020-03-31 09:44:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:04,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:44:04,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:04,506 [518] [DEBUG] [app] Ending request: urn:request:632f39ac-2a7c-4046-b862-3c84c8a902fb (/health/instance) 2020-03-31 09:44:04,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:04,506 [518] [DEBUG] [app] Ending request: urn:request:632f39ac-2a7c-4046-b862-3c84c8a902fb (/health/instance) 2020-03-31 09:44:04,506 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:44:04,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:44:04,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:44:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:44:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:44:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:16.924497+00:00 (in 9.999191 seconds) storagereplication stdout | 2020-03-31 09:44:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:44:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:16.924497+00:00 (in 9.999191 seconds) 2020-03-31 09:44:08,273 [518] [DEBUG] [app] Starting request: urn:request:c53027f6-6768-46e2-8066-b98898b83f37 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:08,273 [518] [DEBUG] [app] Starting request: urn:request:c53027f6-6768-46e2-8066-b98898b83f37 (/health/instance) 2020-03-31 09:44:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:08,278 [548] [DEBUG] [app] Starting request: urn:request:4acee929-ffac-4436-b43e-07ddf9985eb1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:08,278 [548] [DEBUG] [app] Starting request: urn:request:4acee929-ffac-4436-b43e-07ddf9985eb1 (/v1/_internal_ping) 2020-03-31 09:44:08,279 [548] [DEBUG] [app] Ending request: urn:request:4acee929-ffac-4436-b43e-07ddf9985eb1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:08,279 [548] [DEBUG] [app] Ending request: urn:request:4acee929-ffac-4436-b43e-07ddf9985eb1 (/v1/_internal_ping) 2020-03-31 09:44:08,280 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:44:08,280 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:08,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:08,282 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:08,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:08,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:08,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:08,284 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:08,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:08,290 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:08,314 [518] [DEBUG] [app] Starting request: urn:request:c969ff5e-9d03-4d97-b8bd-f7b5c0b61110 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:08,314 [518] [DEBUG] [app] Starting request: urn:request:c969ff5e-9d03-4d97-b8bd-f7b5c0b61110 (/_internal_ping) 2020-03-31 09:44:08,315 [518] [DEBUG] [app] Ending request: urn:request:c969ff5e-9d03-4d97-b8bd-f7b5c0b61110 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:08,315 [518] [DEBUG] [app] Ending request: urn:request:c969ff5e-9d03-4d97-b8bd-f7b5c0b61110 (/_internal_ping) 2020-03-31 09:44:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:44:08,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:44:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:44:08,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:08,328 [535] [DEBUG] [app] Starting request: urn:request:ab2fa092-e91d-4945-8b00-c5445a058741 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:08,328 [535] [DEBUG] [app] Starting request: urn:request:ab2fa092-e91d-4945-8b00-c5445a058741 (/c1/_internal_ping) 2020-03-31 09:44:08,329 [535] [DEBUG] [app] Ending request: urn:request:ab2fa092-e91d-4945-8b00-c5445a058741 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:08,329 [535] [DEBUG] [app] Ending request: urn:request:ab2fa092-e91d-4945-8b00-c5445a058741 (/c1/_internal_ping) 2020-03-31 09:44:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:08,330 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:44:08,330 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:08,331 [518] [DEBUG] [app] Ending request: urn:request:c53027f6-6768-46e2-8066-b98898b83f37 (/health/instance) 2020-03-31 09:44:08,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:08,331 [518] [DEBUG] [app] Ending request: urn:request:c53027f6-6768-46e2-8066-b98898b83f37 (/health/instance) 2020-03-31 09:44:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:44:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:44:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:44:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:21.440707+00:00 (in 9.996031 seconds) 2020-03-31 09:44:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:21 UTC)" (scheduled at 2020-03-31 09:44:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:44:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:21.440707+00:00 (in 9.996031 seconds) 2020-03-31 09:44:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:21 UTC)" (scheduled at 2020-03-31 09:44:11.440707+00:00) 2020-03-31 09:44:11,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:44:11,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:44:11,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:44:11,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:44:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:44:11,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:11,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:21 UTC)" executed successfully 2020-03-31 09:44:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:44:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:24.323352+00:00 (in 9.998899 seconds) 2020-03-31 09:44:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:24 UTC)" (scheduled at 2020-03-31 09:44:14.323352+00:00) 2020-03-31 09:44:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:44:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:24.323352+00:00 (in 9.998899 seconds) 2020-03-31 09:44:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:24 UTC)" (scheduled at 2020-03-31 09:44:14.323352+00:00) 2020-03-31 09:44:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 14, 325495), True, datetime.datetime(2020, 3, 31, 9, 44, 14, 325495), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:44:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 14, 325495), True, datetime.datetime(2020, 3, 31, 9, 44, 14, 325495), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:44:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:44:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:44:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:24 UTC)" executed successfully 2020-03-31 09:44:14,446 [527] [DEBUG] [app] Starting request: urn:request:80308e95-d7e4-4b98-bf58-d44c81aa47e2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:14,446 [527] [DEBUG] [app] Starting request: urn:request:80308e95-d7e4-4b98-bf58-d44c81aa47e2 (/health/instance) 2020-03-31 09:44:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:14,454 [540] [DEBUG] [app] Starting request: urn:request:46cb37a3-9065-44f1-a55e-597fcd493023 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:14,454 [540] [DEBUG] [app] Starting request: urn:request:46cb37a3-9065-44f1-a55e-597fcd493023 (/v1/_internal_ping) 2020-03-31 09:44:14,455 [540] [DEBUG] [app] Ending request: urn:request:46cb37a3-9065-44f1-a55e-597fcd493023 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:14,455 [540] [DEBUG] [app] Ending request: urn:request:46cb37a3-9065-44f1-a55e-597fcd493023 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:44:14,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:44:14,456 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:14,459 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:44:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:44:14,459 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:14,460 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:14,460 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:14,461 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:14,461 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:14,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:14,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:14,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:14,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:14,489 [525] [DEBUG] [app] Starting request: urn:request:23e04673-b852-4340-9f19-c8bec2b97652 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:14,489 [525] [DEBUG] [app] Starting request: urn:request:23e04673-b852-4340-9f19-c8bec2b97652 (/_internal_ping) 2020-03-31 09:44:14,490 [525] [DEBUG] [app] Ending request: urn:request:23e04673-b852-4340-9f19-c8bec2b97652 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:14,490 [525] [DEBUG] [app] Ending request: urn:request:23e04673-b852-4340-9f19-c8bec2b97652 (/_internal_ping) 2020-03-31 09:44:14,491 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:14,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:44:14,491 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:14,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:44:14,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:14,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:14,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:14,496 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:44:14,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:14,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:14,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:14,504 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:14,506 [535] [DEBUG] [app] Starting request: urn:request:c6e19731-e846-42b0-982a-57e673517259 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:14,506 [535] [DEBUG] [app] Starting request: urn:request:c6e19731-e846-42b0-982a-57e673517259 (/c1/_internal_ping) 2020-03-31 09:44:14,507 [535] [DEBUG] [app] Ending request: urn:request:c6e19731-e846-42b0-982a-57e673517259 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:14,507 [535] [DEBUG] [app] Ending request: urn:request:c6e19731-e846-42b0-982a-57e673517259 (/c1/_internal_ping) 2020-03-31 09:44:14,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:14,508 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:14,508 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:14,509 [527] [DEBUG] [app] Ending request: urn:request:80308e95-d7e4-4b98-bf58-d44c81aa47e2 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:44:14,508 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:14,509 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:14,509 [527] [DEBUG] [app] Ending request: urn:request:80308e95-d7e4-4b98-bf58-d44c81aa47e2 (/health/instance) 2020-03-31 09:44:14,510 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:44:14,509 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:14,510 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:44:15,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:44:15,066 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:45.064514+00:00 (in 29.997991 seconds) 2020-03-31 09:44:15,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:45 UTC)" (scheduled at 2020-03-31 09:44:15.064514+00:00) gcworker stdout | 2020-03-31 09:44:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:45.064514+00:00 (in 29.997991 seconds) 2020-03-31 09:44:15,067 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:45 UTC)" (scheduled at 2020-03-31 09:44:15.064514+00:00) 2020-03-31 09:44:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438255, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:44:15,069 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438255, 1209600, 500, 1, 0]) 2020-03-31 09:44:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:44:15,085 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:44:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:44:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:44:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:45 UTC)" executed successfully 2020-03-31 09:44:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:44:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:44:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:26.924497+00:00 (in 9.999232 seconds) storagereplication stdout | 2020-03-31 09:44:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:44:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:26.924497+00:00 (in 9.999232 seconds) 2020-03-31 09:44:18,273 [518] [DEBUG] [app] Starting request: urn:request:fef0f4c5-f02e-4914-9642-5aca88e74813 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:18,273 [518] [DEBUG] [app] Starting request: urn:request:fef0f4c5-f02e-4914-9642-5aca88e74813 (/health/instance) 2020-03-31 09:44:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:18,281 [548] [DEBUG] [app] Starting request: urn:request:c6a4fc87-97c9-4d28-8352-de1080af3f1c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:18,281 [548] [DEBUG] [app] Starting request: urn:request:c6a4fc87-97c9-4d28-8352-de1080af3f1c (/v1/_internal_ping) 2020-03-31 09:44:18,282 [548] [DEBUG] [app] Ending request: urn:request:c6a4fc87-97c9-4d28-8352-de1080af3f1c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:18,282 [548] [DEBUG] [app] Ending request: urn:request:c6a4fc87-97c9-4d28-8352-de1080af3f1c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:18,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:44:18,282 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:18,285 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:18,285 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:18,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:18,286 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:18,287 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:18,287 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:18,339 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:18,339 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:18,354 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:18,354 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:18,357 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:18,357 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:18,359 [518] [DEBUG] [app] Starting request: urn:request:9acce009-d392-4db7-b2f2-91d1d8ce238a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:18,359 [518] [DEBUG] [app] Starting request: urn:request:9acce009-d392-4db7-b2f2-91d1d8ce238a (/_internal_ping) 2020-03-31 09:44:18,359 [518] [DEBUG] [app] Ending request: urn:request:9acce009-d392-4db7-b2f2-91d1d8ce238a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:18,359 [518] [DEBUG] [app] Ending request: urn:request:9acce009-d392-4db7-b2f2-91d1d8ce238a (/_internal_ping) 2020-03-31 09:44:18,360 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:44:18,360 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:18,360 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:44:18,360 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:18,363 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:18,363 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:18,364 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:18,364 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:18,366 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:18,366 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:44:18,368 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:18,368 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:18,371 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:18,371 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:18,373 [535] [DEBUG] [app] Starting request: urn:request:1afb241a-2cb4-4ed6-8ecb-525b788f2042 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:18,373 [535] [DEBUG] [app] Starting request: urn:request:1afb241a-2cb4-4ed6-8ecb-525b788f2042 (/c1/_internal_ping) 2020-03-31 09:44:18,374 [535] [DEBUG] [app] Ending request: urn:request:1afb241a-2cb4-4ed6-8ecb-525b788f2042 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:18,374 [535] [DEBUG] [app] Ending request: urn:request:1afb241a-2cb4-4ed6-8ecb-525b788f2042 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:18,375 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:18,374 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:18,375 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:44:18,374 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:18,376 [518] [DEBUG] [app] Ending request: urn:request:fef0f4c5-f02e-4914-9642-5aca88e74813 (/health/instance) 2020-03-31 09:44:18,376 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:18,376 [518] [DEBUG] [app] Ending request: urn:request:fef0f4c5-f02e-4914-9642-5aca88e74813 (/health/instance) 2020-03-31 09:44:18,376 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:18,377 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.104 49 0.104) gunicorn-web stdout | 2020-03-31 09:44:18,377 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.104 49 0.104) 2020-03-31 09:44:19,281 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:44:19,281 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy 2020-03-31 09:44:19,281 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:44:19,281 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:44:19,281 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:44:19,281 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:44:19,292 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) storagereplication stdout | 2020-03-31 09:44:19,292 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) 2020-03-31 09:44:19,296 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 49, 19, 291783), True, u'5a59868b-ad0f-4f19-8f8f-2a4de8685fec', 1]) storagereplication stdout | 2020-03-31 09:44:19,296 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 49, 19, 291783), True, u'5a59868b-ad0f-4f19-8f8f-2a4de8685fec', 1]) 2020-03-31 09:44:19,302 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 19, 301137), True, datetime.datetime(2020, 3, 31, 9, 44, 19, 301137), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:44:19,302 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 19, 301137), True, datetime.datetime(2020, 3, 31, 9, 44, 19, 301137), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:44:19,307 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:44:19,307 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:19,307 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:19,307 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:44:19,307 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:19,307 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:26 UTC)" executed successfully 2020-03-31 09:44:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:44:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:31.440707+00:00 (in 9.996123 seconds) 2020-03-31 09:44:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:31 UTC)" (scheduled at 2020-03-31 09:44:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:44:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:31.440707+00:00 (in 9.996123 seconds) 2020-03-31 09:44:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:31 UTC)" (scheduled at 2020-03-31 09:44:21.440707+00:00) 2020-03-31 09:44:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:44:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:44:21,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:44:21,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:44:21,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:21,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:44:21,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:21,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:31 UTC)" executed successfully 2020-03-31 09:44:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:44:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:52.113221+00:00 (in 29.999247 seconds) 2020-03-31 09:44:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:52 UTC)" (scheduled at 2020-03-31 09:44:22.113221+00:00) securityworker stdout | 2020-03-31 09:44:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:52.113221+00:00 (in 29.999247 seconds) 2020-03-31 09:44:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:52 UTC)" (scheduled at 2020-03-31 09:44:22.113221+00:00) 2020-03-31 09:44:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:44:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:44:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:44:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:44:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:44:22,131 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:44:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:44:22,131 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:44:22,131 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:44:52 UTC)" executed successfully 2020-03-31 09:44:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:44:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:23,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:24.410927+00:00 (in 0.999303 seconds) 2020-03-31 09:44:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:23 UTC)" (scheduled at 2020-03-31 09:44:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:44:23,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:24.410927+00:00 (in 0.999303 seconds) 2020-03-31 09:44:23,412 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:23 UTC)" (scheduled at 2020-03-31 09:44:23.410371+00:00) 2020-03-31 09:44:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:44:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), u'security_notification/%']) 2020-03-31 09:44:23,428 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), True, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:44:23,428 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), True, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), 0, u'security_notification/%']) 2020-03-31 09:44:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), True, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:44:23,433 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), True, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 44, 23, 412603), u'security_notification/%']) 2020-03-31 09:44:23,437 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:44:23,437 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:44:23,440 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:23,440 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:23 UTC)" executed successfully 2020-03-31 09:44:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:44:23,440 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:23 UTC)" executed successfully 2020-03-31 09:44:23,441 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:44:23,442 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:44:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:34.323352+00:00 (in 9.998961 seconds) notificationworker stdout | 2020-03-31 09:44:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:34.323352+00:00 (in 9.998961 seconds) 2020-03-31 09:44:24,329 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:34 UTC)" (scheduled at 2020-03-31 09:44:24.323352+00:00) 2020-03-31 09:44:24,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:44:24,329 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:34 UTC)" (scheduled at 2020-03-31 09:44:24.323352+00:00) 2020-03-31 09:44:24,329 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:24,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 24, 329538), True, datetime.datetime(2020, 3, 31, 9, 44, 24, 329538), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:44:24,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 24, 329538), True, datetime.datetime(2020, 3, 31, 9, 44, 24, 329538), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:44:24,346 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:44:24,346 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:24,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:44:24,346 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:24,346 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:34 UTC)" executed successfully 2020-03-31 09:44:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:44:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:33.330112+00:00 (in 8.918065 seconds) 2020-03-31 09:44:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:24 UTC)" (scheduled at 2020-03-31 09:44:24.410927+00:00) 2020-03-31 09:44:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:44:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:33.330112+00:00 (in 8.918065 seconds) 2020-03-31 09:44:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:24 UTC)" (scheduled at 2020-03-31 09:44:24.410927+00:00) 2020-03-31 09:44:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:24 UTC)" executed successfully 2020-03-31 09:44:24,445 [527] [DEBUG] [app] Starting request: urn:request:19fd6556-4d9b-4bc2-8083-a97920344613 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:24,445 [527] [DEBUG] [app] Starting request: urn:request:19fd6556-4d9b-4bc2-8083-a97920344613 (/health/instance) 2020-03-31 09:44:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:24,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:24,453 [553] [DEBUG] [app] Starting request: urn:request:786c302c-e598-453d-b7d9-5138597e1708 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:24,453 [553] [DEBUG] [app] Starting request: urn:request:786c302c-e598-453d-b7d9-5138597e1708 (/v1/_internal_ping) 2020-03-31 09:44:24,454 [553] [DEBUG] [app] Ending request: urn:request:786c302c-e598-453d-b7d9-5138597e1708 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:24,454 [553] [DEBUG] [app] Ending request: urn:request:786c302c-e598-453d-b7d9-5138597e1708 (/v1/_internal_ping) 2020-03-31 09:44:24,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:44:24,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:24,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:24,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:24,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:24,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:24,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:24,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:24,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:24,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:24,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:24,460 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:24,464 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:24,464 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:24,477 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:24,477 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:24,481 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:24,483 [527] [DEBUG] [app] Starting request: urn:request:4548d067-c6cb-48d2-8788-d056fdfdc5b4 (/_internal_ping) 2020-03-31 09:44:24,484 [527] [DEBUG] [app] Ending request: urn:request:4548d067-c6cb-48d2-8788-d056fdfdc5b4 (/_internal_ping) 2020-03-31 09:44:24,485 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:24,485 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:44:24,481 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:24,483 [527] [DEBUG] [app] Starting request: urn:request:4548d067-c6cb-48d2-8788-d056fdfdc5b4 (/_internal_ping) 2020-03-31 09:44:24,484 [527] [DEBUG] [app] Ending request: urn:request:4548d067-c6cb-48d2-8788-d056fdfdc5b4 (/_internal_ping) 2020-03-31 09:44:24,485 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:24,485 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:44:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:24,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:24,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:24,499 [535] [DEBUG] [app] Starting request: urn:request:efd8e959-3349-4e84-8e8d-01c2e7fc7eec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:24,499 [535] [DEBUG] [app] Starting request: urn:request:efd8e959-3349-4e84-8e8d-01c2e7fc7eec (/c1/_internal_ping) 2020-03-31 09:44:24,499 [535] [DEBUG] [app] Ending request: urn:request:efd8e959-3349-4e84-8e8d-01c2e7fc7eec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:24,499 [535] [DEBUG] [app] Ending request: urn:request:efd8e959-3349-4e84-8e8d-01c2e7fc7eec (/c1/_internal_ping) 2020-03-31 09:44:24,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:24,500 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:24,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:24,501 [527] [DEBUG] [app] Ending request: urn:request:19fd6556-4d9b-4bc2-8083-a97920344613 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:44:24,500 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:24,501 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:24,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:44:24,501 [527] [DEBUG] [app] Ending request: urn:request:19fd6556-4d9b-4bc2-8083-a97920344613 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:24,501 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:24,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:44:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:44:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:36.924497+00:00 (in 9.999092 seconds) 2020-03-31 09:44:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:36 UTC)" (scheduled at 2020-03-31 09:44:26.924497+00:00) 2020-03-31 09:44:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:44:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:36.924497+00:00 (in 9.999092 seconds) 2020-03-31 09:44:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:36 UTC)" (scheduled at 2020-03-31 09:44:26.924497+00:00) 2020-03-31 09:44:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 26, 926204), True, datetime.datetime(2020, 3, 31, 9, 44, 26, 926204), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:44:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 26, 926204), True, datetime.datetime(2020, 3, 31, 9, 44, 26, 926204), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:44:26,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:44:26,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:44:26,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:36 UTC)" executed successfully 2020-03-31 09:44:28,273 [527] [DEBUG] [app] Starting request: urn:request:fd8fa6ca-1fd2-4582-8141-0af4b61c5464 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:28,273 [527] [DEBUG] [app] Starting request: urn:request:fd8fa6ca-1fd2-4582-8141-0af4b61c5464 (/health/instance) 2020-03-31 09:44:28,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:28,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:28,279 [556] [DEBUG] [app] Starting request: urn:request:e270f90c-ebf3-4392-bb85-a26207398e02 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:28,279 [556] [DEBUG] [app] Starting request: urn:request:e270f90c-ebf3-4392-bb85-a26207398e02 (/v1/_internal_ping) 2020-03-31 09:44:28,280 [556] [DEBUG] [app] Ending request: urn:request:e270f90c-ebf3-4392-bb85-a26207398e02 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:28,280 [556] [DEBUG] [app] Ending request: urn:request:e270f90c-ebf3-4392-bb85-a26207398e02 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:44:28,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:28,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:28,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:44:28,281 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:28,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:28,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:28,284 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:28,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:28,285 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:28,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:28,286 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:28,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:28,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:28,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:28,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:28,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:28,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:28,312 [526] [DEBUG] [app] Starting request: urn:request:9cde5c8a-55d7-4e83-a90e-6d7368fa9b41 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:28,312 [526] [DEBUG] [app] Starting request: urn:request:9cde5c8a-55d7-4e83-a90e-6d7368fa9b41 (/_internal_ping) 2020-03-31 09:44:28,313 [526] [DEBUG] [app] Ending request: urn:request:9cde5c8a-55d7-4e83-a90e-6d7368fa9b41 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:28,313 [526] [DEBUG] [app] Ending request: urn:request:9cde5c8a-55d7-4e83-a90e-6d7368fa9b41 (/_internal_ping) 2020-03-31 09:44:28,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:28,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:44:28,313 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:28,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:44:28,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:28,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:28,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:44:28,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:44:28,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:28,320 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:28,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:28,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:28,328 [534] [DEBUG] [app] Starting request: urn:request:505fdfdf-b5b0-41d9-b2d0-c34892b1e93f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:28,328 [534] [DEBUG] [app] Starting request: urn:request:505fdfdf-b5b0-41d9-b2d0-c34892b1e93f (/c1/_internal_ping) 2020-03-31 09:44:28,329 [534] [DEBUG] [app] Ending request: urn:request:505fdfdf-b5b0-41d9-b2d0-c34892b1e93f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:28,329 [534] [DEBUG] [app] Ending request: urn:request:505fdfdf-b5b0-41d9-b2d0-c34892b1e93f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:44:28,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:44:28,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:44:28,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:28,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:28,331 [527] [DEBUG] [app] Ending request: urn:request:fd8fa6ca-1fd2-4582-8141-0af4b61c5464 (/health/instance) 2020-03-31 09:44:28,332 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:28,331 [527] [DEBUG] [app] Ending request: urn:request:fd8fa6ca-1fd2-4582-8141-0af4b61c5464 (/health/instance) 2020-03-31 09:44:28,332 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:28,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:44:28,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:44:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:44:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:00.337699+00:00 (in 29.998915 seconds) 2020-03-31 09:44:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:00 UTC)" (scheduled at 2020-03-31 09:44:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:44:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:00.337699+00:00 (in 29.998915 seconds) 2020-03-31 09:44:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:00 UTC)" (scheduled at 2020-03-31 09:44:30.337699+00:00) 2020-03-31 09:44:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 44, 30, 339623), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:44:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 44, 30, 339623), False, 50, 1, 0]) 2020-03-31 09:44:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:44:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:44:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:44:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:00 UTC)" executed successfully 2020-03-31 09:44:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:44:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:41.440707+00:00 (in 9.998801 seconds) 2020-03-31 09:44:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:41 UTC)" (scheduled at 2020-03-31 09:44:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:44:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:41.440707+00:00 (in 9.998801 seconds) 2020-03-31 09:44:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:41 UTC)" (scheduled at 2020-03-31 09:44:31.440707+00:00) 2020-03-31 09:44:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:44:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:44:31,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:44:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:44:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:44:31,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:31,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:41 UTC)" executed successfully 2020-03-31 09:44:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:44:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:03.330112+00:00 (in 29.998811 seconds) 2020-03-31 09:44:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:03 UTC)" (scheduled at 2020-03-31 09:44:33.330112+00:00) 2020-03-31 09:44:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:44:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:03.330112+00:00 (in 29.998811 seconds) 2020-03-31 09:44:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:03 UTC)" (scheduled at 2020-03-31 09:44:33.330112+00:00) 2020-03-31 09:44:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 33, 332263), True, datetime.datetime(2020, 3, 31, 9, 44, 33, 332263), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:44:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 33, 332263), True, datetime.datetime(2020, 3, 31, 9, 44, 33, 332263), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:44:33,364 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:44:33,364 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:33,364 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:33,365 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:44:33,364 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:33,365 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:03 UTC)" executed successfully 2020-03-31 09:44:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:44:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:43.539999+00:00 (in 10.172121 seconds) 2020-03-31 09:44:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:33 UTC)" (scheduled at 2020-03-31 09:44:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:44:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:43.539999+00:00 (in 10.172121 seconds) 2020-03-31 09:44:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:33 UTC)" (scheduled at 2020-03-31 09:44:33.366791+00:00) 2020-03-31 09:44:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:44:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 33, 368899), True, datetime.datetime(2020, 3, 31, 9, 44, 33, 368899), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:44:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 33, 368899), True, datetime.datetime(2020, 3, 31, 9, 44, 33, 368899), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:44:33,386 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:44:33,386 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:44:33,386 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:33 UTC)" executed successfully 2020-03-31 09:44:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:44:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:44.323352+00:00 (in 9.999363 seconds) 2020-03-31 09:44:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:44 UTC)" (scheduled at 2020-03-31 09:44:34.323352+00:00) 2020-03-31 09:44:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:44:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:44.323352+00:00 (in 9.999363 seconds) 2020-03-31 09:44:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:44 UTC)" (scheduled at 2020-03-31 09:44:34.323352+00:00) 2020-03-31 09:44:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 34, 324940), True, datetime.datetime(2020, 3, 31, 9, 44, 34, 324940), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:44:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 34, 324940), True, datetime.datetime(2020, 3, 31, 9, 44, 34, 324940), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:44:34,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:44:34,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:44:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:44 UTC)" executed successfully 2020-03-31 09:44:34,446 [527] [DEBUG] [app] Starting request: urn:request:9ad39293-7a51-4949-8b04-22524039ac16 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:34,446 [527] [DEBUG] [app] Starting request: urn:request:9ad39293-7a51-4949-8b04-22524039ac16 (/health/instance) 2020-03-31 09:44:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:34,454 [548] [DEBUG] [app] Starting request: urn:request:f1566d62-23de-418a-b21e-1d87862559fe (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:34,454 [548] [DEBUG] [app] Starting request: urn:request:f1566d62-23de-418a-b21e-1d87862559fe (/v1/_internal_ping) 2020-03-31 09:44:34,455 [548] [DEBUG] [app] Ending request: urn:request:f1566d62-23de-418a-b21e-1d87862559fe (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:34,455 [548] [DEBUG] [app] Ending request: urn:request:f1566d62-23de-418a-b21e-1d87862559fe (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:34,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:44:34,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:34,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:34,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:34,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:34,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:34,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:34,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:34,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:34,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:34,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:34,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:34,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:34,489 [527] [DEBUG] [app] Starting request: urn:request:72ddd054-9d14-4129-b2af-0582f15995e6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:34,489 [527] [DEBUG] [app] Starting request: urn:request:72ddd054-9d14-4129-b2af-0582f15995e6 (/_internal_ping) 2020-03-31 09:44:34,490 [527] [DEBUG] [app] Ending request: urn:request:72ddd054-9d14-4129-b2af-0582f15995e6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:34,490 [527] [DEBUG] [app] Ending request: urn:request:72ddd054-9d14-4129-b2af-0582f15995e6 (/_internal_ping) 2020-03-31 09:44:34,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:34,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:34,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:34,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:34,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:34,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:34,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:34,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:34,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:34,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:34,503 [539] [DEBUG] [app] Starting request: urn:request:72a8e104-c911-4558-97c0-016831be8f13 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:34,503 [539] [DEBUG] [app] Starting request: urn:request:72a8e104-c911-4558-97c0-016831be8f13 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:34,504 [539] [DEBUG] [app] Ending request: urn:request:72a8e104-c911-4558-97c0-016831be8f13 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:34,504 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:44:34,504 [539] [DEBUG] [app] Ending request: urn:request:72a8e104-c911-4558-97c0-016831be8f13 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:34,506 [527] [DEBUG] [app] Ending request: urn:request:9ad39293-7a51-4949-8b04-22524039ac16 (/health/instance) 2020-03-31 09:44:34,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:44:34,504 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:34,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:44:34,506 [527] [DEBUG] [app] Ending request: urn:request:9ad39293-7a51-4949-8b04-22524039ac16 (/health/instance) 2020-03-31 09:44:34,506 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:34,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:44:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:38.010934+00:00 (in 1.085636 seconds) 2020-03-31 09:44:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:46 UTC)" (scheduled at 2020-03-31 09:44:36.924497+00:00) 2020-03-31 09:44:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 36, 925884), True, datetime.datetime(2020, 3, 31, 9, 44, 36, 925884), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:44:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:38.010934+00:00 (in 1.085636 seconds) 2020-03-31 09:44:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:46 UTC)" (scheduled at 2020-03-31 09:44:36.924497+00:00) 2020-03-31 09:44:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 36, 925884), True, datetime.datetime(2020, 3, 31, 9, 44, 36, 925884), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:44:36,941 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:44:36,941 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:36,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:36,941 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:44:36,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:36,941 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:46 UTC)" executed successfully 2020-03-31 09:44:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:44:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:46.924497+00:00 (in 8.912919 seconds) 2020-03-31 09:44:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:38 UTC)" (scheduled at 2020-03-31 09:44:38.010934+00:00) 2020-03-31 09:44:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:44:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:46.924497+00:00 (in 8.912919 seconds) 2020-03-31 09:44:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:38 UTC)" (scheduled at 2020-03-31 09:44:38.010934+00:00) 2020-03-31 09:44:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:44:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:38 UTC)" executed successfully 2020-03-31 09:44:38,273 [518] [DEBUG] [app] Starting request: urn:request:0f9f2249-a590-4fe6-9b65-64fe79f95a91 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:38,273 [518] [DEBUG] [app] Starting request: urn:request:0f9f2249-a590-4fe6-9b65-64fe79f95a91 (/health/instance) 2020-03-31 09:44:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:38,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:38,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:38,280 [553] [DEBUG] [app] Starting request: urn:request:3cf3cf11-c6c7-4bf6-9aae-92a563075cd8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:38,280 [553] [DEBUG] [app] Starting request: urn:request:3cf3cf11-c6c7-4bf6-9aae-92a563075cd8 (/v1/_internal_ping) 2020-03-31 09:44:38,281 [553] [DEBUG] [app] Ending request: urn:request:3cf3cf11-c6c7-4bf6-9aae-92a563075cd8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:38,281 [553] [DEBUG] [app] Ending request: urn:request:3cf3cf11-c6c7-4bf6-9aae-92a563075cd8 (/v1/_internal_ping) 2020-03-31 09:44:38,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:44:38,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:44:38,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:38,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:38,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:38,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:38,285 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:38,285 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:38,286 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:38,286 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:38,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:38,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:38,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:38,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:38,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:38,315 [526] [DEBUG] [app] Starting request: urn:request:12baa4ee-cd4e-4710-b2c9-b6a9342e0b97 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:38,315 [526] [DEBUG] [app] Starting request: urn:request:12baa4ee-cd4e-4710-b2c9-b6a9342e0b97 (/_internal_ping) 2020-03-31 09:44:38,316 [526] [DEBUG] [app] Ending request: urn:request:12baa4ee-cd4e-4710-b2c9-b6a9342e0b97 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:38,316 [526] [DEBUG] [app] Ending request: urn:request:12baa4ee-cd4e-4710-b2c9-b6a9342e0b97 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:44:38,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:38,316 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:38,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:38,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:38,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:38,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:38,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:38,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:38,328 [534] [DEBUG] [app] Starting request: urn:request:0468ecfb-fd5f-4c7a-9a30-5adfbb3a190b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:38,328 [534] [DEBUG] [app] Starting request: urn:request:0468ecfb-fd5f-4c7a-9a30-5adfbb3a190b (/c1/_internal_ping) 2020-03-31 09:44:38,329 [534] [DEBUG] [app] Ending request: urn:request:0468ecfb-fd5f-4c7a-9a30-5adfbb3a190b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:38,329 [534] [DEBUG] [app] Ending request: urn:request:0468ecfb-fd5f-4c7a-9a30-5adfbb3a190b (/c1/_internal_ping) 2020-03-31 09:44:38,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:38,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:38,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:38,331 [518] [DEBUG] [app] Ending request: urn:request:0f9f2249-a590-4fe6-9b65-64fe79f95a91 (/health/instance) 2020-03-31 09:44:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:44:38,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:38,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:44:38,331 [518] [DEBUG] [app] Ending request: urn:request:0f9f2249-a590-4fe6-9b65-64fe79f95a91 (/health/instance) 2020-03-31 09:44:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:38,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:44:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:44:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:51.440707+00:00 (in 9.998660 seconds) 2020-03-31 09:44:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:51 UTC)" (scheduled at 2020-03-31 09:44:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:44:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:51.440707+00:00 (in 9.998660 seconds) 2020-03-31 09:44:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:51 UTC)" (scheduled at 2020-03-31 09:44:41.440707+00:00) 2020-03-31 09:44:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:44:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:44:41,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:44:41,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:44:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:44:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:51 UTC)" executed successfully 2020-03-31 09:44:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:44:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:14.539408+00:00 (in 30.998389 seconds) 2020-03-31 09:44:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:43 UTC)" (scheduled at 2020-03-31 09:44:43.539999+00:00) 2020-03-31 09:44:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:44:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:14.539408+00:00 (in 30.998389 seconds) 2020-03-31 09:44:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:43 UTC)" (scheduled at 2020-03-31 09:44:43.539999+00:00) 2020-03-31 09:44:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:44:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:43 UTC)" executed successfully 2020-03-31 09:44:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:44:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:49.498966+00:00 (in 5.174380 seconds) 2020-03-31 09:44:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:54 UTC)" (scheduled at 2020-03-31 09:44:44.323352+00:00) 2020-03-31 09:44:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:44:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:49.498966+00:00 (in 5.174380 seconds) 2020-03-31 09:44:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:54 UTC)" (scheduled at 2020-03-31 09:44:44.323352+00:00) 2020-03-31 09:44:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 44, 325601), True, datetime.datetime(2020, 3, 31, 9, 44, 44, 325601), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:44:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 44, 325601), True, datetime.datetime(2020, 3, 31, 9, 44, 44, 325601), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:44:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:44:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:44:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:54 UTC)" executed successfully 2020-03-31 09:44:44,446 [527] [DEBUG] [app] Starting request: urn:request:3506d2a4-299a-4d6c-9d3e-29a0bc653d57 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:44,446 [527] [DEBUG] [app] Starting request: urn:request:3506d2a4-299a-4d6c-9d3e-29a0bc653d57 (/health/instance) 2020-03-31 09:44:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:44,454 [548] [DEBUG] [app] Starting request: urn:request:6e2fe12c-964f-412b-82c3-27eccfe9bb16 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:44,454 [548] [DEBUG] [app] Starting request: urn:request:6e2fe12c-964f-412b-82c3-27eccfe9bb16 (/v1/_internal_ping) 2020-03-31 09:44:44,455 [548] [DEBUG] [app] Ending request: urn:request:6e2fe12c-964f-412b-82c3-27eccfe9bb16 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:44,455 [548] [DEBUG] [app] Ending request: urn:request:6e2fe12c-964f-412b-82c3-27eccfe9bb16 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:44,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:44:44,456 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:44,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:44,459 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:44,460 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:44,460 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:44,461 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:44,461 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:44,477 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:44,477 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:44,492 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:44,492 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:44,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:44,498 [527] [DEBUG] [app] Starting request: urn:request:1b4b580f-6d59-472d-a19d-4ef0311ecee2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:44,498 [527] [DEBUG] [app] Starting request: urn:request:1b4b580f-6d59-472d-a19d-4ef0311ecee2 (/_internal_ping) 2020-03-31 09:44:44,499 [527] [DEBUG] [app] Ending request: urn:request:1b4b580f-6d59-472d-a19d-4ef0311ecee2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:44,499 [527] [DEBUG] [app] Ending request: urn:request:1b4b580f-6d59-472d-a19d-4ef0311ecee2 (/_internal_ping) 2020-03-31 09:44:44,499 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:44,499 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:44,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:44,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:44,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:44,503 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:44:44,503 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:44:44,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:44,506 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:44,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:44,507 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:44,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:44,510 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:44,511 [534] [DEBUG] [app] Starting request: urn:request:60f1f5d9-aa70-4c71-a527-dfba554c0aea (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:44,511 [534] [DEBUG] [app] Starting request: urn:request:60f1f5d9-aa70-4c71-a527-dfba554c0aea (/c1/_internal_ping) 2020-03-31 09:44:44,512 [534] [DEBUG] [app] Ending request: urn:request:60f1f5d9-aa70-4c71-a527-dfba554c0aea (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:44,512 [534] [DEBUG] [app] Ending request: urn:request:60f1f5d9-aa70-4c71-a527-dfba554c0aea (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:44,513 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:44,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:44:44,513 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:44,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:44,514 [527] [DEBUG] [app] Ending request: urn:request:3506d2a4-299a-4d6c-9d3e-29a0bc653d57 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:44,514 [527] [DEBUG] [app] Ending request: urn:request:3506d2a4-299a-4d6c-9d3e-29a0bc653d57 (/health/instance) 2020-03-31 09:44:44,515 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:44,515 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:44,515 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:44:44,515 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:44:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:44:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:15.064514+00:00 (in 29.999162 seconds) 2020-03-31 09:44:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:15 UTC)" (scheduled at 2020-03-31 09:44:45.064514+00:00) gcworker stdout | 2020-03-31 09:44:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:15.064514+00:00 (in 29.999162 seconds) 2020-03-31 09:44:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:15 UTC)" (scheduled at 2020-03-31 09:44:45.064514+00:00) 2020-03-31 09:44:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438285, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:44:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438285, 1209600, 500, 1, 0]) 2020-03-31 09:44:45,083 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:44:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:44:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:44:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:15 UTC)" executed successfully 2020-03-31 09:44:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:44:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:56.924497+00:00 (in 9.998861 seconds) 2020-03-31 09:44:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:56 UTC)" (scheduled at 2020-03-31 09:44:46.924497+00:00) 2020-03-31 09:44:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:44:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:56.924497+00:00 (in 9.998861 seconds) 2020-03-31 09:44:46,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:56 UTC)" (scheduled at 2020-03-31 09:44:46.924497+00:00) 2020-03-31 09:44:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 46, 926533), True, datetime.datetime(2020, 3, 31, 9, 44, 46, 926533), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:44:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 46, 926533), True, datetime.datetime(2020, 3, 31, 9, 44, 46, 926533), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:44:46,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:44:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:44:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:46,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:44:56 UTC)" executed successfully 2020-03-31 09:44:48,273 [527] [DEBUG] [app] Starting request: urn:request:b0ef73e9-d98c-43e7-af7a-7900fb7066d9 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:48,273 [527] [DEBUG] [app] Starting request: urn:request:b0ef73e9-d98c-43e7-af7a-7900fb7066d9 (/health/instance) 2020-03-31 09:44:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:48,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:48,278 [542] [DEBUG] [app] Starting request: urn:request:52bfbb8d-261b-4c85-9024-38be2ab360ac (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:48,278 [542] [DEBUG] [app] Starting request: urn:request:52bfbb8d-261b-4c85-9024-38be2ab360ac (/v1/_internal_ping) 2020-03-31 09:44:48,279 [542] [DEBUG] [app] Ending request: urn:request:52bfbb8d-261b-4c85-9024-38be2ab360ac (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:48,279 [542] [DEBUG] [app] Ending request: urn:request:52bfbb8d-261b-4c85-9024-38be2ab360ac (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:48,280 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:48,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:44:48,280 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:48,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:48,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:48,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:48,282 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:48,282 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:48,284 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:48,284 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:48,285 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:48,285 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:48,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:48,290 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:48,302 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:48,302 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:48,306 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:48,306 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:48,308 [518] [DEBUG] [app] Starting request: urn:request:ebe4c005-a46b-481a-9e7c-3cc2108218e4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:48,308 [518] [DEBUG] [app] Starting request: urn:request:ebe4c005-a46b-481a-9e7c-3cc2108218e4 (/_internal_ping) 2020-03-31 09:44:48,309 [518] [DEBUG] [app] Ending request: urn:request:ebe4c005-a46b-481a-9e7c-3cc2108218e4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:48,309 [518] [DEBUG] [app] Ending request: urn:request:ebe4c005-a46b-481a-9e7c-3cc2108218e4 (/_internal_ping) 2020-03-31 09:44:48,309 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:48,309 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:48,309 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:48,309 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:44:48,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:48,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:48,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:48,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:48,323 [539] [DEBUG] [app] Starting request: urn:request:d9b86b48-fda2-44d6-82e1-cd549efcc314 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:48,323 [539] [DEBUG] [app] Starting request: urn:request:d9b86b48-fda2-44d6-82e1-cd549efcc314 (/c1/_internal_ping) 2020-03-31 09:44:48,324 [539] [DEBUG] [app] Ending request: urn:request:d9b86b48-fda2-44d6-82e1-cd549efcc314 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:48,324 [539] [DEBUG] [app] Ending request: urn:request:d9b86b48-fda2-44d6-82e1-cd549efcc314 (/c1/_internal_ping) 2020-03-31 09:44:48,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:44:48,325 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:44:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:48,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:48,328 [527] [DEBUG] [app] Ending request: urn:request:b0ef73e9-d98c-43e7-af7a-7900fb7066d9 (/health/instance) 2020-03-31 09:44:48,328 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:48,328 [527] [DEBUG] [app] Ending request: urn:request:b0ef73e9-d98c-43e7-af7a-7900fb7066d9 (/health/instance) 2020-03-31 09:44:48,328 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:48,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:44:48,329 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:44:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:44:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:49 UTC)" (scheduled at 2020-03-31 09:44:49.498966+00:00) 2020-03-31 09:44:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:49 UTC)" executed successfully 2020-03-31 09:44:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:54.323352+00:00 (in 4.823320 seconds) notificationworker stdout | 2020-03-31 09:44:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:44:49 UTC)" (scheduled at 2020-03-31 09:44:49.498966+00:00) 2020-03-31 09:44:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:49 UTC)" executed successfully 2020-03-31 09:44:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:54.323352+00:00 (in 4.823320 seconds) 2020-03-31 09:44:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:44:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:53.556282+00:00 (in 2.444659 seconds) 2020-03-31 09:44:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:51 UTC)" (scheduled at 2020-03-31 09:44:51.110610+00:00) 2020-03-31 09:44:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:44:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:53.556282+00:00 (in 2.444659 seconds) 2020-03-31 09:44:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:51 UTC)" (scheduled at 2020-03-31 09:44:51.110610+00:00) 2020-03-31 09:44:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 51, 112926), True, datetime.datetime(2020, 3, 31, 9, 44, 51, 112926), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:44:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 51, 112926), True, datetime.datetime(2020, 3, 31, 9, 44, 51, 112926), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:44:51,131 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:44:51,131 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:44:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:51 UTC)" executed successfully 2020-03-31 09:44:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:44:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:01.440707+00:00 (in 9.999094 seconds) 2020-03-31 09:44:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:01 UTC)" (scheduled at 2020-03-31 09:44:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:44:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:01.440707+00:00 (in 9.999094 seconds) 2020-03-31 09:44:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:01 UTC)" (scheduled at 2020-03-31 09:44:51.440707+00:00) 2020-03-31 09:44:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:44:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:44:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:44:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:44:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:44:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:01 UTC)" executed successfully 2020-03-31 09:44:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:44:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:22 UTC)" (scheduled at 2020-03-31 09:44:52.113221+00:00) securityworker stdout | 2020-03-31 09:44:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:22 UTC)" (scheduled at 2020-03-31 09:44:52.113221+00:00) 2020-03-31 09:44:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:22.113221+00:00 (in 29.998701 seconds) 2020-03-31 09:44:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:44:52,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:22.113221+00:00 (in 29.998701 seconds) 2020-03-31 09:44:52,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:44:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:44:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:44:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:44:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:44:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:44:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:52,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:22 UTC)" executed successfully 2020-03-31 09:44:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:44:53,557 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:53,558 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:59.560326+00:00 (in 6.002676 seconds) 2020-03-31 09:44:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:53 UTC)" (scheduled at 2020-03-31 09:44:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:44:53,558 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:44:59.560326+00:00 (in 6.002676 seconds) 2020-03-31 09:44:53,558 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:53 UTC)" (scheduled at 2020-03-31 09:44:53.556282+00:00) 2020-03-31 09:44:53,560 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:44:53,560 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), u'namespacegc/%']) 2020-03-31 09:44:53,575 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), True, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:44:53,575 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), True, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), 0, u'namespacegc/%']) 2020-03-31 09:44:53,580 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), True, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:44:53,580 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), True, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 44, 53, 558618), u'namespacegc/%']) 2020-03-31 09:44:53,585 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:44:53,585 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:53,586 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:44:53,586 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:49:53 UTC)" executed successfully 2020-03-31 09:44:53,588 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:44:53,588 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:53,589 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:44:53,589 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:53,590 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:44:53,590 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:44:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:44:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:04.323352+00:00 (in 9.998272 seconds) 2020-03-31 09:44:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:04 UTC)" (scheduled at 2020-03-31 09:44:54.323352+00:00) 2020-03-31 09:44:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:44:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:04.323352+00:00 (in 9.998272 seconds) 2020-03-31 09:44:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:04 UTC)" (scheduled at 2020-03-31 09:44:54.323352+00:00) 2020-03-31 09:44:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 54, 326031), True, datetime.datetime(2020, 3, 31, 9, 44, 54, 326031), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:44:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 54, 326031), True, datetime.datetime(2020, 3, 31, 9, 44, 54, 326031), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:44:54,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:44:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:44:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:04 UTC)" executed successfully 2020-03-31 09:44:54,446 [527] [DEBUG] [app] Starting request: urn:request:32cce6d4-27e6-4e15-88cb-c5903ac6fe31 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:54,446 [527] [DEBUG] [app] Starting request: urn:request:32cce6d4-27e6-4e15-88cb-c5903ac6fe31 (/health/instance) 2020-03-31 09:44:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:54,453 [538] [DEBUG] [app] Starting request: urn:request:997edff1-3b7d-42f5-9281-98522123b92b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:54,453 [538] [DEBUG] [app] Starting request: urn:request:997edff1-3b7d-42f5-9281-98522123b92b (/v1/_internal_ping) 2020-03-31 09:44:54,454 [538] [DEBUG] [app] Ending request: urn:request:997edff1-3b7d-42f5-9281-98522123b92b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:54,454 [538] [DEBUG] [app] Ending request: urn:request:997edff1-3b7d-42f5-9281-98522123b92b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:54,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:44:54,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:54,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:44:54,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:44:54,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:44:54,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:54,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:54,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:54,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:54,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:54,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:54,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:54,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:54,485 [527] [DEBUG] [app] Starting request: urn:request:567e2c84-5c49-49f2-ab20-345517da4d80 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:54,485 [527] [DEBUG] [app] Starting request: urn:request:567e2c84-5c49-49f2-ab20-345517da4d80 (/_internal_ping) 2020-03-31 09:44:54,486 [527] [DEBUG] [app] Ending request: urn:request:567e2c84-5c49-49f2-ab20-345517da4d80 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:44:54,486 [527] [DEBUG] [app] Ending request: urn:request:567e2c84-5c49-49f2-ab20-345517da4d80 (/_internal_ping) 2020-03-31 09:44:54,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:54,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:54,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:54,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:54,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:54,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:54,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:44:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:54,499 [535] [DEBUG] [app] Starting request: urn:request:309e2f1b-cd87-4f06-8402-564de69faae4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:54,499 [535] [DEBUG] [app] Starting request: urn:request:309e2f1b-cd87-4f06-8402-564de69faae4 (/c1/_internal_ping) 2020-03-31 09:44:54,499 [535] [DEBUG] [app] Ending request: urn:request:309e2f1b-cd87-4f06-8402-564de69faae4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:54,499 [535] [DEBUG] [app] Ending request: urn:request:309e2f1b-cd87-4f06-8402-564de69faae4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:44:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:54,500 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:54,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:44:54,500 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:54,502 [527] [DEBUG] [app] Ending request: urn:request:32cce6d4-27e6-4e15-88cb-c5903ac6fe31 (/health/instance) 2020-03-31 09:44:54,502 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:54,502 [527] [DEBUG] [app] Ending request: urn:request:32cce6d4-27e6-4e15-88cb-c5903ac6fe31 (/health/instance) 2020-03-31 09:44:54,502 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:54,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:44:54,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:44:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:06.924497+00:00 (in 9.999256 seconds) 2020-03-31 09:44:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:06 UTC)" (scheduled at 2020-03-31 09:44:56.924497+00:00) 2020-03-31 09:44:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:44:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:44:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:06.924497+00:00 (in 9.999256 seconds) 2020-03-31 09:44:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:06 UTC)" (scheduled at 2020-03-31 09:44:56.924497+00:00) 2020-03-31 09:44:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:44:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 56, 925916), True, datetime.datetime(2020, 3, 31, 9, 44, 56, 925916), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:44:56,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 44, 56, 925916), True, datetime.datetime(2020, 3, 31, 9, 44, 56, 925916), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:44:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:44:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:44:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:44:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:56,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:06 UTC)" executed successfully 2020-03-31 09:44:58,273 [526] [DEBUG] [app] Starting request: urn:request:dd1cfc8f-177c-42d5-b3f1-557d19612131 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:58,273 [526] [DEBUG] [app] Starting request: urn:request:dd1cfc8f-177c-42d5-b3f1-557d19612131 (/health/instance) 2020-03-31 09:44:58,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:58,275 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:58,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:58,279 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:58,281 [538] [DEBUG] [app] Starting request: urn:request:c377368c-45ca-4575-9cd6-c4cbf5286d25 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:58,281 [538] [DEBUG] [app] Starting request: urn:request:c377368c-45ca-4575-9cd6-c4cbf5286d25 (/v1/_internal_ping) 2020-03-31 09:44:58,282 [538] [DEBUG] [app] Ending request: urn:request:c377368c-45ca-4575-9cd6-c4cbf5286d25 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:44:58,282 [538] [DEBUG] [app] Ending request: urn:request:c377368c-45ca-4575-9cd6-c4cbf5286d25 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:58,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:58,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:44:58,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:58,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:44:58,283 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:58,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:44:58,284 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:44:58,285 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:44:58,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:44:58,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:44:58,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:44:58,287 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:44:58,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:44:58,293 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:44:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:44:58,308 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:44:58,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:58,311 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:58,313 [518] [DEBUG] [app] Starting request: urn:request:a0f67d09-f4c9-4c2f-8613-ae60e1db95f0 (/_internal_ping) 2020-03-31 09:44:58,313 [518] [DEBUG] [app] Ending request: urn:request:a0f67d09-f4c9-4c2f-8613-ae60e1db95f0 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:58,313 [518] [DEBUG] [app] Starting request: urn:request:a0f67d09-f4c9-4c2f-8613-ae60e1db95f0 (/_internal_ping) 2020-03-31 09:44:58,313 [518] [DEBUG] [app] Ending request: urn:request:a0f67d09-f4c9-4c2f-8613-ae60e1db95f0 (/_internal_ping) 2020-03-31 09:44:58,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:58,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:44:58,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:44:58,314 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:58,317 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:58,317 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:58,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:44:58,318 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:44:58,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:58,321 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:44:58,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:44:58,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:44:58,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:44:58,325 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:44:58,326 [535] [DEBUG] [app] Starting request: urn:request:90b787a0-0e92-46d7-bf76-5da1bc37c4da (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:58,326 [535] [DEBUG] [app] Starting request: urn:request:90b787a0-0e92-46d7-bf76-5da1bc37c4da (/c1/_internal_ping) 2020-03-31 09:44:58,327 [535] [DEBUG] [app] Ending request: urn:request:90b787a0-0e92-46d7-bf76-5da1bc37c4da (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:44:58,327 [535] [DEBUG] [app] Ending request: urn:request:90b787a0-0e92-46d7-bf76-5da1bc37c4da (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:58,327 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:44:58,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:44:58,327 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:44:58,328 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:44:58,329 [526] [DEBUG] [app] Ending request: urn:request:dd1cfc8f-177c-42d5-b3f1-557d19612131 (/health/instance) gunicorn-web stdout | 2020-03-31 09:44:58,329 [526] [DEBUG] [app] Ending request: urn:request:dd1cfc8f-177c-42d5-b3f1-557d19612131 (/health/instance) 2020-03-31 09:44:58,329 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:44:58,329 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:44:58,330 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:44:58,330 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:44:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:44:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:44:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:44:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:44:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:51.110610+00:00 (in 51.549362 seconds) 2020-03-31 09:44:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:59 UTC)" (scheduled at 2020-03-31 09:44:59.560326+00:00) 2020-03-31 09:44:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:44:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:51.110610+00:00 (in 51.549362 seconds) 2020-03-31 09:44:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:59 UTC)" (scheduled at 2020-03-31 09:44:59.560326+00:00) 2020-03-31 09:44:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:44:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:45:59 UTC)" executed successfully 2020-03-31 09:45:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:45:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:30.337699+00:00 (in 29.998918 seconds) 2020-03-31 09:45:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:30 UTC)" (scheduled at 2020-03-31 09:45:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:45:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:30.337699+00:00 (in 29.998918 seconds) 2020-03-31 09:45:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:30 UTC)" (scheduled at 2020-03-31 09:45:00.337699+00:00) 2020-03-31 09:45:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 45, 0, 339641), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:45:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 45, 0, 339641), False, 50, 1, 0]) 2020-03-31 09:45:00,543 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:45:00,543 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:45:00,543 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:00,545 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:45:00,543 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:00,545 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:30 UTC)" executed successfully 2020-03-31 09:45:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:45:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:01 UTC)" (scheduled at 2020-03-31 09:45:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:45:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:01 UTC)" (scheduled at 2020-03-31 09:45:01.440707+00:00) 2020-03-31 09:45:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:45:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:45:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:11.440707+00:00 (in 9.998922 seconds) repositoryactioncounter stdout | 2020-03-31 09:45:01,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:11.440707+00:00 (in 9.998922 seconds) 2020-03-31 09:45:01,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:45:01,460 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:45:01,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:45:01,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:45:01,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:11 UTC)" executed successfully 2020-03-31 09:45:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:24.410927+00:00 (in 21.079807 seconds) 2020-03-31 09:45:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:33 UTC)" (scheduled at 2020-03-31 09:45:03.330112+00:00) 2020-03-31 09:45:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:45:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:45:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:24.410927+00:00 (in 21.079807 seconds) 2020-03-31 09:45:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:33 UTC)" (scheduled at 2020-03-31 09:45:03.330112+00:00) 2020-03-31 09:45:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 3, 331945), True, datetime.datetime(2020, 3, 31, 9, 45, 3, 331945), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:45:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 3, 331945), True, datetime.datetime(2020, 3, 31, 9, 45, 3, 331945), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:45:03,352 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:45:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:45:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:33 UTC)" executed successfully 2020-03-31 09:45:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:45:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:14.323352+00:00 (in 9.999250 seconds) 2020-03-31 09:45:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:14 UTC)" (scheduled at 2020-03-31 09:45:04.323352+00:00) 2020-03-31 09:45:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:45:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:14.323352+00:00 (in 9.999250 seconds) 2020-03-31 09:45:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:14 UTC)" (scheduled at 2020-03-31 09:45:04.323352+00:00) 2020-03-31 09:45:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 4, 324770), True, datetime.datetime(2020, 3, 31, 9, 45, 4, 324770), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:45:04,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 4, 324770), True, datetime.datetime(2020, 3, 31, 9, 45, 4, 324770), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:45:04,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:45:04,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:04,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:45:04,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:04,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:14 UTC)" executed successfully 2020-03-31 09:45:04,457 [518] [DEBUG] [app] Starting request: urn:request:44c41577-0c2b-4baf-aa6f-5669db674ec8 (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:04,457 [518] [DEBUG] [app] Starting request: urn:request:44c41577-0c2b-4baf-aa6f-5669db674ec8 (/health/instance) 2020-03-31 09:45:04,459 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:04,459 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:04,463 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:04,463 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:04,465 [542] [DEBUG] [app] Starting request: urn:request:a7f1bbad-119f-4fde-bcf8-9421b5de5e67 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:04,465 [542] [DEBUG] [app] Starting request: urn:request:a7f1bbad-119f-4fde-bcf8-9421b5de5e67 (/v1/_internal_ping) 2020-03-31 09:45:04,466 [542] [DEBUG] [app] Ending request: urn:request:a7f1bbad-119f-4fde-bcf8-9421b5de5e67 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:04,466 [542] [DEBUG] [app] Ending request: urn:request:a7f1bbad-119f-4fde-bcf8-9421b5de5e67 (/v1/_internal_ping) 2020-03-31 09:45:04,467 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:04,467 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:45:04,467 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:04,468 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:04,467 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:04,469 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:45:04,468 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:04,470 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:04,469 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:04,471 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:04,470 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:04,471 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:04,479 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:04,479 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:04,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:04,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:04,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:04,499 [518] [DEBUG] [app] Starting request: urn:request:9bf5f1c7-13c8-4b50-b1c4-37e1bbc726e5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:04,499 [518] [DEBUG] [app] Starting request: urn:request:9bf5f1c7-13c8-4b50-b1c4-37e1bbc726e5 (/_internal_ping) 2020-03-31 09:45:04,499 [518] [DEBUG] [app] Ending request: urn:request:9bf5f1c7-13c8-4b50-b1c4-37e1bbc726e5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:04,499 [518] [DEBUG] [app] Ending request: urn:request:9bf5f1c7-13c8-4b50-b1c4-37e1bbc726e5 (/_internal_ping) 2020-03-31 09:45:04,500 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:04,500 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:45:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:45:04,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:04,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:04,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:04,511 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:04,512 [537] [DEBUG] [app] Starting request: urn:request:42a75af3-aff0-4bba-97e7-7166238c53a7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:04,512 [537] [DEBUG] [app] Starting request: urn:request:42a75af3-aff0-4bba-97e7-7166238c53a7 (/c1/_internal_ping) 2020-03-31 09:45:04,513 [537] [DEBUG] [app] Ending request: urn:request:42a75af3-aff0-4bba-97e7-7166238c53a7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:04,513 [537] [DEBUG] [app] Ending request: urn:request:42a75af3-aff0-4bba-97e7-7166238c53a7 (/c1/_internal_ping) 2020-03-31 09:45:04,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:04,514 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:04,514 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:04,515 [518] [DEBUG] [app] Ending request: urn:request:44c41577-0c2b-4baf-aa6f-5669db674ec8 (/health/instance) 2020-03-31 09:45:04,516 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:45:04,514 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:04,515 [518] [DEBUG] [app] Ending request: urn:request:44c41577-0c2b-4baf-aa6f-5669db674ec8 (/health/instance) 2020-03-31 09:45:04,516 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:04,517 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:45:04,517 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:45:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:45:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:16.924497+00:00 (in 9.999324 seconds) 2020-03-31 09:45:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:16 UTC)" (scheduled at 2020-03-31 09:45:06.924497+00:00) 2020-03-31 09:45:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:45:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:16.924497+00:00 (in 9.999324 seconds) 2020-03-31 09:45:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:16 UTC)" (scheduled at 2020-03-31 09:45:06.924497+00:00) 2020-03-31 09:45:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 6, 926043), True, datetime.datetime(2020, 3, 31, 9, 45, 6, 926043), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:45:06,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 6, 926043), True, datetime.datetime(2020, 3, 31, 9, 45, 6, 926043), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:45:06,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:45:06,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:45:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:16 UTC)" executed successfully 2020-03-31 09:45:08,273 [518] [DEBUG] [app] Starting request: urn:request:c06cfb48-6e74-4f9a-bc95-c11578cc83bf (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:08,273 [518] [DEBUG] [app] Starting request: urn:request:c06cfb48-6e74-4f9a-bc95-c11578cc83bf (/health/instance) 2020-03-31 09:45:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:08,278 [551] [DEBUG] [app] Starting request: urn:request:72229aa1-6b2d-4e43-88ea-cdd968aacb07 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:08,278 [551] [DEBUG] [app] Starting request: urn:request:72229aa1-6b2d-4e43-88ea-cdd968aacb07 (/v1/_internal_ping) 2020-03-31 09:45:08,279 [551] [DEBUG] [app] Ending request: urn:request:72229aa1-6b2d-4e43-88ea-cdd968aacb07 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:08,279 [551] [DEBUG] [app] Ending request: urn:request:72229aa1-6b2d-4e43-88ea-cdd968aacb07 (/v1/_internal_ping) 2020-03-31 09:45:08,280 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:45:08,280 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:08,282 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:45:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:45:08,282 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:08,283 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:08,283 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:08,284 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:08,284 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:08,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:08,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:08,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:08,314 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:08,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:08,319 [526] [DEBUG] [app] Starting request: urn:request:db2bf7b2-ae76-4770-8821-74d36d0b0555 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:08,319 [526] [DEBUG] [app] Starting request: urn:request:db2bf7b2-ae76-4770-8821-74d36d0b0555 (/_internal_ping) 2020-03-31 09:45:08,320 [526] [DEBUG] [app] Ending request: urn:request:db2bf7b2-ae76-4770-8821-74d36d0b0555 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:08,320 [526] [DEBUG] [app] Ending request: urn:request:db2bf7b2-ae76-4770-8821-74d36d0b0555 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:45:08,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:08,321 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:08,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:08,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:45:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:45:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:45:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:08,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:08,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:08,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:08,333 [541] [DEBUG] [app] Starting request: urn:request:6251c57a-4f4d-4084-9381-d2b389020834 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:08,333 [541] [DEBUG] [app] Starting request: urn:request:6251c57a-4f4d-4084-9381-d2b389020834 (/c1/_internal_ping) 2020-03-31 09:45:08,334 [541] [DEBUG] [app] Ending request: urn:request:6251c57a-4f4d-4084-9381-d2b389020834 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:08,334 [541] [DEBUG] [app] Ending request: urn:request:6251c57a-4f4d-4084-9381-d2b389020834 (/c1/_internal_ping) 2020-03-31 09:45:08,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:08,334 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:08,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:45:08,334 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:08,336 [518] [DEBUG] [app] Ending request: urn:request:c06cfb48-6e74-4f9a-bc95-c11578cc83bf (/health/instance) 2020-03-31 09:45:08,336 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:45:08,336 [518] [DEBUG] [app] Ending request: urn:request:c06cfb48-6e74-4f9a-bc95-c11578cc83bf (/health/instance) 2020-03-31 09:45:08,336 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:08,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:45:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:45:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:21.440707+00:00 (in 9.995545 seconds) 2020-03-31 09:45:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:21 UTC)" (scheduled at 2020-03-31 09:45:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:45:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:21.440707+00:00 (in 9.995545 seconds) 2020-03-31 09:45:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:21 UTC)" (scheduled at 2020-03-31 09:45:11.440707+00:00) 2020-03-31 09:45:11,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:45:11,448 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:45:11,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:45:11,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:45:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:45:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:21 UTC)" executed successfully 2020-03-31 09:45:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:45:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:24.323352+00:00 (in 9.999242 seconds) 2020-03-31 09:45:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:24 UTC)" (scheduled at 2020-03-31 09:45:14.323352+00:00) 2020-03-31 09:45:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:45:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:24.323352+00:00 (in 9.999242 seconds) 2020-03-31 09:45:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:24 UTC)" (scheduled at 2020-03-31 09:45:14.323352+00:00) 2020-03-31 09:45:14,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 14, 324751), True, datetime.datetime(2020, 3, 31, 9, 45, 14, 324751), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:45:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 14, 324751), True, datetime.datetime(2020, 3, 31, 9, 45, 14, 324751), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:45:14,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:45:14,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:45:14,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:14,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:24 UTC)" executed successfully 2020-03-31 09:45:14,446 [526] [DEBUG] [app] Starting request: urn:request:5ad3b607-9ad0-43f8-8ac5-f6a7049819dd (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:14,446 [526] [DEBUG] [app] Starting request: urn:request:5ad3b607-9ad0-43f8-8ac5-f6a7049819dd (/health/instance) 2020-03-31 09:45:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:14,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:14,453 [546] [DEBUG] [app] Starting request: urn:request:b7b73d56-1051-4230-8c70-7615d30b7d73 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:14,453 [546] [DEBUG] [app] Starting request: urn:request:b7b73d56-1051-4230-8c70-7615d30b7d73 (/v1/_internal_ping) 2020-03-31 09:45:14,455 [546] [DEBUG] [app] Ending request: urn:request:b7b73d56-1051-4230-8c70-7615d30b7d73 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:14,455 [546] [DEBUG] [app] Ending request: urn:request:b7b73d56-1051-4230-8c70-7615d30b7d73 (/v1/_internal_ping) 2020-03-31 09:45:14,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:45:14,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:14,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:14,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:45:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:45:14,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:14,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:14,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:14,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:14,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:14,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:14,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:14,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:14,486 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:14,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:14,492 [527] [DEBUG] [app] Starting request: urn:request:56246b28-068d-41ec-90e9-8baa59241883 (/_internal_ping) 2020-03-31 09:45:14,493 [527] [DEBUG] [app] Ending request: urn:request:56246b28-068d-41ec-90e9-8baa59241883 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:45:14,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:14,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:14,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:14,492 [527] [DEBUG] [app] Starting request: urn:request:56246b28-068d-41ec-90e9-8baa59241883 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:14,493 [527] [DEBUG] [app] Ending request: urn:request:56246b28-068d-41ec-90e9-8baa59241883 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:45:14,494 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:14,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:14,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:14,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:14,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:45:14,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:45:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:14,500 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:14,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:45:14,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:45:14,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:14,505 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:14,506 [541] [DEBUG] [app] Starting request: urn:request:835d3a67-1f0f-4458-a9c0-c0ab068ea2c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:14,506 [541] [DEBUG] [app] Starting request: urn:request:835d3a67-1f0f-4458-a9c0-c0ab068ea2c2 (/c1/_internal_ping) 2020-03-31 09:45:14,508 [541] [DEBUG] [app] Ending request: urn:request:835d3a67-1f0f-4458-a9c0-c0ab068ea2c2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:14,508 [541] [DEBUG] [app] Ending request: urn:request:835d3a67-1f0f-4458-a9c0-c0ab068ea2c2 (/c1/_internal_ping) 2020-03-31 09:45:14,508 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:14,508 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:14,508 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:45:14,508 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:14,510 [526] [DEBUG] [app] Ending request: urn:request:5ad3b607-9ad0-43f8-8ac5-f6a7049819dd (/health/instance) 2020-03-31 09:45:14,510 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:14,510 [526] [DEBUG] [app] Ending request: urn:request:5ad3b607-9ad0-43f8-8ac5-f6a7049819dd (/health/instance) 2020-03-31 09:45:14,510 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:14,510 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) gunicorn-web stdout | 2020-03-31 09:45:14,510 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.066 49 0.066) 2020-03-31 09:45:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:45:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:33.366791+00:00 (in 18.826553 seconds) 2020-03-31 09:45:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:50:14 UTC)" (scheduled at 2020-03-31 09:45:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:45:14,540 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:33.366791+00:00 (in 18.826553 seconds) 2020-03-31 09:45:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:50:14 UTC)" (scheduled at 2020-03-31 09:45:14.539408+00:00) 2020-03-31 09:45:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:45:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), u'exportactionlogs/%']) 2020-03-31 09:45:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), True, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:45:14,556 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), True, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), 0, u'exportactionlogs/%']) 2020-03-31 09:45:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), True, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:45:14,560 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), True, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 45, 14, 541052), u'exportactionlogs/%']) 2020-03-31 09:45:14,564 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:45:14,564 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:45:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:14,567 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:50:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:45:14,567 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:50:14 UTC)" executed successfully 2020-03-31 09:45:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:45:14,568 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:14,568 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:45:14,568 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:45:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:45:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:45.064514+00:00 (in 29.999280 seconds) 2020-03-31 09:45:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:45 UTC)" (scheduled at 2020-03-31 09:45:15.064514+00:00) gcworker stdout | 2020-03-31 09:45:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:45.064514+00:00 (in 29.999280 seconds) 2020-03-31 09:45:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:45 UTC)" (scheduled at 2020-03-31 09:45:15.064514+00:00) 2020-03-31 09:45:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438315, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:45:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438315, 1209600, 500, 1, 0]) 2020-03-31 09:45:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:45:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:45:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:45:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:45 UTC)" executed successfully 2020-03-31 09:45:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:45:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:26.924497+00:00 (in 9.999045 seconds) 2020-03-31 09:45:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:26 UTC)" (scheduled at 2020-03-31 09:45:16.924497+00:00) 2020-03-31 09:45:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:45:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:26.924497+00:00 (in 9.999045 seconds) 2020-03-31 09:45:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:26 UTC)" (scheduled at 2020-03-31 09:45:16.924497+00:00) 2020-03-31 09:45:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 16, 926268), True, datetime.datetime(2020, 3, 31, 9, 45, 16, 926268), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:45:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 16, 926268), True, datetime.datetime(2020, 3, 31, 9, 45, 16, 926268), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:45:16,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:45:16,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:45:16,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:26 UTC)" executed successfully 2020-03-31 09:45:18,273 [527] [DEBUG] [app] Starting request: urn:request:16b1a898-15ad-4ad0-ba5d-b5f98a0fafd7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:18,273 [527] [DEBUG] [app] Starting request: urn:request:16b1a898-15ad-4ad0-ba5d-b5f98a0fafd7 (/health/instance) 2020-03-31 09:45:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:18,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:18,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:18,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:18,280 [553] [DEBUG] [app] Starting request: urn:request:2966e062-b51f-4ba3-b14b-03d0c6664ab6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:18,280 [553] [DEBUG] [app] Starting request: urn:request:2966e062-b51f-4ba3-b14b-03d0c6664ab6 (/v1/_internal_ping) 2020-03-31 09:45:18,281 [553] [DEBUG] [app] Ending request: urn:request:2966e062-b51f-4ba3-b14b-03d0c6664ab6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:18,281 [553] [DEBUG] [app] Ending request: urn:request:2966e062-b51f-4ba3-b14b-03d0c6664ab6 (/v1/_internal_ping) 2020-03-31 09:45:18,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:45:18,282 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:45:18,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:18,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:18,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:45:18,284 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:18,285 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:18,285 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:18,286 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:18,286 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:18,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:18,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:18,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:18,310 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:18,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:18,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:18,314 [527] [DEBUG] [app] Starting request: urn:request:76ae2012-aec4-493f-83ab-c5db70183d41 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:18,314 [527] [DEBUG] [app] Starting request: urn:request:76ae2012-aec4-493f-83ab-c5db70183d41 (/_internal_ping) 2020-03-31 09:45:18,315 [527] [DEBUG] [app] Ending request: urn:request:76ae2012-aec4-493f-83ab-c5db70183d41 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:18,315 [527] [DEBUG] [app] Ending request: urn:request:76ae2012-aec4-493f-83ab-c5db70183d41 (/_internal_ping) 2020-03-31 09:45:18,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:18,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:18,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:18,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:18,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:18,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:18,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:45:18,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:18,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:18,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:45:18,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:18,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:18,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:18,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:18,327 [535] [DEBUG] [app] Starting request: urn:request:0b4e7f8e-9162-44e1-9c9d-da7602caed5d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:18,327 [535] [DEBUG] [app] Starting request: urn:request:0b4e7f8e-9162-44e1-9c9d-da7602caed5d (/c1/_internal_ping) 2020-03-31 09:45:18,328 [535] [DEBUG] [app] Ending request: urn:request:0b4e7f8e-9162-44e1-9c9d-da7602caed5d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:18,328 [535] [DEBUG] [app] Ending request: urn:request:0b4e7f8e-9162-44e1-9c9d-da7602caed5d (/c1/_internal_ping) 2020-03-31 09:45:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:18,329 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:18,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:18,331 [527] [DEBUG] [app] Ending request: urn:request:16b1a898-15ad-4ad0-ba5d-b5f98a0fafd7 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:45:18,329 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:18,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:18,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:45:18,331 [527] [DEBUG] [app] Ending request: urn:request:16b1a898-15ad-4ad0-ba5d-b5f98a0fafd7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:18,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:18,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:45:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:45:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:31.440707+00:00 (in 9.998720 seconds) 2020-03-31 09:45:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:31 UTC)" (scheduled at 2020-03-31 09:45:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:45:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:31.440707+00:00 (in 9.998720 seconds) 2020-03-31 09:45:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:31 UTC)" (scheduled at 2020-03-31 09:45:21.440707+00:00) 2020-03-31 09:45:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:45:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:45:21,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:45:21,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:45:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:45:21,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:21,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:31 UTC)" executed successfully 2020-03-31 09:45:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:45:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:52.113221+00:00 (in 29.997537 seconds) 2020-03-31 09:45:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:52 UTC)" (scheduled at 2020-03-31 09:45:22.113221+00:00) securityworker stdout | 2020-03-31 09:45:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:52.113221+00:00 (in 29.997537 seconds) 2020-03-31 09:45:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:52 UTC)" (scheduled at 2020-03-31 09:45:22.113221+00:00) 2020-03-31 09:45:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:45:22,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:45:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:45:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:45:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:45:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:45:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:45:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:45:52 UTC)" executed successfully 2020-03-31 09:45:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:45:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:34.323352+00:00 (in 9.999339 seconds) 2020-03-31 09:45:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:34 UTC)" (scheduled at 2020-03-31 09:45:24.323352+00:00) 2020-03-31 09:45:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:45:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:34.323352+00:00 (in 9.999339 seconds) 2020-03-31 09:45:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:34 UTC)" (scheduled at 2020-03-31 09:45:24.323352+00:00) 2020-03-31 09:45:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 24, 324782), True, datetime.datetime(2020, 3, 31, 9, 45, 24, 324782), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:45:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 24, 324782), True, datetime.datetime(2020, 3, 31, 9, 45, 24, 324782), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:45:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:45:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:45:24,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:24,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:34 UTC)" executed successfully 2020-03-31 09:45:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:45:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:33.330112+00:00 (in 8.918434 seconds) 2020-03-31 09:45:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:24 UTC)" (scheduled at 2020-03-31 09:45:24.410927+00:00) 2020-03-31 09:45:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:45:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:33.330112+00:00 (in 8.918434 seconds) 2020-03-31 09:45:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:24 UTC)" (scheduled at 2020-03-31 09:45:24.410927+00:00) 2020-03-31 09:45:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:45:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:24 UTC)" executed successfully 2020-03-31 09:45:24,445 [518] [DEBUG] [app] Starting request: urn:request:1014cdcd-ca42-46a0-8151-08c0c043131f (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:24,445 [518] [DEBUG] [app] Starting request: urn:request:1014cdcd-ca42-46a0-8151-08c0c043131f (/health/instance) 2020-03-31 09:45:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:24,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:24,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:24,453 [544] [DEBUG] [app] Starting request: urn:request:1df2122a-2ea7-49b9-9f53-f71e4a525dcc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:24,453 [544] [DEBUG] [app] Starting request: urn:request:1df2122a-2ea7-49b9-9f53-f71e4a525dcc (/v1/_internal_ping) 2020-03-31 09:45:24,454 [544] [DEBUG] [app] Ending request: urn:request:1df2122a-2ea7-49b9-9f53-f71e4a525dcc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:24,454 [544] [DEBUG] [app] Ending request: urn:request:1df2122a-2ea7-49b9-9f53-f71e4a525dcc (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:45:24,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:45:24,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:24,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:45:24,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:24,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:24,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:24,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:24,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:24,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:24,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:24,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:24,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:24,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:24,485 [527] [DEBUG] [app] Starting request: urn:request:25864c99-279d-4192-a555-f1da4e03d572 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:24,485 [527] [DEBUG] [app] Starting request: urn:request:25864c99-279d-4192-a555-f1da4e03d572 (/_internal_ping) 2020-03-31 09:45:24,486 [527] [DEBUG] [app] Ending request: urn:request:25864c99-279d-4192-a555-f1da4e03d572 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:24,486 [527] [DEBUG] [app] Ending request: urn:request:25864c99-279d-4192-a555-f1da4e03d572 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:45:24,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:24,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:24,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:24,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:45:24,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:45:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:45:24,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:24,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:24,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:24,499 [536] [DEBUG] [app] Starting request: urn:request:a267a7ce-5e8c-4aa4-b742-01b1507b760d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:24,499 [536] [DEBUG] [app] Starting request: urn:request:a267a7ce-5e8c-4aa4-b742-01b1507b760d (/c1/_internal_ping) 2020-03-31 09:45:24,500 [536] [DEBUG] [app] Ending request: urn:request:a267a7ce-5e8c-4aa4-b742-01b1507b760d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:24,500 [536] [DEBUG] [app] Ending request: urn:request:a267a7ce-5e8c-4aa4-b742-01b1507b760d (/c1/_internal_ping) 2020-03-31 09:45:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:24,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:24,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:45:24,501 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:24,503 [518] [DEBUG] [app] Ending request: urn:request:1014cdcd-ca42-46a0-8151-08c0c043131f (/health/instance) 2020-03-31 09:45:24,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:24,503 [518] [DEBUG] [app] Ending request: urn:request:1014cdcd-ca42-46a0-8151-08c0c043131f (/health/instance) 2020-03-31 09:45:24,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:24,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:45:24,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:45:26,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:45:26,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:36.924497+00:00 (in 9.996919 seconds) 2020-03-31 09:45:26,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:36 UTC)" (scheduled at 2020-03-31 09:45:26.924497+00:00) 2020-03-31 09:45:26,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:45:26,927 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:36.924497+00:00 (in 9.996919 seconds) 2020-03-31 09:45:26,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:36 UTC)" (scheduled at 2020-03-31 09:45:26.924497+00:00) 2020-03-31 09:45:26,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 26, 928358), True, datetime.datetime(2020, 3, 31, 9, 45, 26, 928358), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:45:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 26, 928358), True, datetime.datetime(2020, 3, 31, 9, 45, 26, 928358), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:45:26,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:26,950 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:45:26,950 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:26,950 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:26,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:45:26,950 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:36 UTC)" executed successfully 2020-03-31 09:45:28,272 [527] [DEBUG] [app] Starting request: urn:request:ba206c78-10c6-4e76-a667-e5a659922b72 (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:28,272 [527] [DEBUG] [app] Starting request: urn:request:ba206c78-10c6-4e76-a667-e5a659922b72 (/health/instance) 2020-03-31 09:45:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:28,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:28,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:28,280 [538] [DEBUG] [app] Starting request: urn:request:9e496263-3376-4217-84ec-7169eda1fab0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:28,280 [538] [DEBUG] [app] Starting request: urn:request:9e496263-3376-4217-84ec-7169eda1fab0 (/v1/_internal_ping) 2020-03-31 09:45:28,281 [538] [DEBUG] [app] Ending request: urn:request:9e496263-3376-4217-84ec-7169eda1fab0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:28,281 [538] [DEBUG] [app] Ending request: urn:request:9e496263-3376-4217-84ec-7169eda1fab0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:28,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:45:28,282 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:28,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:28,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:45:28,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:45:28,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:28,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:28,286 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:28,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:28,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:28,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:28,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:28,307 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:28,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:28,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:28,313 [525] [DEBUG] [app] Starting request: urn:request:68c88cd8-b47c-4a47-b582-1146e3ec846e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:28,313 [525] [DEBUG] [app] Starting request: urn:request:68c88cd8-b47c-4a47-b582-1146e3ec846e (/_internal_ping) 2020-03-31 09:45:28,314 [525] [DEBUG] [app] Ending request: urn:request:68c88cd8-b47c-4a47-b582-1146e3ec846e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:28,314 [525] [DEBUG] [app] Ending request: urn:request:68c88cd8-b47c-4a47-b582-1146e3ec846e (/_internal_ping) 2020-03-31 09:45:28,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:28,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:45:28,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:28,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:45:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:28,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:28,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:45:28,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:28,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:28,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:45:28,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:28,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:28,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:28,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:28,328 [535] [DEBUG] [app] Starting request: urn:request:a0f544d8-9740-4982-9f04-76cf869706bb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:28,328 [535] [DEBUG] [app] Starting request: urn:request:a0f544d8-9740-4982-9f04-76cf869706bb (/c1/_internal_ping) 2020-03-31 09:45:28,329 [535] [DEBUG] [app] Ending request: urn:request:a0f544d8-9740-4982-9f04-76cf869706bb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:28,329 [535] [DEBUG] [app] Ending request: urn:request:a0f544d8-9740-4982-9f04-76cf869706bb (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:28,330 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:28,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:45:28,330 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:28,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:28,332 [527] [DEBUG] [app] Ending request: urn:request:ba206c78-10c6-4e76-a667-e5a659922b72 (/health/instance) 2020-03-31 09:45:28,332 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:28,332 [527] [DEBUG] [app] Ending request: urn:request:ba206c78-10c6-4e76-a667-e5a659922b72 (/health/instance) 2020-03-31 09:45:28,332 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:28,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:45:28,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:45:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:45:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:00.337699+00:00 (in 29.999126 seconds) 2020-03-31 09:45:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:00 UTC)" (scheduled at 2020-03-31 09:45:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:45:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:00.337699+00:00 (in 29.999126 seconds) 2020-03-31 09:45:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:00 UTC)" (scheduled at 2020-03-31 09:45:30.337699+00:00) 2020-03-31 09:45:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 45, 30, 339242), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:45:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 45, 30, 339242), False, 50, 1, 0]) 2020-03-31 09:45:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:45:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:45:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:45:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:00 UTC)" executed successfully 2020-03-31 09:45:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:45:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:41.440707+00:00 (in 9.999079 seconds) 2020-03-31 09:45:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:41 UTC)" (scheduled at 2020-03-31 09:45:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:45:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:41.440707+00:00 (in 9.999079 seconds) 2020-03-31 09:45:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:41 UTC)" (scheduled at 2020-03-31 09:45:31.440707+00:00) 2020-03-31 09:45:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:45:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:45:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:45:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:45:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:45:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:41 UTC)" executed successfully 2020-03-31 09:45:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:45:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:03.330112+00:00 (in 29.998963 seconds) 2020-03-31 09:45:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:03 UTC)" (scheduled at 2020-03-31 09:45:33.330112+00:00) 2020-03-31 09:45:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:45:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:03.330112+00:00 (in 29.998963 seconds) 2020-03-31 09:45:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:03 UTC)" (scheduled at 2020-03-31 09:45:33.330112+00:00) 2020-03-31 09:45:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 33, 332351), True, datetime.datetime(2020, 3, 31, 9, 45, 33, 332351), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:45:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 33, 332351), True, datetime.datetime(2020, 3, 31, 9, 45, 33, 332351), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:45:33,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:33,355 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:45:33,355 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:33,355 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:33,356 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:45:33,356 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:03 UTC)" executed successfully 2020-03-31 09:45:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:45:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:43.539999+00:00 (in 10.172337 seconds) 2020-03-31 09:45:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:33 UTC)" (scheduled at 2020-03-31 09:45:33.366791+00:00) 2020-03-31 09:45:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:45:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:43.539999+00:00 (in 10.172337 seconds) 2020-03-31 09:45:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:33 UTC)" (scheduled at 2020-03-31 09:45:33.366791+00:00) 2020-03-31 09:45:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 33, 368450), True, datetime.datetime(2020, 3, 31, 9, 45, 33, 368450), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:45:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 33, 368450), True, datetime.datetime(2020, 3, 31, 9, 45, 33, 368450), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:45:33,385 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:45:33,385 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:45:33,385 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:33,386 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:33 UTC)" executed successfully 2020-03-31 09:45:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:45:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:44.323352+00:00 (in 9.999237 seconds) 2020-03-31 09:45:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:44 UTC)" (scheduled at 2020-03-31 09:45:34.323352+00:00) 2020-03-31 09:45:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:45:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:44.323352+00:00 (in 9.999237 seconds) 2020-03-31 09:45:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:44 UTC)" (scheduled at 2020-03-31 09:45:34.323352+00:00) 2020-03-31 09:45:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 34, 325009), True, datetime.datetime(2020, 3, 31, 9, 45, 34, 325009), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:45:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 34, 325009), True, datetime.datetime(2020, 3, 31, 9, 45, 34, 325009), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:45:34,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:45:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:45:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:44 UTC)" executed successfully 2020-03-31 09:45:34,446 [518] [DEBUG] [app] Starting request: urn:request:02f4f7c6-9fc3-47c0-a5d3-1278388ada55 (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:34,446 [518] [DEBUG] [app] Starting request: urn:request:02f4f7c6-9fc3-47c0-a5d3-1278388ada55 (/health/instance) 2020-03-31 09:45:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:34,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:34,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:34,452 [549] [DEBUG] [app] Starting request: urn:request:7eaf2cd1-b3ea-4668-9fd7-099b73688ce7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:34,452 [549] [DEBUG] [app] Starting request: urn:request:7eaf2cd1-b3ea-4668-9fd7-099b73688ce7 (/v1/_internal_ping) 2020-03-31 09:45:34,453 [549] [DEBUG] [app] Ending request: urn:request:7eaf2cd1-b3ea-4668-9fd7-099b73688ce7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:34,453 [549] [DEBUG] [app] Ending request: urn:request:7eaf2cd1-b3ea-4668-9fd7-099b73688ce7 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:34,454 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:45:34,454 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:34,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:34,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:34,456 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:45:34,456 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:34,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:34,458 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:34,458 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:34,458 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:34,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:34,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:34,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:34,482 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:34,484 [525] [DEBUG] [app] Starting request: urn:request:70c26179-7325-4e8c-8b2d-e4b1d8894ef3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:34,484 [525] [DEBUG] [app] Starting request: urn:request:70c26179-7325-4e8c-8b2d-e4b1d8894ef3 (/_internal_ping) 2020-03-31 09:45:34,485 [525] [DEBUG] [app] Ending request: urn:request:70c26179-7325-4e8c-8b2d-e4b1d8894ef3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:34,485 [525] [DEBUG] [app] Ending request: urn:request:70c26179-7325-4e8c-8b2d-e4b1d8894ef3 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:45:34,485 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:34,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:34,485 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:34,485 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:45:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:34,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:45:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:34,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:34,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:34,499 [535] [DEBUG] [app] Starting request: urn:request:4e72466b-a277-411d-8998-16015a6b658f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:34,499 [535] [DEBUG] [app] Starting request: urn:request:4e72466b-a277-411d-8998-16015a6b658f (/c1/_internal_ping) 2020-03-31 09:45:34,500 [535] [DEBUG] [app] Ending request: urn:request:4e72466b-a277-411d-8998-16015a6b658f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:34,500 [535] [DEBUG] [app] Ending request: urn:request:4e72466b-a277-411d-8998-16015a6b658f (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:34,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:45:34,501 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:34,503 [518] [DEBUG] [app] Ending request: urn:request:02f4f7c6-9fc3-47c0-a5d3-1278388ada55 (/health/instance) 2020-03-31 09:45:34,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:34,503 [518] [DEBUG] [app] Ending request: urn:request:02f4f7c6-9fc3-47c0-a5d3-1278388ada55 (/health/instance) 2020-03-31 09:45:34,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:45:34,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:45:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:45:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:38.010934+00:00 (in 1.085646 seconds) 2020-03-31 09:45:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:46 UTC)" (scheduled at 2020-03-31 09:45:36.924497+00:00) 2020-03-31 09:45:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:45:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:38.010934+00:00 (in 1.085646 seconds) 2020-03-31 09:45:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:46 UTC)" (scheduled at 2020-03-31 09:45:36.924497+00:00) 2020-03-31 09:45:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 36, 925985), True, datetime.datetime(2020, 3, 31, 9, 45, 36, 925985), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:45:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 36, 925985), True, datetime.datetime(2020, 3, 31, 9, 45, 36, 925985), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:45:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:45:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:45:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:46 UTC)" executed successfully 2020-03-31 09:45:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:45:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:46.924497+00:00 (in 8.912856 seconds) 2020-03-31 09:45:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:38 UTC)" (scheduled at 2020-03-31 09:45:38.010934+00:00) 2020-03-31 09:45:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:45:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:46.924497+00:00 (in 8.912856 seconds) 2020-03-31 09:45:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:38 UTC)" (scheduled at 2020-03-31 09:45:38.010934+00:00) 2020-03-31 09:45:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:38 UTC)" executed successfully 2020-03-31 09:45:38,273 [518] [DEBUG] [app] Starting request: urn:request:d220151d-2e85-486c-8fd1-ddb95996f49e (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:38,273 [518] [DEBUG] [app] Starting request: urn:request:d220151d-2e85-486c-8fd1-ddb95996f49e (/health/instance) 2020-03-31 09:45:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:38,278 [556] [DEBUG] [app] Starting request: urn:request:94338112-97bc-4a68-9ebc-3ab8d5f7556f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:38,278 [556] [DEBUG] [app] Starting request: urn:request:94338112-97bc-4a68-9ebc-3ab8d5f7556f (/v1/_internal_ping) 2020-03-31 09:45:38,279 [556] [DEBUG] [app] Ending request: urn:request:94338112-97bc-4a68-9ebc-3ab8d5f7556f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:38,279 [556] [DEBUG] [app] Ending request: urn:request:94338112-97bc-4a68-9ebc-3ab8d5f7556f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:38,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:38,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:45:38,280 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:38,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:38,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:38,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:45:38,282 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:38,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:38,284 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:38,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:38,285 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:38,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:38,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:38,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:38,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:38,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:38,313 [518] [DEBUG] [app] Starting request: urn:request:01b13c3a-4878-4654-8588-5cd6a6fb13a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:38,313 [518] [DEBUG] [app] Starting request: urn:request:01b13c3a-4878-4654-8588-5cd6a6fb13a0 (/_internal_ping) 2020-03-31 09:45:38,314 [518] [DEBUG] [app] Ending request: urn:request:01b13c3a-4878-4654-8588-5cd6a6fb13a0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:38,314 [518] [DEBUG] [app] Ending request: urn:request:01b13c3a-4878-4654-8588-5cd6a6fb13a0 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:45:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:38,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:45:38,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:45:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:38,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:45:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:38,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:38,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:38,327 [535] [DEBUG] [app] Starting request: urn:request:35722709-91bf-4526-aa7e-b1fb4daa27cd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:38,327 [535] [DEBUG] [app] Starting request: urn:request:35722709-91bf-4526-aa7e-b1fb4daa27cd (/c1/_internal_ping) 2020-03-31 09:45:38,328 [535] [DEBUG] [app] Ending request: urn:request:35722709-91bf-4526-aa7e-b1fb4daa27cd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:38,328 [535] [DEBUG] [app] Ending request: urn:request:35722709-91bf-4526-aa7e-b1fb4daa27cd (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:45:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:38,328 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:38,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:45:38,328 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:38,330 [518] [DEBUG] [app] Ending request: urn:request:d220151d-2e85-486c-8fd1-ddb95996f49e (/health/instance) 2020-03-31 09:45:38,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:38,330 [518] [DEBUG] [app] Ending request: urn:request:d220151d-2e85-486c-8fd1-ddb95996f49e (/health/instance) 2020-03-31 09:45:38,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:45:38,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:45:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:45:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:51.440707+00:00 (in 9.998960 seconds) 2020-03-31 09:45:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:51 UTC)" (scheduled at 2020-03-31 09:45:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:45:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:51.440707+00:00 (in 9.998960 seconds) 2020-03-31 09:45:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:51 UTC)" (scheduled at 2020-03-31 09:45:41.440707+00:00) 2020-03-31 09:45:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:45:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:45:41,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:45:41,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:45:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:45:41,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:41,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:51 UTC)" executed successfully 2020-03-31 09:45:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:45:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:43 UTC)" (scheduled at 2020-03-31 09:45:43.539999+00:00) 2020-03-31 09:45:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2020-03-31 09:45:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:43 UTC)" (scheduled at 2020-03-31 09:45:43.539999+00:00) 2020-03-31 09:45:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:43 UTC)" executed successfully 2020-03-31 09:45:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:33.366791+00:00 (in 49.826099 seconds) exportactionlogsworker stdout | 2020-03-31 09:45:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:43 UTC)" executed successfully 2020-03-31 09:45:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:33.366791+00:00 (in 49.826099 seconds) 2020-03-31 09:45:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:45:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:54 UTC)" (scheduled at 2020-03-31 09:45:44.323352+00:00) 2020-03-31 09:45:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:45:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:54 UTC)" (scheduled at 2020-03-31 09:45:44.323352+00:00) 2020-03-31 09:45:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:49.498966+00:00 (in 5.174751 seconds) notificationworker stdout | 2020-03-31 09:45:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:49.498966+00:00 (in 5.174751 seconds) 2020-03-31 09:45:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 44, 325237), True, datetime.datetime(2020, 3, 31, 9, 45, 44, 325237), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:45:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 44, 325237), True, datetime.datetime(2020, 3, 31, 9, 45, 44, 325237), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:45:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:45:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:45:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:54 UTC)" executed successfully 2020-03-31 09:45:44,446 [518] [DEBUG] [app] Starting request: urn:request:8cc23ffb-9231-4cba-9537-f0dda490428d (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:44,446 [518] [DEBUG] [app] Starting request: urn:request:8cc23ffb-9231-4cba-9537-f0dda490428d (/health/instance) 2020-03-31 09:45:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:44,453 [544] [DEBUG] [app] Starting request: urn:request:c1a18edd-fc54-4580-999c-7e39edef61f3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:44,453 [544] [DEBUG] [app] Starting request: urn:request:c1a18edd-fc54-4580-999c-7e39edef61f3 (/v1/_internal_ping) 2020-03-31 09:45:44,454 [544] [DEBUG] [app] Ending request: urn:request:c1a18edd-fc54-4580-999c-7e39edef61f3 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:44,454 [544] [DEBUG] [app] Ending request: urn:request:c1a18edd-fc54-4580-999c-7e39edef61f3 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:44,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:45:44,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:44,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:45:44,458 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:44,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:44,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:44,459 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:44,460 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:44,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:44,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:44,485 [518] [DEBUG] [app] Starting request: urn:request:387b0aca-36e7-447a-a001-6bc178b60250 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:44,485 [518] [DEBUG] [app] Starting request: urn:request:387b0aca-36e7-447a-a001-6bc178b60250 (/_internal_ping) 2020-03-31 09:45:44,486 [518] [DEBUG] [app] Ending request: urn:request:387b0aca-36e7-447a-a001-6bc178b60250 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:44,486 [518] [DEBUG] [app] Ending request: urn:request:387b0aca-36e7-447a-a001-6bc178b60250 (/_internal_ping) 2020-03-31 09:45:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:44,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:44,490 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:45:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:45:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:45:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:44,499 [541] [DEBUG] [app] Starting request: urn:request:012fec8b-fd24-4d1f-8ef8-3142298b9f07 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:44,499 [541] [DEBUG] [app] Starting request: urn:request:012fec8b-fd24-4d1f-8ef8-3142298b9f07 (/c1/_internal_ping) 2020-03-31 09:45:44,500 [541] [DEBUG] [app] Ending request: urn:request:012fec8b-fd24-4d1f-8ef8-3142298b9f07 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:44,500 [541] [DEBUG] [app] Ending request: urn:request:012fec8b-fd24-4d1f-8ef8-3142298b9f07 (/c1/_internal_ping) 2020-03-31 09:45:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:44,500 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:44,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:45:44,500 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:44,502 [518] [DEBUG] [app] Ending request: urn:request:8cc23ffb-9231-4cba-9537-f0dda490428d (/health/instance) 2020-03-31 09:45:44,502 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:44,502 [518] [DEBUG] [app] Ending request: urn:request:8cc23ffb-9231-4cba-9537-f0dda490428d (/health/instance) 2020-03-31 09:45:44,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:44,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:45:44,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:45:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:45:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:15.064514+00:00 (in 29.999214 seconds) 2020-03-31 09:45:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:15 UTC)" (scheduled at 2020-03-31 09:45:45.064514+00:00) gcworker stdout | 2020-03-31 09:45:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:15.064514+00:00 (in 29.999214 seconds) 2020-03-31 09:45:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:15 UTC)" (scheduled at 2020-03-31 09:45:45.064514+00:00) 2020-03-31 09:45:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438345, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:45:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438345, 1209600, 500, 1, 0]) 2020-03-31 09:45:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:45:45,083 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:45:45,083 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:45:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:45:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:15 UTC)" executed successfully 2020-03-31 09:45:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:45:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:56.924497+00:00 (in 9.999217 seconds) 2020-03-31 09:45:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:56 UTC)" (scheduled at 2020-03-31 09:45:46.924497+00:00) 2020-03-31 09:45:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:45:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:56.924497+00:00 (in 9.999217 seconds) 2020-03-31 09:45:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:56 UTC)" (scheduled at 2020-03-31 09:45:46.924497+00:00) 2020-03-31 09:45:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 46, 926148), True, datetime.datetime(2020, 3, 31, 9, 45, 46, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:45:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 46, 926148), True, datetime.datetime(2020, 3, 31, 9, 45, 46, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:45:46,945 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:45:46,945 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:45:46,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:46,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:45:56 UTC)" executed successfully 2020-03-31 09:45:48,273 [525] [DEBUG] [app] Starting request: urn:request:e564619b-09b6-44bd-af32-743d11ef309c (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:48,273 [525] [DEBUG] [app] Starting request: urn:request:e564619b-09b6-44bd-af32-743d11ef309c (/health/instance) 2020-03-31 09:45:48,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:48,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:48,279 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:48,279 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:48,280 [547] [DEBUG] [app] Starting request: urn:request:3384ab9d-9f28-445f-aaf2-57fc665bff14 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:48,280 [547] [DEBUG] [app] Starting request: urn:request:3384ab9d-9f28-445f-aaf2-57fc665bff14 (/v1/_internal_ping) 2020-03-31 09:45:48,282 [547] [DEBUG] [app] Ending request: urn:request:3384ab9d-9f28-445f-aaf2-57fc665bff14 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:48,282 [547] [DEBUG] [app] Ending request: urn:request:3384ab9d-9f28-445f-aaf2-57fc665bff14 (/v1/_internal_ping) 2020-03-31 09:45:48,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:48,283 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:45:48,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:48,283 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:48,284 [525] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:45:48,284 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:48,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:45:48,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:48,287 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:48,287 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:48,288 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:48,288 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:48,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:48,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:48,308 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:48,308 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:48,312 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:48,312 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:48,313 [525] [DEBUG] [app] Starting request: urn:request:871de6cf-69aa-4960-b728-7f70f7ff57b4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:48,313 [525] [DEBUG] [app] Starting request: urn:request:871de6cf-69aa-4960-b728-7f70f7ff57b4 (/_internal_ping) 2020-03-31 09:45:48,314 [525] [DEBUG] [app] Ending request: urn:request:871de6cf-69aa-4960-b728-7f70f7ff57b4 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:48,314 [525] [DEBUG] [app] Ending request: urn:request:871de6cf-69aa-4960-b728-7f70f7ff57b4 (/_internal_ping) 2020-03-31 09:45:48,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:48,314 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:48,315 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:45:48,315 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:48,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:48,318 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:48,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:45:48,318 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:45:48,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:48,321 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:45:48,321 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:48,321 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:48,325 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:48,325 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:48,326 [536] [DEBUG] [app] Starting request: urn:request:dc63b5d9-f9e1-4516-b173-63df11ade4c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:48,326 [536] [DEBUG] [app] Starting request: urn:request:dc63b5d9-f9e1-4516-b173-63df11ade4c9 (/c1/_internal_ping) 2020-03-31 09:45:48,327 [536] [DEBUG] [app] Ending request: urn:request:dc63b5d9-f9e1-4516-b173-63df11ade4c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:48,327 [536] [DEBUG] [app] Ending request: urn:request:dc63b5d9-f9e1-4516-b173-63df11ade4c9 (/c1/_internal_ping) 2020-03-31 09:45:48,327 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:45:48,327 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:48,327 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:45:48,329 [525] [DEBUG] [app] Ending request: urn:request:e564619b-09b6-44bd-af32-743d11ef309c (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:45:48,327 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:48,329 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:48,330 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:45:48,329 [525] [DEBUG] [app] Ending request: urn:request:e564619b-09b6-44bd-af32-743d11ef309c (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:48,329 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:48,330 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:45:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:45:49,501 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:54.323352+00:00 (in 4.821892 seconds) 2020-03-31 09:45:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:49 UTC)" (scheduled at 2020-03-31 09:45:49.498966+00:00) 2020-03-31 09:45:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:45:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:54.323352+00:00 (in 4.821892 seconds) 2020-03-31 09:45:49,502 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:49 UTC)" (scheduled at 2020-03-31 09:45:49.498966+00:00) 2020-03-31 09:45:49,502 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:49,502 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:49 UTC)" executed successfully 2020-03-31 09:45:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:45:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:59.560326+00:00 (in 8.449024 seconds) 2020-03-31 09:45:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:51 UTC)" (scheduled at 2020-03-31 09:45:51.110610+00:00) 2020-03-31 09:45:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:45:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:45:59.560326+00:00 (in 8.449024 seconds) 2020-03-31 09:45:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:51 UTC)" (scheduled at 2020-03-31 09:45:51.110610+00:00) 2020-03-31 09:45:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 51, 112317), True, datetime.datetime(2020, 3, 31, 9, 45, 51, 112317), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:45:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 51, 112317), True, datetime.datetime(2020, 3, 31, 9, 45, 51, 112317), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:45:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:45:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:45:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:51 UTC)" executed successfully 2020-03-31 09:45:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:45:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:01 UTC)" (scheduled at 2020-03-31 09:45:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:45:51,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:01 UTC)" (scheduled at 2020-03-31 09:45:51.440707+00:00) 2020-03-31 09:45:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:01.440707+00:00 (in 9.999214 seconds) repositoryactioncounter stdout | 2020-03-31 09:45:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:01.440707+00:00 (in 9.999214 seconds) 2020-03-31 09:45:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:45:51,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:45:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:45:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:45:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:45:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:01 UTC)" executed successfully 2020-03-31 09:45:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:45:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:22.113221+00:00 (in 29.999063 seconds) 2020-03-31 09:45:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:22 UTC)" (scheduled at 2020-03-31 09:45:52.113221+00:00) securityworker stdout | 2020-03-31 09:45:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:22.113221+00:00 (in 29.999063 seconds) 2020-03-31 09:45:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:22 UTC)" (scheduled at 2020-03-31 09:45:52.113221+00:00) 2020-03-31 09:45:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:45:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:45:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:45:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:45:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:45:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:45:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:45:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:22 UTC)" executed successfully 2020-03-31 09:45:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:45:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:04.323352+00:00 (in 9.999217 seconds) 2020-03-31 09:45:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:04 UTC)" (scheduled at 2020-03-31 09:45:54.323352+00:00) 2020-03-31 09:45:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:45:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:04.323352+00:00 (in 9.999217 seconds) 2020-03-31 09:45:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:04 UTC)" (scheduled at 2020-03-31 09:45:54.323352+00:00) 2020-03-31 09:45:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 54, 324947), True, datetime.datetime(2020, 3, 31, 9, 45, 54, 324947), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:45:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 54, 324947), True, datetime.datetime(2020, 3, 31, 9, 45, 54, 324947), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:45:54,339 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:45:54,339 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:54,339 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:54,339 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:45:54,339 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:54,339 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:04 UTC)" executed successfully 2020-03-31 09:45:54,446 [526] [DEBUG] [app] Starting request: urn:request:241ad8cc-12e9-4fc4-ac37-f9f3ad8baa1a (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:54,446 [526] [DEBUG] [app] Starting request: urn:request:241ad8cc-12e9-4fc4-ac37-f9f3ad8baa1a (/health/instance) 2020-03-31 09:45:54,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:54,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:54,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:54,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:54,453 [548] [DEBUG] [app] Starting request: urn:request:a34c54ca-edf3-452a-80ca-49f06c9fc330 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:54,453 [548] [DEBUG] [app] Starting request: urn:request:a34c54ca-edf3-452a-80ca-49f06c9fc330 (/v1/_internal_ping) 2020-03-31 09:45:54,455 [548] [DEBUG] [app] Ending request: urn:request:a34c54ca-edf3-452a-80ca-49f06c9fc330 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:54,455 [548] [DEBUG] [app] Ending request: urn:request:a34c54ca-edf3-452a-80ca-49f06c9fc330 (/v1/_internal_ping) 2020-03-31 09:45:54,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:54,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:45:54,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:54,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:54,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:54,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:45:54,457 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:54,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:54,457 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:54,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:54,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:54,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:54,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:54,470 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:54,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:54,485 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:54,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:54,489 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:54,491 [518] [DEBUG] [app] Starting request: urn:request:2d5595d3-9748-4479-9411-29db0569ff8b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:54,491 [518] [DEBUG] [app] Starting request: urn:request:2d5595d3-9748-4479-9411-29db0569ff8b (/_internal_ping) 2020-03-31 09:45:54,492 [518] [DEBUG] [app] Ending request: urn:request:2d5595d3-9748-4479-9411-29db0569ff8b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:54,492 [518] [DEBUG] [app] Ending request: urn:request:2d5595d3-9748-4479-9411-29db0569ff8b (/_internal_ping) 2020-03-31 09:45:54,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:54,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:45:54,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:54,492 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:45:54,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:54,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:54,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:45:54,496 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:45:54,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:54,499 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:45:54,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:54,500 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:54,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:54,503 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:54,505 [536] [DEBUG] [app] Starting request: urn:request:461643fd-6cd3-4ce0-8739-c2d3d623a8bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:54,505 [536] [DEBUG] [app] Starting request: urn:request:461643fd-6cd3-4ce0-8739-c2d3d623a8bd (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:45:54,505 [536] [DEBUG] [app] Ending request: urn:request:461643fd-6cd3-4ce0-8739-c2d3d623a8bd (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:45:54,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:54,506 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:45:54,505 [536] [DEBUG] [app] Ending request: urn:request:461643fd-6cd3-4ce0-8739-c2d3d623a8bd (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:54,506 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:54,508 [526] [DEBUG] [app] Ending request: urn:request:241ad8cc-12e9-4fc4-ac37-f9f3ad8baa1a (/health/instance) 2020-03-31 09:45:54,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:45:54,506 [536] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:54,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:45:54,508 [526] [DEBUG] [app] Ending request: urn:request:241ad8cc-12e9-4fc4-ac37-f9f3ad8baa1a (/health/instance) 2020-03-31 09:45:54,508 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:54,509 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:45:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:45:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:06.924497+00:00 (in 9.999115 seconds) 2020-03-31 09:45:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:06 UTC)" (scheduled at 2020-03-31 09:45:56.924497+00:00) 2020-03-31 09:45:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:45:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:06.924497+00:00 (in 9.999115 seconds) 2020-03-31 09:45:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:06 UTC)" (scheduled at 2020-03-31 09:45:56.924497+00:00) 2020-03-31 09:45:56,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:45:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 56, 926062), True, datetime.datetime(2020, 3, 31, 9, 45, 56, 926062), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:45:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 45, 56, 926062), True, datetime.datetime(2020, 3, 31, 9, 45, 56, 926062), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:45:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:45:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:45:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:45:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:45:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:06 UTC)" executed successfully 2020-03-31 09:45:58,273 [526] [DEBUG] [app] Starting request: urn:request:9bcd4104-1084-4a5e-9768-e7869d04a673 (/health/instance) gunicorn-web stdout | 2020-03-31 09:45:58,273 [526] [DEBUG] [app] Starting request: urn:request:9bcd4104-1084-4a5e-9768-e7869d04a673 (/health/instance) 2020-03-31 09:45:58,276 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:58,276 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:58,278 [538] [DEBUG] [app] Starting request: urn:request:71cdbce7-50cd-4b2b-9b9a-cfd23defe84e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:58,278 [538] [DEBUG] [app] Starting request: urn:request:71cdbce7-50cd-4b2b-9b9a-cfd23defe84e (/v1/_internal_ping) 2020-03-31 09:45:58,279 [538] [DEBUG] [app] Ending request: urn:request:71cdbce7-50cd-4b2b-9b9a-cfd23defe84e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:45:58,279 [538] [DEBUG] [app] Ending request: urn:request:71cdbce7-50cd-4b2b-9b9a-cfd23defe84e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:45:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:58,280 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:45:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:58,281 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:45:58,280 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:45:58,281 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:45:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:45:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:45:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:45:58,284 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:45:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:45:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:45:58,290 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:45:58,290 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:45:58,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:45:58,311 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:45:58,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:58,314 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:58,315 [526] [DEBUG] [app] Starting request: urn:request:78ba934a-f4cf-405b-989b-b82407689476 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:58,315 [526] [DEBUG] [app] Starting request: urn:request:78ba934a-f4cf-405b-989b-b82407689476 (/_internal_ping) 2020-03-31 09:45:58,316 [526] [DEBUG] [app] Ending request: urn:request:78ba934a-f4cf-405b-989b-b82407689476 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:45:58,316 [526] [DEBUG] [app] Ending request: urn:request:78ba934a-f4cf-405b-989b-b82407689476 (/_internal_ping) 2020-03-31 09:45:58,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:45:58,317 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:58,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:45:58,317 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:45:58,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:58,320 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:58,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:45:58,321 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:45:58,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:58,323 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:45:58,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:45:58,324 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:45:58,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:45:58,327 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:45:58,328 [537] [DEBUG] [app] Starting request: urn:request:8833a361-bfa4-4c01-8c79-d8358c056621 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:58,328 [537] [DEBUG] [app] Starting request: urn:request:8833a361-bfa4-4c01-8c79-d8358c056621 (/c1/_internal_ping) 2020-03-31 09:45:58,329 [537] [DEBUG] [app] Ending request: urn:request:8833a361-bfa4-4c01-8c79-d8358c056621 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:45:58,329 [537] [DEBUG] [app] Ending request: urn:request:8833a361-bfa4-4c01-8c79-d8358c056621 (/c1/_internal_ping) 2020-03-31 09:45:58,330 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:45:58,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:45:58,330 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:45:58,331 [526] [DEBUG] [app] Ending request: urn:request:9bcd4104-1084-4a5e-9768-e7869d04a673 (/health/instance) 2020-03-31 09:45:58,331 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:45:58,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:45:58,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:45:58,331 [526] [DEBUG] [app] Ending request: urn:request:9bcd4104-1084-4a5e-9768-e7869d04a673 (/health/instance) 2020-03-31 09:45:58,331 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:45:58,331 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:45:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:45:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:45:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:45:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:45:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:51.110610+00:00 (in 51.549443 seconds) 2020-03-31 09:45:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:59 UTC)" (scheduled at 2020-03-31 09:45:59.560326+00:00) 2020-03-31 09:45:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:45:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:51.110610+00:00 (in 51.549443 seconds) 2020-03-31 09:45:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:59 UTC)" (scheduled at 2020-03-31 09:45:59.560326+00:00) 2020-03-31 09:45:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:45:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:46:59 UTC)" executed successfully 2020-03-31 09:46:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:46:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:30.337699+00:00 (in 29.999217 seconds) 2020-03-31 09:46:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:30 UTC)" (scheduled at 2020-03-31 09:46:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:46:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:30.337699+00:00 (in 29.999217 seconds) 2020-03-31 09:46:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:30 UTC)" (scheduled at 2020-03-31 09:46:00.337699+00:00) 2020-03-31 09:46:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 46, 0, 339393), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:46:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 46, 0, 339393), False, 50, 1, 0]) 2020-03-31 09:46:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:46:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:46:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:46:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:30 UTC)" executed successfully 2020-03-31 09:46:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:46:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:11.440707+00:00 (in 9.997542 seconds) 2020-03-31 09:46:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:11 UTC)" (scheduled at 2020-03-31 09:46:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:46:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:11.440707+00:00 (in 9.997542 seconds) 2020-03-31 09:46:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:11 UTC)" (scheduled at 2020-03-31 09:46:01.440707+00:00) 2020-03-31 09:46:01,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:46:01,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:46:01,463 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:46:01,463 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:46:01,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:01,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:46:01,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:01,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:11 UTC)" executed successfully 2020-03-31 09:46:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:46:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:24.410927+00:00 (in 21.079698 seconds) 2020-03-31 09:46:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:33 UTC)" (scheduled at 2020-03-31 09:46:03.330112+00:00) 2020-03-31 09:46:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:46:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:24.410927+00:00 (in 21.079698 seconds) 2020-03-31 09:46:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:33 UTC)" (scheduled at 2020-03-31 09:46:03.330112+00:00) 2020-03-31 09:46:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 3, 332070), True, datetime.datetime(2020, 3, 31, 9, 46, 3, 332070), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:46:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 3, 332070), True, datetime.datetime(2020, 3, 31, 9, 46, 3, 332070), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:46:03,348 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:46:03,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:46:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:03,349 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:33 UTC)" executed successfully 2020-03-31 09:46:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:46:04,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:14.323352+00:00 (in 9.999210 seconds) 2020-03-31 09:46:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:14 UTC)" (scheduled at 2020-03-31 09:46:04.323352+00:00) 2020-03-31 09:46:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:46:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:14.323352+00:00 (in 9.999210 seconds) 2020-03-31 09:46:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:14 UTC)" (scheduled at 2020-03-31 09:46:04.323352+00:00) 2020-03-31 09:46:04,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 4, 325001), True, datetime.datetime(2020, 3, 31, 9, 46, 4, 325001), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:46:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 4, 325001), True, datetime.datetime(2020, 3, 31, 9, 46, 4, 325001), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:46:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:46:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:46:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:14 UTC)" executed successfully 2020-03-31 09:46:04,446 [527] [DEBUG] [app] Starting request: urn:request:710d006f-629f-4c3d-9dd8-3b4fcfffe588 (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:04,446 [527] [DEBUG] [app] Starting request: urn:request:710d006f-629f-4c3d-9dd8-3b4fcfffe588 (/health/instance) 2020-03-31 09:46:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:04,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:04,453 [546] [DEBUG] [app] Starting request: urn:request:46a57d80-27c5-40d8-ae5a-50ebc177cea8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:04,453 [546] [DEBUG] [app] Starting request: urn:request:46a57d80-27c5-40d8-ae5a-50ebc177cea8 (/v1/_internal_ping) 2020-03-31 09:46:04,454 [546] [DEBUG] [app] Ending request: urn:request:46a57d80-27c5-40d8-ae5a-50ebc177cea8 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:04,454 [546] [DEBUG] [app] Ending request: urn:request:46a57d80-27c5-40d8-ae5a-50ebc177cea8 (/v1/_internal_ping) 2020-03-31 09:46:04,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:46:04,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:04,457 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:46:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:46:04,457 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:04,458 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:04,458 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:04,459 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:04,459 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:04,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:04,466 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:04,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:04,478 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:04,482 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:04,483 [518] [DEBUG] [app] Starting request: urn:request:d3b59d81-c76d-4be7-a75a-8f483098a51d (/_internal_ping) 2020-03-31 09:46:04,484 [518] [DEBUG] [app] Ending request: urn:request:d3b59d81-c76d-4be7-a75a-8f483098a51d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:04,482 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:04,485 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:04,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:04,483 [518] [DEBUG] [app] Starting request: urn:request:d3b59d81-c76d-4be7-a75a-8f483098a51d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:04,484 [518] [DEBUG] [app] Ending request: urn:request:d3b59d81-c76d-4be7-a75a-8f483098a51d (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:04,485 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:04,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:04,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:04,488 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:46:04,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:46:04,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:04,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:46:04,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:46:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:04,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:04,498 [543] [DEBUG] [app] Starting request: urn:request:f7b112fa-dee9-4cca-9d9a-c817d5f260a2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:04,498 [543] [DEBUG] [app] Starting request: urn:request:f7b112fa-dee9-4cca-9d9a-c817d5f260a2 (/c1/_internal_ping) 2020-03-31 09:46:04,498 [543] [DEBUG] [app] Ending request: urn:request:f7b112fa-dee9-4cca-9d9a-c817d5f260a2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:04,498 [543] [DEBUG] [app] Ending request: urn:request:f7b112fa-dee9-4cca-9d9a-c817d5f260a2 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:04,499 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:46:04,499 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:04,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:04,501 [527] [DEBUG] [app] Ending request: urn:request:710d006f-629f-4c3d-9dd8-3b4fcfffe588 (/health/instance) 2020-03-31 09:46:04,501 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:04,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:46:04,501 [527] [DEBUG] [app] Ending request: urn:request:710d006f-629f-4c3d-9dd8-3b4fcfffe588 (/health/instance) 2020-03-31 09:46:04,501 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:04,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:46:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:46:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:16.924497+00:00 (in 9.999316 seconds) 2020-03-31 09:46:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:16 UTC)" (scheduled at 2020-03-31 09:46:06.924497+00:00) 2020-03-31 09:46:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:46:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:16.924497+00:00 (in 9.999316 seconds) 2020-03-31 09:46:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:16 UTC)" (scheduled at 2020-03-31 09:46:06.924497+00:00) 2020-03-31 09:46:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 6, 925924), True, datetime.datetime(2020, 3, 31, 9, 46, 6, 925924), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:46:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 6, 925924), True, datetime.datetime(2020, 3, 31, 9, 46, 6, 925924), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:46:06,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:46:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:46:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:16 UTC)" executed successfully 2020-03-31 09:46:08,273 [518] [DEBUG] [app] Starting request: urn:request:23c3af2a-2955-46bb-9111-114abbbe352e (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:08,273 [518] [DEBUG] [app] Starting request: urn:request:23c3af2a-2955-46bb-9111-114abbbe352e (/health/instance) 2020-03-31 09:46:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:08,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:08,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:08,282 [538] [DEBUG] [app] Starting request: urn:request:c2e00dab-589c-4c58-8735-4195990af78e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:08,282 [538] [DEBUG] [app] Starting request: urn:request:c2e00dab-589c-4c58-8735-4195990af78e (/v1/_internal_ping) 2020-03-31 09:46:08,283 [538] [DEBUG] [app] Ending request: urn:request:c2e00dab-589c-4c58-8735-4195990af78e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:08,283 [538] [DEBUG] [app] Ending request: urn:request:c2e00dab-589c-4c58-8735-4195990af78e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:08,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:46:08,283 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:08,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:08,286 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:08,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:08,287 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:08,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:08,288 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:08,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:08,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:08,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:08,315 [527] [DEBUG] [app] Starting request: urn:request:ebca14a0-6460-4737-8d3d-b71cd599c6c8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:08,315 [527] [DEBUG] [app] Starting request: urn:request:ebca14a0-6460-4737-8d3d-b71cd599c6c8 (/_internal_ping) 2020-03-31 09:46:08,316 [527] [DEBUG] [app] Ending request: urn:request:ebca14a0-6460-4737-8d3d-b71cd599c6c8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:08,316 [527] [DEBUG] [app] Ending request: urn:request:ebca14a0-6460-4737-8d3d-b71cd599c6c8 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:46:08,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:08,317 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:08,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:46:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:46:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:46:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:46:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:08,329 [541] [DEBUG] [app] Starting request: urn:request:a67679de-c7b7-4a61-a847-294ae061ce7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:08,329 [541] [DEBUG] [app] Starting request: urn:request:a67679de-c7b7-4a61-a847-294ae061ce7c (/c1/_internal_ping) 2020-03-31 09:46:08,330 [541] [DEBUG] [app] Ending request: urn:request:a67679de-c7b7-4a61-a847-294ae061ce7c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:08,330 [541] [DEBUG] [app] Ending request: urn:request:a67679de-c7b7-4a61-a847-294ae061ce7c (/c1/_internal_ping) 2020-03-31 09:46:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:08,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:46:08,331 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:08,333 [518] [DEBUG] [app] Ending request: urn:request:23c3af2a-2955-46bb-9111-114abbbe352e (/health/instance) 2020-03-31 09:46:08,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:08,333 [518] [DEBUG] [app] Ending request: urn:request:23c3af2a-2955-46bb-9111-114abbbe352e (/health/instance) 2020-03-31 09:46:08,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:46:08,334 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:46:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:46:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:21.440707+00:00 (in 9.998780 seconds) 2020-03-31 09:46:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:21 UTC)" (scheduled at 2020-03-31 09:46:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:46:11,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:21.440707+00:00 (in 9.998780 seconds) 2020-03-31 09:46:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:21 UTC)" (scheduled at 2020-03-31 09:46:11.440707+00:00) 2020-03-31 09:46:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:46:11,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:46:11,466 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:46:11,466 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:46:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:46:11,466 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:11,466 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:21 UTC)" executed successfully 2020-03-31 09:46:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:46:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:24.323352+00:00 (in 9.999095 seconds) 2020-03-31 09:46:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:24 UTC)" (scheduled at 2020-03-31 09:46:14.323352+00:00) 2020-03-31 09:46:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:46:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:24.323352+00:00 (in 9.999095 seconds) 2020-03-31 09:46:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:24 UTC)" (scheduled at 2020-03-31 09:46:14.323352+00:00) 2020-03-31 09:46:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 14, 325184), True, datetime.datetime(2020, 3, 31, 9, 46, 14, 325184), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:46:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 14, 325184), True, datetime.datetime(2020, 3, 31, 9, 46, 14, 325184), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:46:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:46:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:46:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:14,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:24 UTC)" executed successfully 2020-03-31 09:46:14,446 [527] [DEBUG] [app] Starting request: urn:request:f32bc471-d67e-4ee1-b7b4-c145bce3ca55 (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:14,446 [527] [DEBUG] [app] Starting request: urn:request:f32bc471-d67e-4ee1-b7b4-c145bce3ca55 (/health/instance) 2020-03-31 09:46:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:14,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:14,454 [538] [DEBUG] [app] Starting request: urn:request:4c1d369b-3691-4729-973d-63511a45236b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:14,454 [538] [DEBUG] [app] Starting request: urn:request:4c1d369b-3691-4729-973d-63511a45236b (/v1/_internal_ping) 2020-03-31 09:46:14,455 [538] [DEBUG] [app] Ending request: urn:request:4c1d369b-3691-4729-973d-63511a45236b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:14,455 [538] [DEBUG] [app] Ending request: urn:request:4c1d369b-3691-4729-973d-63511a45236b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:46:14,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:14,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:14,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:14,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:14,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:14,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:14,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:14,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:14,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:14,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:14,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:14,484 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:14,485 [527] [DEBUG] [app] Starting request: urn:request:aee02fc2-ee50-40ef-9d51-9a338630bef3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:14,485 [527] [DEBUG] [app] Starting request: urn:request:aee02fc2-ee50-40ef-9d51-9a338630bef3 (/_internal_ping) 2020-03-31 09:46:14,486 [527] [DEBUG] [app] Ending request: urn:request:aee02fc2-ee50-40ef-9d51-9a338630bef3 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:14,486 [527] [DEBUG] [app] Ending request: urn:request:aee02fc2-ee50-40ef-9d51-9a338630bef3 (/_internal_ping) 2020-03-31 09:46:14,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:46:14,487 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:14,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:46:14,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:14,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:14,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:14,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:46:14,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:46:14,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:46:14,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:46:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:14,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:14,499 [537] [DEBUG] [app] Starting request: urn:request:199b573c-68df-4ecf-8cc3-f5f9fc1910b4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:14,499 [537] [DEBUG] [app] Starting request: urn:request:199b573c-68df-4ecf-8cc3-f5f9fc1910b4 (/c1/_internal_ping) 2020-03-31 09:46:14,500 [537] [DEBUG] [app] Ending request: urn:request:199b573c-68df-4ecf-8cc3-f5f9fc1910b4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:14,500 [537] [DEBUG] [app] Ending request: urn:request:199b573c-68df-4ecf-8cc3-f5f9fc1910b4 (/c1/_internal_ping) 2020-03-31 09:46:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:14,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:46:14,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:14,502 [527] [DEBUG] [app] Ending request: urn:request:f32bc471-d67e-4ee1-b7b4-c145bce3ca55 (/health/instance) 2020-03-31 09:46:14,502 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:14,502 [527] [DEBUG] [app] Ending request: urn:request:f32bc471-d67e-4ee1-b7b4-c145bce3ca55 (/health/instance) 2020-03-31 09:46:14,502 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:14,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:46:14,503 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:46:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:46:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:45.064514+00:00 (in 29.999205 seconds) 2020-03-31 09:46:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:45 UTC)" (scheduled at 2020-03-31 09:46:15.064514+00:00) gcworker stdout | 2020-03-31 09:46:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:45.064514+00:00 (in 29.999205 seconds) 2020-03-31 09:46:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:45 UTC)" (scheduled at 2020-03-31 09:46:15.064514+00:00) 2020-03-31 09:46:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438375, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:46:15,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438375, 1209600, 500, 1, 0]) 2020-03-31 09:46:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:46:15,082 [174] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2020-03-31 09:46:15,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:46:15,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:46:15,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:45 UTC)" executed successfully 2020-03-31 09:46:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:46:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:26.924497+00:00 (in 9.999226 seconds) 2020-03-31 09:46:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:26 UTC)" (scheduled at 2020-03-31 09:46:16.924497+00:00) 2020-03-31 09:46:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:46:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:26.924497+00:00 (in 9.999226 seconds) 2020-03-31 09:46:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:26 UTC)" (scheduled at 2020-03-31 09:46:16.924497+00:00) 2020-03-31 09:46:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 16, 925947), True, datetime.datetime(2020, 3, 31, 9, 46, 16, 925947), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:46:16,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 16, 925947), True, datetime.datetime(2020, 3, 31, 9, 46, 16, 925947), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:46:16,941 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:46:16,941 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:16,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:46:16,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:16,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:26 UTC)" executed successfully 2020-03-31 09:46:18,273 [518] [DEBUG] [app] Starting request: urn:request:be9c72bd-e171-4529-a6d6-9c157c544389 (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:18,273 [518] [DEBUG] [app] Starting request: urn:request:be9c72bd-e171-4529-a6d6-9c157c544389 (/health/instance) 2020-03-31 09:46:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:18,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:18,281 [556] [DEBUG] [app] Starting request: urn:request:67a78e0d-ad6c-4d2f-8643-bcf25392ebf2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:18,281 [556] [DEBUG] [app] Starting request: urn:request:67a78e0d-ad6c-4d2f-8643-bcf25392ebf2 (/v1/_internal_ping) 2020-03-31 09:46:18,282 [556] [DEBUG] [app] Ending request: urn:request:67a78e0d-ad6c-4d2f-8643-bcf25392ebf2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:18,282 [556] [DEBUG] [app] Ending request: urn:request:67a78e0d-ad6c-4d2f-8643-bcf25392ebf2 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:18,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:46:18,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:18,282 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:18,283 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:18,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:18,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:18,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:18,286 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:18,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:18,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:18,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:18,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:18,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:18,311 [525] [DEBUG] [app] Starting request: urn:request:9fc1b3cc-0511-4f26-9ce2-fcb911c2ab33 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:18,311 [525] [DEBUG] [app] Starting request: urn:request:9fc1b3cc-0511-4f26-9ce2-fcb911c2ab33 (/_internal_ping) 2020-03-31 09:46:18,312 [525] [DEBUG] [app] Ending request: urn:request:9fc1b3cc-0511-4f26-9ce2-fcb911c2ab33 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:18,312 [525] [DEBUG] [app] Ending request: urn:request:9fc1b3cc-0511-4f26-9ce2-fcb911c2ab33 (/_internal_ping) 2020-03-31 09:46:18,313 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:18,313 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:46:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:46:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:46:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:46:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:46:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:18,326 [541] [DEBUG] [app] Starting request: urn:request:71efb811-9e4c-409b-9133-4dc544c5a079 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:18,326 [541] [DEBUG] [app] Starting request: urn:request:71efb811-9e4c-409b-9133-4dc544c5a079 (/c1/_internal_ping) 2020-03-31 09:46:18,327 [541] [DEBUG] [app] Ending request: urn:request:71efb811-9e4c-409b-9133-4dc544c5a079 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:18,327 [541] [DEBUG] [app] Ending request: urn:request:71efb811-9e4c-409b-9133-4dc544c5a079 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:18,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:46:18,328 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:18,330 [518] [DEBUG] [app] Ending request: urn:request:be9c72bd-e171-4529-a6d6-9c157c544389 (/health/instance) 2020-03-31 09:46:18,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:18,330 [518] [DEBUG] [app] Ending request: urn:request:be9c72bd-e171-4529-a6d6-9c157c544389 (/health/instance) 2020-03-31 09:46:18,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:46:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:46:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:46:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:31.440707+00:00 (in 9.998922 seconds) 2020-03-31 09:46:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:31 UTC)" (scheduled at 2020-03-31 09:46:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:46:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:31.440707+00:00 (in 9.998922 seconds) 2020-03-31 09:46:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:31 UTC)" (scheduled at 2020-03-31 09:46:21.440707+00:00) 2020-03-31 09:46:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:46:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:46:21,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:46:21,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:46:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:46:21,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:21,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:31 UTC)" executed successfully 2020-03-31 09:46:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:46:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:52.113221+00:00 (in 29.999228 seconds) 2020-03-31 09:46:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:52 UTC)" (scheduled at 2020-03-31 09:46:22.113221+00:00) securityworker stdout | 2020-03-31 09:46:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:52.113221+00:00 (in 29.999228 seconds) 2020-03-31 09:46:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:52 UTC)" (scheduled at 2020-03-31 09:46:22.113221+00:00) 2020-03-31 09:46:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:46:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:46:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:46:22,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:46:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:46:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:46:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:46:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:46:52 UTC)" executed successfully 2020-03-31 09:46:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:46:24,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:34.323352+00:00 (in 9.999255 seconds) 2020-03-31 09:46:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:34 UTC)" (scheduled at 2020-03-31 09:46:24.323352+00:00) 2020-03-31 09:46:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:46:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:34.323352+00:00 (in 9.999255 seconds) 2020-03-31 09:46:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:34 UTC)" (scheduled at 2020-03-31 09:46:24.323352+00:00) 2020-03-31 09:46:24,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 24, 324931), True, datetime.datetime(2020, 3, 31, 9, 46, 24, 324931), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:46:24,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 24, 324931), True, datetime.datetime(2020, 3, 31, 9, 46, 24, 324931), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:46:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:46:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:46:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:24,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:34 UTC)" executed successfully 2020-03-31 09:46:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:46:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:24,411 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:24 UTC)" (scheduled at 2020-03-31 09:46:24.410927+00:00) 2020-03-31 09:46:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:46:24,411 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:24 UTC)" (scheduled at 2020-03-31 09:46:24.410927+00:00) 2020-03-31 09:46:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:24 UTC)" executed successfully 2020-03-31 09:46:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:33.330112+00:00 (in 8.918502 seconds) security_notification_worker stdout | 2020-03-31 09:46:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:33.330112+00:00 (in 8.918502 seconds) 2020-03-31 09:46:24,446 [526] [DEBUG] [app] Starting request: urn:request:d43f5b1b-8ac0-487f-a523-87eb38016eba (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:24,446 [526] [DEBUG] [app] Starting request: urn:request:d43f5b1b-8ac0-487f-a523-87eb38016eba (/health/instance) 2020-03-31 09:46:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:24,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:24,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:24,451 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:24,453 [546] [DEBUG] [app] Starting request: urn:request:96f93f72-c242-4a74-b22f-3bd436e5b4bc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:24,453 [546] [DEBUG] [app] Starting request: urn:request:96f93f72-c242-4a74-b22f-3bd436e5b4bc (/v1/_internal_ping) 2020-03-31 09:46:24,454 [546] [DEBUG] [app] Ending request: urn:request:96f93f72-c242-4a74-b22f-3bd436e5b4bc (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:24,454 [546] [DEBUG] [app] Ending request: urn:request:96f93f72-c242-4a74-b22f-3bd436e5b4bc (/v1/_internal_ping) 2020-03-31 09:46:24,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:24,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:46:24,455 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:24,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:24,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:46:24,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:46:24,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:46:24,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:24,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:24,459 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:24,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:24,460 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:24,465 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:24,465 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:24,478 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:24,478 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:24,481 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:24,481 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:24,483 [518] [DEBUG] [app] Starting request: urn:request:d4ed056c-d5c0-41ef-9604-bfa6819c95a9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:24,483 [518] [DEBUG] [app] Starting request: urn:request:d4ed056c-d5c0-41ef-9604-bfa6819c95a9 (/_internal_ping) 2020-03-31 09:46:24,484 [518] [DEBUG] [app] Ending request: urn:request:d4ed056c-d5c0-41ef-9604-bfa6819c95a9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:24,484 [518] [DEBUG] [app] Ending request: urn:request:d4ed056c-d5c0-41ef-9604-bfa6819c95a9 (/_internal_ping) 2020-03-31 09:46:24,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:24,485 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:24,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:24,485 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:46:24,488 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:24,488 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:24,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:46:24,489 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:46:24,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:24,492 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:46:24,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:46:24,493 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:46:24,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:24,496 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:24,497 [534] [DEBUG] [app] Starting request: urn:request:3c0acb3e-a3b6-4b25-87b2-b771fce5c646 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:24,497 [534] [DEBUG] [app] Starting request: urn:request:3c0acb3e-a3b6-4b25-87b2-b771fce5c646 (/c1/_internal_ping) 2020-03-31 09:46:24,498 [534] [DEBUG] [app] Ending request: urn:request:3c0acb3e-a3b6-4b25-87b2-b771fce5c646 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:24,498 [534] [DEBUG] [app] Ending request: urn:request:3c0acb3e-a3b6-4b25-87b2-b771fce5c646 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:24,498 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:24,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:46:24,498 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:24,499 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:24,500 [526] [DEBUG] [app] Ending request: urn:request:d43f5b1b-8ac0-487f-a523-87eb38016eba (/health/instance) 2020-03-31 09:46:24,501 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:24,500 [526] [DEBUG] [app] Ending request: urn:request:d43f5b1b-8ac0-487f-a523-87eb38016eba (/health/instance) 2020-03-31 09:46:24,501 [526] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:46:24,501 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:46:24,501 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:46:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:36.924497+00:00 (in 9.999219 seconds) 2020-03-31 09:46:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:36 UTC)" (scheduled at 2020-03-31 09:46:26.924497+00:00) 2020-03-31 09:46:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:46:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:36.924497+00:00 (in 9.999219 seconds) 2020-03-31 09:46:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 26, 926166), True, datetime.datetime(2020, 3, 31, 9, 46, 26, 926166), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:46:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:36 UTC)" (scheduled at 2020-03-31 09:46:26.924497+00:00) 2020-03-31 09:46:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:46:26,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 26, 926166), True, datetime.datetime(2020, 3, 31, 9, 46, 26, 926166), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:46:26,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:46:26,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:46:26,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:26,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:36 UTC)" executed successfully 2020-03-31 09:46:28,274 [518] [DEBUG] [app] Starting request: urn:request:ea7baff4-c0bb-4e6d-aab5-0a3f4e0403e6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:28,274 [518] [DEBUG] [app] Starting request: urn:request:ea7baff4-c0bb-4e6d-aab5-0a3f4e0403e6 (/health/instance) 2020-03-31 09:46:28,277 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:28,277 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:28,283 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:28,283 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:28,285 [552] [DEBUG] [app] Starting request: urn:request:73f00827-5a03-43c7-b920-3a5d21095b88 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:28,285 [552] [DEBUG] [app] Starting request: urn:request:73f00827-5a03-43c7-b920-3a5d21095b88 (/v1/_internal_ping) 2020-03-31 09:46:28,286 [552] [DEBUG] [app] Ending request: urn:request:73f00827-5a03-43c7-b920-3a5d21095b88 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:28,286 [552] [DEBUG] [app] Ending request: urn:request:73f00827-5a03-43c7-b920-3a5d21095b88 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:28,287 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:28,287 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:28,288 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:28,289 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:46:28,287 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:28,287 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:28,290 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:28,291 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:46:28,288 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:46:28,289 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:28,290 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:28,291 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:28,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:28,313 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:28,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:28,320 [518] [DEBUG] [app] Starting request: urn:request:60c3a363-beff-4e19-a8d6-660fd4bd417e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:28,320 [518] [DEBUG] [app] Starting request: urn:request:60c3a363-beff-4e19-a8d6-660fd4bd417e (/_internal_ping) 2020-03-31 09:46:28,321 [518] [DEBUG] [app] Ending request: urn:request:60c3a363-beff-4e19-a8d6-660fd4bd417e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:28,321 [518] [DEBUG] [app] Ending request: urn:request:60c3a363-beff-4e19-a8d6-660fd4bd417e (/_internal_ping) 2020-03-31 09:46:28,322 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:46:28,322 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:46:28,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:28,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:46:28,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:46:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:28,331 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:46:28,332 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:46:28,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:28,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:28,337 [534] [DEBUG] [app] Starting request: urn:request:af129689-a372-4c8f-93b5-a9c8100bdd5a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:28,337 [534] [DEBUG] [app] Starting request: urn:request:af129689-a372-4c8f-93b5-a9c8100bdd5a (/c1/_internal_ping) 2020-03-31 09:46:28,338 [534] [DEBUG] [app] Ending request: urn:request:af129689-a372-4c8f-93b5-a9c8100bdd5a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:28,338 [534] [DEBUG] [app] Ending request: urn:request:af129689-a372-4c8f-93b5-a9c8100bdd5a (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:28,339 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:28,339 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:46:28,339 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:28,341 [518] [DEBUG] [app] Ending request: urn:request:ea7baff4-c0bb-4e6d-aab5-0a3f4e0403e6 (/health/instance) 2020-03-31 09:46:28,342 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:46:28,339 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:28,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:46:28,341 [518] [DEBUG] [app] Ending request: urn:request:ea7baff4-c0bb-4e6d-aab5-0a3f4e0403e6 (/health/instance) 2020-03-31 09:46:28,342 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:28,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:46:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:46:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:00.337699+00:00 (in 29.998950 seconds) 2020-03-31 09:46:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:00 UTC)" (scheduled at 2020-03-31 09:46:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:46:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:00.337699+00:00 (in 29.998950 seconds) 2020-03-31 09:46:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:00 UTC)" (scheduled at 2020-03-31 09:46:30.337699+00:00) 2020-03-31 09:46:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 46, 30, 339557), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:46:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 46, 30, 339557), False, 50, 1, 0]) 2020-03-31 09:46:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:46:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:46:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:46:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:30,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:00 UTC)" executed successfully 2020-03-31 09:46:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:46:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:41.440707+00:00 (in 9.996587 seconds) 2020-03-31 09:46:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:41 UTC)" (scheduled at 2020-03-31 09:46:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:46:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:41.440707+00:00 (in 9.996587 seconds) 2020-03-31 09:46:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:41 UTC)" (scheduled at 2020-03-31 09:46:31.440707+00:00) 2020-03-31 09:46:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:46:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:46:31,467 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:46:31,467 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:46:31,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:31,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:46:31,467 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:31,467 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:41 UTC)" executed successfully 2020-03-31 09:46:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:46:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:03.330112+00:00 (in 29.998356 seconds) 2020-03-31 09:46:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:03 UTC)" (scheduled at 2020-03-31 09:46:33.330112+00:00) 2020-03-31 09:46:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:46:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:03.330112+00:00 (in 29.998356 seconds) 2020-03-31 09:46:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:03 UTC)" (scheduled at 2020-03-31 09:46:33.330112+00:00) 2020-03-31 09:46:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 33, 332769), True, datetime.datetime(2020, 3, 31, 9, 46, 33, 332769), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:46:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 33, 332769), True, datetime.datetime(2020, 3, 31, 9, 46, 33, 332769), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:46:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:46:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:46:33,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:33,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:03 UTC)" executed successfully 2020-03-31 09:46:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:46:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:43.539999+00:00 (in 10.171374 seconds) 2020-03-31 09:46:33,369 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:33 UTC)" (scheduled at 2020-03-31 09:46:33.366791+00:00) 2020-03-31 09:46:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:46:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:43.539999+00:00 (in 10.171374 seconds) 2020-03-31 09:46:33,369 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:33 UTC)" (scheduled at 2020-03-31 09:46:33.366791+00:00) 2020-03-31 09:46:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:33,372 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 33, 369584), True, datetime.datetime(2020, 3, 31, 9, 46, 33, 369584), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:46:33,372 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 33, 369584), True, datetime.datetime(2020, 3, 31, 9, 46, 33, 369584), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:46:33,390 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:46:33,390 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:33,390 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:33,390 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:46:33,390 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:33,390 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:33 UTC)" executed successfully 2020-03-31 09:46:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:46:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:44.323352+00:00 (in 9.999124 seconds) 2020-03-31 09:46:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:44 UTC)" (scheduled at 2020-03-31 09:46:34.323352+00:00) 2020-03-31 09:46:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:46:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:44.323352+00:00 (in 9.999124 seconds) 2020-03-31 09:46:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:44 UTC)" (scheduled at 2020-03-31 09:46:34.323352+00:00) 2020-03-31 09:46:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 34, 325295), True, datetime.datetime(2020, 3, 31, 9, 46, 34, 325295), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:46:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 34, 325295), True, datetime.datetime(2020, 3, 31, 9, 46, 34, 325295), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:46:34,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:46:34,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:46:34,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:34,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:44 UTC)" executed successfully 2020-03-31 09:46:34,446 [526] [DEBUG] [app] Starting request: urn:request:ba32012c-72e7-4edf-ae2f-c550a9496dcc (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:34,446 [526] [DEBUG] [app] Starting request: urn:request:ba32012c-72e7-4edf-ae2f-c550a9496dcc (/health/instance) 2020-03-31 09:46:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:34,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:34,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:34,453 [555] [DEBUG] [app] Starting request: urn:request:3b58d89e-747f-451e-9cd2-103958f73fd5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:34,453 [555] [DEBUG] [app] Starting request: urn:request:3b58d89e-747f-451e-9cd2-103958f73fd5 (/v1/_internal_ping) 2020-03-31 09:46:34,455 [555] [DEBUG] [app] Ending request: urn:request:3b58d89e-747f-451e-9cd2-103958f73fd5 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:34,455 [555] [DEBUG] [app] Ending request: urn:request:3b58d89e-747f-451e-9cd2-103958f73fd5 (/v1/_internal_ping) 2020-03-31 09:46:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:46:34,456 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:34,456 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:46:34,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:34,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:34,458 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:34,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:34,460 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:34,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:34,460 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:34,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:34,467 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:34,481 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:34,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:34,484 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:34,485 [526] [DEBUG] [app] Starting request: urn:request:d00e83b1-21bd-4cb2-b422-65ca35dd9dbc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:34,485 [526] [DEBUG] [app] Starting request: urn:request:d00e83b1-21bd-4cb2-b422-65ca35dd9dbc (/_internal_ping) 2020-03-31 09:46:34,486 [526] [DEBUG] [app] Ending request: urn:request:d00e83b1-21bd-4cb2-b422-65ca35dd9dbc (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:34,486 [526] [DEBUG] [app] Ending request: urn:request:d00e83b1-21bd-4cb2-b422-65ca35dd9dbc (/_internal_ping) 2020-03-31 09:46:34,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:34,487 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:34,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:34,487 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:34,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:34,490 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:34,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:46:34,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:46:34,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:34,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:46:34,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:46:34,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:46:34,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:34,498 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:34,499 [534] [DEBUG] [app] Starting request: urn:request:86589d14-12ca-4a2a-b886-6c80694e0fd2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:34,499 [534] [DEBUG] [app] Starting request: urn:request:86589d14-12ca-4a2a-b886-6c80694e0fd2 (/c1/_internal_ping) 2020-03-31 09:46:34,500 [534] [DEBUG] [app] Ending request: urn:request:86589d14-12ca-4a2a-b886-6c80694e0fd2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:34,500 [534] [DEBUG] [app] Ending request: urn:request:86589d14-12ca-4a2a-b886-6c80694e0fd2 (/c1/_internal_ping) 2020-03-31 09:46:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:34,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:34,501 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:46:34,501 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:34,502 [526] [DEBUG] [app] Ending request: urn:request:ba32012c-72e7-4edf-ae2f-c550a9496dcc (/health/instance) 2020-03-31 09:46:34,503 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:34,502 [526] [DEBUG] [app] Ending request: urn:request:ba32012c-72e7-4edf-ae2f-c550a9496dcc (/health/instance) 2020-03-31 09:46:34,503 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:34,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:46:34,503 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:46:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:46:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:38.010934+00:00 (in 1.085609 seconds) 2020-03-31 09:46:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:46 UTC)" (scheduled at 2020-03-31 09:46:36.924497+00:00) 2020-03-31 09:46:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:46:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:38.010934+00:00 (in 1.085609 seconds) 2020-03-31 09:46:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:46 UTC)" (scheduled at 2020-03-31 09:46:36.924497+00:00) 2020-03-31 09:46:36,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 36, 926148), True, datetime.datetime(2020, 3, 31, 9, 46, 36, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:46:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 36, 926148), True, datetime.datetime(2020, 3, 31, 9, 46, 36, 926148), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:46:36,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:46:36,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:46:36,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:36,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:46 UTC)" executed successfully 2020-03-31 09:46:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:46:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:46.924497+00:00 (in 8.912776 seconds) 2020-03-31 09:46:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:38 UTC)" (scheduled at 2020-03-31 09:46:38.010934+00:00) 2020-03-31 09:46:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. storagereplication stdout | 2020-03-31 09:46:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:46.924497+00:00 (in 8.912776 seconds) 2020-03-31 09:46:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:38 UTC)" (scheduled at 2020-03-31 09:46:38.010934+00:00) 2020-03-31 09:46:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:46:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:38 UTC)" executed successfully 2020-03-31 09:46:38,273 [527] [DEBUG] [app] Starting request: urn:request:b4c85c9f-5864-4c43-b6ae-f119f07d6223 (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:38,273 [527] [DEBUG] [app] Starting request: urn:request:b4c85c9f-5864-4c43-b6ae-f119f07d6223 (/health/instance) 2020-03-31 09:46:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:38,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:38,281 [555] [DEBUG] [app] Starting request: urn:request:fc47ffb3-6f5e-4d4e-92d1-19ad64ce2ed4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:38,281 [555] [DEBUG] [app] Starting request: urn:request:fc47ffb3-6f5e-4d4e-92d1-19ad64ce2ed4 (/v1/_internal_ping) 2020-03-31 09:46:38,282 [555] [DEBUG] [app] Ending request: urn:request:fc47ffb3-6f5e-4d4e-92d1-19ad64ce2ed4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:38,282 [555] [DEBUG] [app] Ending request: urn:request:fc47ffb3-6f5e-4d4e-92d1-19ad64ce2ed4 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:46:38,282 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:38,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:38,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:38,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:38,285 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:38,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:38,287 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:38,287 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:38,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:38,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:38,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:38,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:38,313 [526] [DEBUG] [app] Starting request: urn:request:652cc905-2a1c-4270-892f-49b6e9ff06e9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:38,313 [526] [DEBUG] [app] Starting request: urn:request:652cc905-2a1c-4270-892f-49b6e9ff06e9 (/_internal_ping) 2020-03-31 09:46:38,314 [526] [DEBUG] [app] Ending request: urn:request:652cc905-2a1c-4270-892f-49b6e9ff06e9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:38,314 [526] [DEBUG] [app] Ending request: urn:request:652cc905-2a1c-4270-892f-49b6e9ff06e9 (/_internal_ping) 2020-03-31 09:46:38,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:38,315 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:46:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:38,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:46:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:38,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:46:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:46:38,324 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:46:38,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:38,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:38,329 [534] [DEBUG] [app] Starting request: urn:request:a3567267-07f3-43fa-858b-01da7b9c5bd9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:38,329 [534] [DEBUG] [app] Starting request: urn:request:a3567267-07f3-43fa-858b-01da7b9c5bd9 (/c1/_internal_ping) 2020-03-31 09:46:38,329 [534] [DEBUG] [app] Ending request: urn:request:a3567267-07f3-43fa-858b-01da7b9c5bd9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:38,329 [534] [DEBUG] [app] Ending request: urn:request:a3567267-07f3-43fa-858b-01da7b9c5bd9 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:38,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:38,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:46:38,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:38,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:38,332 [527] [DEBUG] [app] Ending request: urn:request:b4c85c9f-5864-4c43-b6ae-f119f07d6223 (/health/instance) 2020-03-31 09:46:38,332 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:38,332 [527] [DEBUG] [app] Ending request: urn:request:b4c85c9f-5864-4c43-b6ae-f119f07d6223 (/health/instance) 2020-03-31 09:46:38,332 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:46:38,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:46:38,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:46:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:46:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:51.440707+00:00 (in 9.999088 seconds) 2020-03-31 09:46:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:51 UTC)" (scheduled at 2020-03-31 09:46:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:46:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:51.440707+00:00 (in 9.999088 seconds) 2020-03-31 09:46:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:51 UTC)" (scheduled at 2020-03-31 09:46:41.440707+00:00) 2020-03-31 09:46:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:46:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:46:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:46:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:46:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:46:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:51 UTC)" executed successfully 2020-03-31 09:46:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:46:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:33.366791+00:00 (in 49.825723 seconds) 2020-03-31 09:46:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:43 UTC)" (scheduled at 2020-03-31 09:46:43.539999+00:00) 2020-03-31 09:46:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:46:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:33.366791+00:00 (in 49.825723 seconds) 2020-03-31 09:46:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:43 UTC)" (scheduled at 2020-03-31 09:46:43.539999+00:00) 2020-03-31 09:46:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:43 UTC)" executed successfully 2020-03-31 09:46:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:46:44,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:49.498966+00:00 (in 5.173254 seconds) 2020-03-31 09:46:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:54 UTC)" (scheduled at 2020-03-31 09:46:44.323352+00:00) 2020-03-31 09:46:44,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:46:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:49.498966+00:00 (in 5.173254 seconds) 2020-03-31 09:46:44,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:54 UTC)" (scheduled at 2020-03-31 09:46:44.323352+00:00) 2020-03-31 09:46:44,326 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:44,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 44, 326632), True, datetime.datetime(2020, 3, 31, 9, 46, 44, 326632), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:46:44,329 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 44, 326632), True, datetime.datetime(2020, 3, 31, 9, 46, 44, 326632), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:46:44,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:46:44,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:46:44,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:44,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:54 UTC)" executed successfully 2020-03-31 09:46:44,446 [518] [DEBUG] [app] Starting request: urn:request:fa946fd4-219d-4f88-bcd9-91f240b04199 (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:44,446 [518] [DEBUG] [app] Starting request: urn:request:fa946fd4-219d-4f88-bcd9-91f240b04199 (/health/instance) 2020-03-31 09:46:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:44,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:44,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:44,454 [553] [DEBUG] [app] Starting request: urn:request:bf00e3e1-ecfc-4e37-8022-285b92d7f9ef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:44,454 [553] [DEBUG] [app] Starting request: urn:request:bf00e3e1-ecfc-4e37-8022-285b92d7f9ef (/v1/_internal_ping) 2020-03-31 09:46:44,455 [553] [DEBUG] [app] Ending request: urn:request:bf00e3e1-ecfc-4e37-8022-285b92d7f9ef (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:44,455 [553] [DEBUG] [app] Ending request: urn:request:bf00e3e1-ecfc-4e37-8022-285b92d7f9ef (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:44,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:46:44,456 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:44,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:44,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:44,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:44,459 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:44,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:44,460 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:44,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:44,461 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:44,486 [518] [DEBUG] [app] Starting request: urn:request:f5757005-07e0-488c-85da-b75500b53a90 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:44,486 [518] [DEBUG] [app] Starting request: urn:request:f5757005-07e0-488c-85da-b75500b53a90 (/_internal_ping) 2020-03-31 09:46:44,487 [518] [DEBUG] [app] Ending request: urn:request:f5757005-07e0-488c-85da-b75500b53a90 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:44,487 [518] [DEBUG] [app] Ending request: urn:request:f5757005-07e0-488c-85da-b75500b53a90 (/_internal_ping) 2020-03-31 09:46:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:46:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:46:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:46:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:46:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:46:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:44,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:44,499 [534] [DEBUG] [app] Starting request: urn:request:fae02e68-ca4f-4eeb-8514-70d8d98ec89b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:44,499 [534] [DEBUG] [app] Starting request: urn:request:fae02e68-ca4f-4eeb-8514-70d8d98ec89b (/c1/_internal_ping) 2020-03-31 09:46:44,500 [534] [DEBUG] [app] Ending request: urn:request:fae02e68-ca4f-4eeb-8514-70d8d98ec89b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:44,500 [534] [DEBUG] [app] Ending request: urn:request:fae02e68-ca4f-4eeb-8514-70d8d98ec89b (/c1/_internal_ping) 2020-03-31 09:46:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:44,500 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:44,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:44,502 [518] [DEBUG] [app] Ending request: urn:request:fa946fd4-219d-4f88-bcd9-91f240b04199 (/health/instance) 2020-03-31 09:46:44,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:46:44,500 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:44,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:46:44,502 [518] [DEBUG] [app] Ending request: urn:request:fa946fd4-219d-4f88-bcd9-91f240b04199 (/health/instance) 2020-03-31 09:46:44,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:44,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:46:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:46:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:15.064514+00:00 (in 29.999292 seconds) 2020-03-31 09:46:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:15 UTC)" (scheduled at 2020-03-31 09:46:45.064514+00:00) gcworker stdout | 2020-03-31 09:46:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:15.064514+00:00 (in 29.999292 seconds) 2020-03-31 09:46:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:15 UTC)" (scheduled at 2020-03-31 09:46:45.064514+00:00) 2020-03-31 09:46:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438405, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:46:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438405, 1209600, 500, 1, 0]) 2020-03-31 09:46:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:46:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:46:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:46:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:45,082 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:15 UTC)" executed successfully 2020-03-31 09:46:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:46:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:56.924497+00:00 (in 9.999149 seconds) 2020-03-31 09:46:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:56 UTC)" (scheduled at 2020-03-31 09:46:46.924497+00:00) 2020-03-31 09:46:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:46:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:56.924497+00:00 (in 9.999149 seconds) 2020-03-31 09:46:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:56 UTC)" (scheduled at 2020-03-31 09:46:46.924497+00:00) 2020-03-31 09:46:46,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 46, 926063), True, datetime.datetime(2020, 3, 31, 9, 46, 46, 926063), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:46:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 46, 926063), True, datetime.datetime(2020, 3, 31, 9, 46, 46, 926063), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:46:46,941 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:46:46,941 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:46,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:46,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:46:46,941 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:46,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:56 UTC)" executed successfully 2020-03-31 09:46:48,273 [527] [DEBUG] [app] Starting request: urn:request:a2823c2e-67eb-4bb8-95be-176f2c1468ca (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:48,273 [527] [DEBUG] [app] Starting request: urn:request:a2823c2e-67eb-4bb8-95be-176f2c1468ca (/health/instance) 2020-03-31 09:46:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:48,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:48,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:48,280 [554] [DEBUG] [app] Starting request: urn:request:a26ace6d-46a5-49e1-ab36-6f6e2930f522 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:48,280 [554] [DEBUG] [app] Starting request: urn:request:a26ace6d-46a5-49e1-ab36-6f6e2930f522 (/v1/_internal_ping) 2020-03-31 09:46:48,282 [554] [DEBUG] [app] Ending request: urn:request:a26ace6d-46a5-49e1-ab36-6f6e2930f522 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:48,282 [554] [DEBUG] [app] Ending request: urn:request:a26ace6d-46a5-49e1-ab36-6f6e2930f522 (/v1/_internal_ping) 2020-03-31 09:46:48,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:46:48,283 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:48,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:46:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:48,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:48,285 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:48,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:48,287 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:48,288 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:48,288 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:48,293 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:48,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:48,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:48,312 [518] [DEBUG] [app] Starting request: urn:request:5de7628e-d64e-4448-8650-f9d06e1d45b0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:48,312 [518] [DEBUG] [app] Starting request: urn:request:5de7628e-d64e-4448-8650-f9d06e1d45b0 (/_internal_ping) 2020-03-31 09:46:48,313 [518] [DEBUG] [app] Ending request: urn:request:5de7628e-d64e-4448-8650-f9d06e1d45b0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:48,313 [518] [DEBUG] [app] Ending request: urn:request:5de7628e-d64e-4448-8650-f9d06e1d45b0 (/_internal_ping) 2020-03-31 09:46:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:46:48,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:48,314 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:46:48,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:48,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:46:48,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:48,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:46:48,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:46:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:48,326 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:48,328 [543] [DEBUG] [app] Starting request: urn:request:9a4c993e-c2ae-42e9-a8ec-234efe28d337 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:48,328 [543] [DEBUG] [app] Starting request: urn:request:9a4c993e-c2ae-42e9-a8ec-234efe28d337 (/c1/_internal_ping) 2020-03-31 09:46:48,328 [543] [DEBUG] [app] Ending request: urn:request:9a4c993e-c2ae-42e9-a8ec-234efe28d337 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:48,328 [543] [DEBUG] [app] Ending request: urn:request:9a4c993e-c2ae-42e9-a8ec-234efe28d337 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:48,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:48,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:48,329 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:46:48,329 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:48,331 [527] [DEBUG] [app] Ending request: urn:request:a2823c2e-67eb-4bb8-95be-176f2c1468ca (/health/instance) 2020-03-31 09:46:48,331 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:48,331 [527] [DEBUG] [app] Ending request: urn:request:a2823c2e-67eb-4bb8-95be-176f2c1468ca (/health/instance) 2020-03-31 09:46:48,331 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:48,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:46:48,332 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:46:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:46:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:54.323352+00:00 (in 4.823615 seconds) 2020-03-31 09:46:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:49 UTC)" (scheduled at 2020-03-31 09:46:49.498966+00:00) 2020-03-31 09:46:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:46:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:54.323352+00:00 (in 4.823615 seconds) 2020-03-31 09:46:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:49 UTC)" (scheduled at 2020-03-31 09:46:49.498966+00:00) 2020-03-31 09:46:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:49 UTC)" executed successfully 2020-03-31 09:46:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:46:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:59.560326+00:00 (in 8.448671 seconds) 2020-03-31 09:46:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:51 UTC)" (scheduled at 2020-03-31 09:46:51.110610+00:00) 2020-03-31 09:46:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:46:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:46:59.560326+00:00 (in 8.448671 seconds) 2020-03-31 09:46:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:51 UTC)" (scheduled at 2020-03-31 09:46:51.110610+00:00) 2020-03-31 09:46:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 51, 112330), True, datetime.datetime(2020, 3, 31, 9, 46, 51, 112330), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:46:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 51, 112330), True, datetime.datetime(2020, 3, 31, 9, 46, 51, 112330), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:46:51,128 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:46:51,128 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:46:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:51 UTC)" executed successfully 2020-03-31 09:46:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:46:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:01.440707+00:00 (in 9.998525 seconds) 2020-03-31 09:46:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:01 UTC)" (scheduled at 2020-03-31 09:46:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:46:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:01.440707+00:00 (in 9.998525 seconds) 2020-03-31 09:46:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:01 UTC)" (scheduled at 2020-03-31 09:46:51.440707+00:00) 2020-03-31 09:46:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:46:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:46:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:46:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:46:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:46:51,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:51,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:01 UTC)" executed successfully 2020-03-31 09:46:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:46:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:22.113221+00:00 (in 29.999250 seconds) 2020-03-31 09:46:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:22 UTC)" (scheduled at 2020-03-31 09:46:52.113221+00:00) securityworker stdout | 2020-03-31 09:46:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:22.113221+00:00 (in 29.999250 seconds) 2020-03-31 09:46:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:22 UTC)" (scheduled at 2020-03-31 09:46:52.113221+00:00) 2020-03-31 09:46:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:46:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:46:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:46:52,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:46:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:46:52,129 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:46:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:46:52,129 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:22 UTC)" executed successfully 2020-03-31 09:46:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:46:54,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:04.323352+00:00 (in 9.999203 seconds) 2020-03-31 09:46:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:04 UTC)" (scheduled at 2020-03-31 09:46:54.323352+00:00) 2020-03-31 09:46:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:46:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:04.323352+00:00 (in 9.999203 seconds) 2020-03-31 09:46:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:04 UTC)" (scheduled at 2020-03-31 09:46:54.323352+00:00) 2020-03-31 09:46:54,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 54, 325040), True, datetime.datetime(2020, 3, 31, 9, 46, 54, 325040), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:46:54,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 54, 325040), True, datetime.datetime(2020, 3, 31, 9, 46, 54, 325040), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:46:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:46:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:46:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:04 UTC)" executed successfully 2020-03-31 09:46:54,446 [518] [DEBUG] [app] Starting request: urn:request:976e7f91-68ba-4710-a9fe-dde001fac0db (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:54,446 [518] [DEBUG] [app] Starting request: urn:request:976e7f91-68ba-4710-a9fe-dde001fac0db (/health/instance) 2020-03-31 09:46:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:54,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:54,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:54,454 [546] [DEBUG] [app] Starting request: urn:request:97b24556-d9d7-4eb4-b1cc-416870c2cde1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:54,454 [546] [DEBUG] [app] Starting request: urn:request:97b24556-d9d7-4eb4-b1cc-416870c2cde1 (/v1/_internal_ping) 2020-03-31 09:46:54,456 [546] [DEBUG] [app] Ending request: urn:request:97b24556-d9d7-4eb4-b1cc-416870c2cde1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:54,456 [546] [DEBUG] [app] Ending request: urn:request:97b24556-d9d7-4eb4-b1cc-416870c2cde1 (/v1/_internal_ping) 2020-03-31 09:46:54,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:46:54,456 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:54,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:54,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:46:54,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:46:54,458 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:54,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:54,460 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:54,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:54,461 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:54,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:54,486 [526] [DEBUG] [app] Starting request: urn:request:8deeea6c-6150-4d82-ad91-9ea499198a50 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:54,486 [526] [DEBUG] [app] Starting request: urn:request:8deeea6c-6150-4d82-ad91-9ea499198a50 (/_internal_ping) 2020-03-31 09:46:54,487 [526] [DEBUG] [app] Ending request: urn:request:8deeea6c-6150-4d82-ad91-9ea499198a50 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:54,487 [526] [DEBUG] [app] Ending request: urn:request:8deeea6c-6150-4d82-ad91-9ea499198a50 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:46:54,488 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:54,488 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:46:54,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:46:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:54,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:46:54,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:46:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:54,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:54,501 [534] [DEBUG] [app] Starting request: urn:request:e4b4aa73-1114-43ec-b784-001368f5f000 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:54,501 [534] [DEBUG] [app] Starting request: urn:request:e4b4aa73-1114-43ec-b784-001368f5f000 (/c1/_internal_ping) 2020-03-31 09:46:54,502 [534] [DEBUG] [app] Ending request: urn:request:e4b4aa73-1114-43ec-b784-001368f5f000 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:54,502 [534] [DEBUG] [app] Ending request: urn:request:e4b4aa73-1114-43ec-b784-001368f5f000 (/c1/_internal_ping) 2020-03-31 09:46:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:54,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:54,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:46:54,504 [518] [DEBUG] [app] Ending request: urn:request:976e7f91-68ba-4710-a9fe-dde001fac0db (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:46:54,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:54,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:46:54,504 [518] [DEBUG] [app] Ending request: urn:request:976e7f91-68ba-4710-a9fe-dde001fac0db (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:54,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:54,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:46:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:56 UTC)" (scheduled at 2020-03-31 09:46:56.924497+00:00) 2020-03-31 09:46:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:46:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 56, 925838), True, datetime.datetime(2020, 3, 31, 9, 46, 56, 925838), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:46:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:03.010318+00:00 (in 6.085079 seconds) storagereplication stdout | 2020-03-31 09:46:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:46:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:46:56 UTC)" (scheduled at 2020-03-31 09:46:56.924497+00:00) 2020-03-31 09:46:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:46:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 46, 56, 925838), True, datetime.datetime(2020, 3, 31, 9, 46, 56, 925838), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:46:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:03.010318+00:00 (in 6.085079 seconds) 2020-03-31 09:46:56,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:46:56,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:46:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:46:56,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:56,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:06 UTC)" executed successfully 2020-03-31 09:46:58,273 [527] [DEBUG] [app] Starting request: urn:request:e8554554-a631-469d-a5e1-70acb6066df7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:46:58,273 [527] [DEBUG] [app] Starting request: urn:request:e8554554-a631-469d-a5e1-70acb6066df7 (/health/instance) 2020-03-31 09:46:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:58,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:58,281 [556] [DEBUG] [app] Starting request: urn:request:f3b362fd-f673-4b95-8e47-2bbdc35ac858 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:58,281 [556] [DEBUG] [app] Starting request: urn:request:f3b362fd-f673-4b95-8e47-2bbdc35ac858 (/v1/_internal_ping) 2020-03-31 09:46:58,282 [556] [DEBUG] [app] Ending request: urn:request:f3b362fd-f673-4b95-8e47-2bbdc35ac858 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:46:58,282 [556] [DEBUG] [app] Ending request: urn:request:f3b362fd-f673-4b95-8e47-2bbdc35ac858 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:58,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:46:58,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:46:58,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:46:58,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:46:58,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:46:58,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:46:58,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:46:58,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:46:58,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:46:58,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:46:58,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:46:58,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:46:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:46:58,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:46:58,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:58,313 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:58,314 [527] [DEBUG] [app] Starting request: urn:request:3c444113-e972-48be-8078-9076e996bd04 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:58,314 [527] [DEBUG] [app] Starting request: urn:request:3c444113-e972-48be-8078-9076e996bd04 (/_internal_ping) 2020-03-31 09:46:58,315 [527] [DEBUG] [app] Ending request: urn:request:3c444113-e972-48be-8078-9076e996bd04 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:46:58,315 [527] [DEBUG] [app] Ending request: urn:request:3c444113-e972-48be-8078-9076e996bd04 (/_internal_ping) 2020-03-31 09:46:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:46:58,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:46:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:46:58,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:58,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:46:58,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:58,320 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:46:58,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:58,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:46:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:46:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:46:58,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:46:58,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:46:58,330 [534] [DEBUG] [app] Starting request: urn:request:aeb323c4-f1ca-40f0-84bd-289faa5920c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:58,330 [534] [DEBUG] [app] Starting request: urn:request:aeb323c4-f1ca-40f0-84bd-289faa5920c9 (/c1/_internal_ping) 2020-03-31 09:46:58,331 [534] [DEBUG] [app] Ending request: urn:request:aeb323c4-f1ca-40f0-84bd-289faa5920c9 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:46:58,331 [534] [DEBUG] [app] Ending request: urn:request:aeb323c4-f1ca-40f0-84bd-289faa5920c9 (/c1/_internal_ping) 2020-03-31 09:46:58,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:46:58,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:46:58,332 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:46:58,332 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:46:58,333 [527] [DEBUG] [app] Ending request: urn:request:e8554554-a631-469d-a5e1-70acb6066df7 (/health/instance) 2020-03-31 09:46:58,333 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:46:58,334 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:46:58,333 [527] [DEBUG] [app] Ending request: urn:request:e8554554-a631-469d-a5e1-70acb6066df7 (/health/instance) 2020-03-31 09:46:58,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:46:58,334 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:46:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:46:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:46:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:46:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:46:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:51.110610+00:00 (in 51.549350 seconds) 2020-03-31 09:46:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:59 UTC)" (scheduled at 2020-03-31 09:46:59.560326+00:00) namespacegcworker stdout | 2020-03-31 09:46:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:51.110610+00:00 (in 51.549350 seconds) 2020-03-31 09:46:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:59 UTC)" (scheduled at 2020-03-31 09:46:59.560326+00:00) 2020-03-31 09:46:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:46:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:46:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:47:59 UTC)" executed successfully 2020-03-31 09:47:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:47:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:30.337699+00:00 (in 29.998926 seconds) buildlogsarchiver stdout | 2020-03-31 09:47:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:30.337699+00:00 (in 29.998926 seconds) 2020-03-31 09:47:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:30 UTC)" (scheduled at 2020-03-31 09:47:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:47:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:30 UTC)" (scheduled at 2020-03-31 09:47:00.337699+00:00) 2020-03-31 09:47:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 47, 0, 340089), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:47:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 47, 0, 340089), False, 50, 1, 0]) 2020-03-31 09:47:00,361 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:47:00,361 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:47:00,361 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:47:00,361 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:00,362 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:47:00,362 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:30 UTC)" executed successfully 2020-03-31 09:47:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:47:01,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:11.440707+00:00 (in 9.997755 seconds) 2020-03-31 09:47:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:11 UTC)" (scheduled at 2020-03-31 09:47:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:47:01,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:11.440707+00:00 (in 9.997755 seconds) 2020-03-31 09:47:01,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:11 UTC)" (scheduled at 2020-03-31 09:47:01.440707+00:00) 2020-03-31 09:47:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:47:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:47:01,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:47:01,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:47:01,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:01,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:47:01,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:01,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:11 UTC)" executed successfully 2020-03-31 09:47:03,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:47:03,013 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:03,014 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:03 UTC)" (scheduled at 2020-03-31 09:47:03.010318+00:00) storagereplication stdout | 2020-03-31 09:47:03,014 [191] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:03 UTC)" (scheduled at 2020-03-31 09:47:03.010318+00:00) 2020-03-31 09:47:03,016 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:47:03,016 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), u'imagestoragereplication/%']) 2020-03-31 09:47:03,016 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:06.924497+00:00 (in 3.910343 seconds) storagereplication stdout | 2020-03-31 09:47:03,016 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:06.924497+00:00 (in 3.910343 seconds) 2020-03-31 09:47:03,030 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), True, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), 0, u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:47:03,030 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), True, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), 0, u'imagestoragereplication/%']) 2020-03-31 09:47:03,036 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), True, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), u'imagestoragereplication/%']) storagereplication stdout | 2020-03-31 09:47:03,036 [191] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), True, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), 0, u'imagestoragereplication/%', False, datetime.datetime(2020, 3, 31, 9, 47, 3, 14883), u'imagestoragereplication/%']) 2020-03-31 09:47:03,040 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:47:03,040 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:03,043 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 storagereplication stdout | 2020-03-31 09:47:03,043 [191] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:03,043 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:03 UTC)" executed successfully 2020-03-31 09:47:03,044 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:47:03,043 [191] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:03 UTC)" executed successfully 2020-03-31 09:47:03,044 [191] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:03,045 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics storagereplication stdout | 2020-03-31 09:47:03,045 [191] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:47:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:47:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:24.410927+00:00 (in 21.079926 seconds) 2020-03-31 09:47:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:33 UTC)" (scheduled at 2020-03-31 09:47:03.330112+00:00) 2020-03-31 09:47:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:47:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:24.410927+00:00 (in 21.079926 seconds) 2020-03-31 09:47:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:33 UTC)" (scheduled at 2020-03-31 09:47:03.330112+00:00) 2020-03-31 09:47:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 3, 331764), True, datetime.datetime(2020, 3, 31, 9, 47, 3, 331764), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:47:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 3, 331764), True, datetime.datetime(2020, 3, 31, 9, 47, 3, 331764), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:47:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:03,352 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:47:03,352 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:03,352 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:47:03,352 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:33 UTC)" executed successfully 2020-03-31 09:47:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:14 UTC)" (scheduled at 2020-03-31 09:47:04.323352+00:00) 2020-03-31 09:47:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:47:04,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:14 UTC)" (scheduled at 2020-03-31 09:47:04.323352+00:00) 2020-03-31 09:47:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:04,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:14.323352+00:00 (in 9.998946 seconds) notificationworker stdout | 2020-03-31 09:47:04,327 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:14.323352+00:00 (in 9.998946 seconds) 2020-03-31 09:47:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 4, 325240), True, datetime.datetime(2020, 3, 31, 9, 47, 4, 325240), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:47:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 4, 325240), True, datetime.datetime(2020, 3, 31, 9, 47, 4, 325240), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:47:04,343 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:47:04,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:04,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:04,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:14 UTC)" executed successfully 2020-03-31 09:47:04,446 [518] [DEBUG] [app] Starting request: urn:request:0369a70d-eb8d-41e4-a6a1-1af19068f5cb (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:04,446 [518] [DEBUG] [app] Starting request: urn:request:0369a70d-eb8d-41e4-a6a1-1af19068f5cb (/health/instance) 2020-03-31 09:47:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:04,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:04,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:04,454 [538] [DEBUG] [app] Starting request: urn:request:dfed7d3d-edcb-44e1-9ec1-1f9f8226d703 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:04,454 [538] [DEBUG] [app] Starting request: urn:request:dfed7d3d-edcb-44e1-9ec1-1f9f8226d703 (/v1/_internal_ping) 2020-03-31 09:47:04,455 [538] [DEBUG] [app] Ending request: urn:request:dfed7d3d-edcb-44e1-9ec1-1f9f8226d703 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:04,455 [538] [DEBUG] [app] Ending request: urn:request:dfed7d3d-edcb-44e1-9ec1-1f9f8226d703 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:04,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:47:04,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:04,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:47:04,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:04,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:04,459 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:04,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:04,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:04,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:04,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:04,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:04,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:04,486 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:04,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:04,491 [518] [DEBUG] [app] Starting request: urn:request:9e65609f-5106-4539-a60d-9171c5214446 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:04,491 [518] [DEBUG] [app] Starting request: urn:request:9e65609f-5106-4539-a60d-9171c5214446 (/_internal_ping) 2020-03-31 09:47:04,492 [518] [DEBUG] [app] Ending request: urn:request:9e65609f-5106-4539-a60d-9171c5214446 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:04,492 [518] [DEBUG] [app] Ending request: urn:request:9e65609f-5106-4539-a60d-9171c5214446 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:47:04,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:04,492 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:04,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:47:04,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:47:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:04,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:47:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:04,505 [537] [DEBUG] [app] Starting request: urn:request:bb3c923f-87ad-4bde-ab34-ea07b0ad1fe7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:04,505 [537] [DEBUG] [app] Starting request: urn:request:bb3c923f-87ad-4bde-ab34-ea07b0ad1fe7 (/c1/_internal_ping) 2020-03-31 09:47:04,506 [537] [DEBUG] [app] Ending request: urn:request:bb3c923f-87ad-4bde-ab34-ea07b0ad1fe7 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:04,506 [537] [DEBUG] [app] Ending request: urn:request:bb3c923f-87ad-4bde-ab34-ea07b0ad1fe7 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:04,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:47:04,506 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:04,507 [518] [DEBUG] [app] Ending request: urn:request:0369a70d-eb8d-41e4-a6a1-1af19068f5cb (/health/instance) 2020-03-31 09:47:04,508 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:04,507 [518] [DEBUG] [app] Ending request: urn:request:0369a70d-eb8d-41e4-a6a1-1af19068f5cb (/health/instance) 2020-03-31 09:47:04,508 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:04,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:47:04,508 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:47:06,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:47:06,927 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:16.924497+00:00 (in 9.996400 seconds) 2020-03-31 09:47:06,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:16 UTC)" (scheduled at 2020-03-31 09:47:06.924497+00:00) 2020-03-31 09:47:06,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:47:06,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:16.924497+00:00 (in 9.996400 seconds) 2020-03-31 09:47:06,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:16 UTC)" (scheduled at 2020-03-31 09:47:06.924497+00:00) 2020-03-31 09:47:06,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:06,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 6, 928878), True, datetime.datetime(2020, 3, 31, 9, 47, 6, 928878), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:47:06,931 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 6, 928878), True, datetime.datetime(2020, 3, 31, 9, 47, 6, 928878), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:47:06,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:47:06,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:47:06,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:06,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:16 UTC)" executed successfully 2020-03-31 09:47:08,273 [518] [DEBUG] [app] Starting request: urn:request:e85f03cf-59bb-4f46-a65e-0d586c08a4c7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:08,273 [518] [DEBUG] [app] Starting request: urn:request:e85f03cf-59bb-4f46-a65e-0d586c08a4c7 (/health/instance) 2020-03-31 09:47:08,278 [555] [DEBUG] [app] Starting request: urn:request:5cbb1511-b303-405f-b459-425a2194b230 (/v1/_internal_ping) 2020-03-31 09:47:08,280 [555] [DEBUG] [app] Ending request: urn:request:5cbb1511-b303-405f-b459-425a2194b230 (/v1/_internal_ping) 2020-03-31 09:47:08,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2020-03-31 09:47:08,278 [555] [DEBUG] [app] Starting request: urn:request:5cbb1511-b303-405f-b459-425a2194b230 (/v1/_internal_ping) 2020-03-31 09:47:08,280 [555] [DEBUG] [app] Ending request: urn:request:5cbb1511-b303-405f-b459-425a2194b230 (/v1/_internal_ping) 2020-03-31 09:47:08,280 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:08,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:08,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:08,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:08,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2020-03-31 09:47:08,287 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:08,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:08,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:08,288 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:08,289 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:08,294 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:08,309 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:08,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:08,314 [518] [DEBUG] [app] Starting request: urn:request:45ce3ba7-e8af-4532-b723-fa848e8fc889 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:08,314 [518] [DEBUG] [app] Starting request: urn:request:45ce3ba7-e8af-4532-b723-fa848e8fc889 (/_internal_ping) 2020-03-31 09:47:08,315 [518] [DEBUG] [app] Ending request: urn:request:45ce3ba7-e8af-4532-b723-fa848e8fc889 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:08,315 [518] [DEBUG] [app] Ending request: urn:request:45ce3ba7-e8af-4532-b723-fa848e8fc889 (/_internal_ping) 2020-03-31 09:47:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:08,316 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:08,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:47:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:47:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:08,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:47:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:47:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:08,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:08,328 [532] [DEBUG] [app] Starting request: urn:request:dac1de09-a602-4cfd-8a78-ccde6e3347d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:08,328 [532] [DEBUG] [app] Starting request: urn:request:dac1de09-a602-4cfd-8a78-ccde6e3347d2 (/c1/_internal_ping) 2020-03-31 09:47:08,329 [532] [DEBUG] [app] Ending request: urn:request:dac1de09-a602-4cfd-8a78-ccde6e3347d2 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:08,329 [532] [DEBUG] [app] Ending request: urn:request:dac1de09-a602-4cfd-8a78-ccde6e3347d2 (/c1/_internal_ping) 2020-03-31 09:47:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:08,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:08,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:47:08,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:08,332 [518] [DEBUG] [app] Ending request: urn:request:e85f03cf-59bb-4f46-a65e-0d586c08a4c7 (/health/instance) 2020-03-31 09:47:08,332 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:08,332 [518] [DEBUG] [app] Ending request: urn:request:e85f03cf-59bb-4f46-a65e-0d586c08a4c7 (/health/instance) 2020-03-31 09:47:08,332 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:47:08,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:47:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:47:11,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:21.440707+00:00 (in 9.997024 seconds) 2020-03-31 09:47:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:21 UTC)" (scheduled at 2020-03-31 09:47:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:47:11,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:21.440707+00:00 (in 9.997024 seconds) 2020-03-31 09:47:11,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:21 UTC)" (scheduled at 2020-03-31 09:47:11.440707+00:00) 2020-03-31 09:47:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:47:11,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:47:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:47:11,463 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:47:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:47:11,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:47:11,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:21 UTC)" executed successfully 2020-03-31 09:47:14,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:14,328 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:14 UTC)" (scheduled at 2020-03-31 09:47:14.323352+00:00) 2020-03-31 09:47:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:47:14,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:14 UTC)" (scheduled at 2020-03-31 09:47:14.323352+00:00) 2020-03-31 09:47:14,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:14,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 14, 328936), True, datetime.datetime(2020, 3, 31, 9, 47, 14, 328936), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:47:14,332 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 14, 328936), True, datetime.datetime(2020, 3, 31, 9, 47, 14, 328936), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:47:14,332 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:24.323352+00:00 (in 9.995023 seconds) notificationworker stdout | 2020-03-31 09:47:14,332 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:24.323352+00:00 (in 9.995023 seconds) 2020-03-31 09:47:14,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:47:14,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:14,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:14,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:24 UTC)" executed successfully 2020-03-31 09:47:14,446 [518] [DEBUG] [app] Starting request: urn:request:d06ba77b-7dd3-4fba-9300-8bdd7e3594bb (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:14,446 [518] [DEBUG] [app] Starting request: urn:request:d06ba77b-7dd3-4fba-9300-8bdd7e3594bb (/health/instance) 2020-03-31 09:47:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:14,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:14,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:14,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:14,455 [549] [DEBUG] [app] Starting request: urn:request:5407e9df-a442-4953-be1c-3e3f19a1ea54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:14,455 [549] [DEBUG] [app] Starting request: urn:request:5407e9df-a442-4953-be1c-3e3f19a1ea54 (/v1/_internal_ping) 2020-03-31 09:47:14,456 [549] [DEBUG] [app] Ending request: urn:request:5407e9df-a442-4953-be1c-3e3f19a1ea54 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:14,456 [549] [DEBUG] [app] Ending request: urn:request:5407e9df-a442-4953-be1c-3e3f19a1ea54 (/v1/_internal_ping) 2020-03-31 09:47:14,457 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:47:14,457 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:47:14,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:14,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:14,459 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:14,459 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:14,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:14,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:14,461 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:14,461 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:14,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:14,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:14,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:14,486 [527] [DEBUG] [app] Starting request: urn:request:771599ed-bcf2-476b-ba2c-045800ada194 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:14,486 [527] [DEBUG] [app] Starting request: urn:request:771599ed-bcf2-476b-ba2c-045800ada194 (/_internal_ping) 2020-03-31 09:47:14,487 [527] [DEBUG] [app] Ending request: urn:request:771599ed-bcf2-476b-ba2c-045800ada194 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:14,487 [527] [DEBUG] [app] Ending request: urn:request:771599ed-bcf2-476b-ba2c-045800ada194 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:47:14,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:14,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:14,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:14,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:14,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:14,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:47:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:14,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:14,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:14,501 [532] [DEBUG] [app] Starting request: urn:request:7b6a0f16-bfe5-41c1-a5b0-55accfacf492 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:14,501 [532] [DEBUG] [app] Starting request: urn:request:7b6a0f16-bfe5-41c1-a5b0-55accfacf492 (/c1/_internal_ping) 2020-03-31 09:47:14,502 [532] [DEBUG] [app] Ending request: urn:request:7b6a0f16-bfe5-41c1-a5b0-55accfacf492 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:14,502 [532] [DEBUG] [app] Ending request: urn:request:7b6a0f16-bfe5-41c1-a5b0-55accfacf492 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:14,503 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:14,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:47:14,503 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:14,504 [518] [DEBUG] [app] Ending request: urn:request:d06ba77b-7dd3-4fba-9300-8bdd7e3594bb (/health/instance) 2020-03-31 09:47:14,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:14,504 [518] [DEBUG] [app] Ending request: urn:request:d06ba77b-7dd3-4fba-9300-8bdd7e3594bb (/health/instance) 2020-03-31 09:47:14,504 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:14,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:47:14,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:47:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:47:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:45.064514+00:00 (in 29.998942 seconds) 2020-03-31 09:47:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:45 UTC)" (scheduled at 2020-03-31 09:47:15.064514+00:00) gcworker stdout | 2020-03-31 09:47:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:45.064514+00:00 (in 29.998942 seconds) 2020-03-31 09:47:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:45 UTC)" (scheduled at 2020-03-31 09:47:15.064514+00:00) 2020-03-31 09:47:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438435, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:47:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438435, 1209600, 500, 1, 0]) 2020-03-31 09:47:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:47:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:47:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:47:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:15,085 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:45 UTC)" executed successfully 2020-03-31 09:47:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:47:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:26.924497+00:00 (in 9.999273 seconds) 2020-03-31 09:47:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:26 UTC)" (scheduled at 2020-03-31 09:47:16.924497+00:00) 2020-03-31 09:47:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:47:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:26.924497+00:00 (in 9.999273 seconds) 2020-03-31 09:47:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:26 UTC)" (scheduled at 2020-03-31 09:47:16.924497+00:00) 2020-03-31 09:47:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 16, 925872), True, datetime.datetime(2020, 3, 31, 9, 47, 16, 925872), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:47:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 16, 925872), True, datetime.datetime(2020, 3, 31, 9, 47, 16, 925872), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:47:16,944 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:47:16,944 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:47:16,945 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:16,945 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:26 UTC)" executed successfully 2020-03-31 09:47:18,273 [518] [DEBUG] [app] Starting request: urn:request:00227d4f-5b1b-40f9-b011-df7ccd84d987 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:18,273 [518] [DEBUG] [app] Starting request: urn:request:00227d4f-5b1b-40f9-b011-df7ccd84d987 (/health/instance) 2020-03-31 09:47:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:18,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:18,278 [554] [DEBUG] [app] Starting request: urn:request:bc44b105-3cc1-435b-a723-a6de3bbd55e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:18,278 [554] [DEBUG] [app] Starting request: urn:request:bc44b105-3cc1-435b-a723-a6de3bbd55e9 (/v1/_internal_ping) 2020-03-31 09:47:18,280 [554] [DEBUG] [app] Ending request: urn:request:bc44b105-3cc1-435b-a723-a6de3bbd55e9 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:18,280 [554] [DEBUG] [app] Ending request: urn:request:bc44b105-3cc1-435b-a723-a6de3bbd55e9 (/v1/_internal_ping) 2020-03-31 09:47:18,281 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:47:18,281 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:47:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:18,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:18,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:18,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:18,285 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:18,285 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:18,286 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:18,286 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:18,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:18,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:18,313 [518] [DEBUG] [app] Starting request: urn:request:2ac2a036-1b11-4800-88de-4b695097672e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:18,313 [518] [DEBUG] [app] Starting request: urn:request:2ac2a036-1b11-4800-88de-4b695097672e (/_internal_ping) 2020-03-31 09:47:18,313 [518] [DEBUG] [app] Ending request: urn:request:2ac2a036-1b11-4800-88de-4b695097672e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:18,313 [518] [DEBUG] [app] Ending request: urn:request:2ac2a036-1b11-4800-88de-4b695097672e (/_internal_ping) 2020-03-31 09:47:18,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:18,314 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:18,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:18,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:47:18,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:47:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:18,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:47:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:18,326 [543] [DEBUG] [app] Starting request: urn:request:3f52a8ac-5961-411a-b2c1-980122e3f15f (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:18,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:47:18,326 [543] [DEBUG] [app] Starting request: urn:request:3f52a8ac-5961-411a-b2c1-980122e3f15f (/c1/_internal_ping) 2020-03-31 09:47:18,327 [543] [DEBUG] [app] Ending request: urn:request:3f52a8ac-5961-411a-b2c1-980122e3f15f (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:18,327 [543] [DEBUG] [app] Ending request: urn:request:3f52a8ac-5961-411a-b2c1-980122e3f15f (/c1/_internal_ping) 2020-03-31 09:47:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:18,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:18,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:18,330 [518] [DEBUG] [app] Ending request: urn:request:00227d4f-5b1b-40f9-b011-df7ccd84d987 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:47:18,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:18,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:47:18,330 [518] [DEBUG] [app] Ending request: urn:request:00227d4f-5b1b-40f9-b011-df7ccd84d987 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:18,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:18,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:47:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:47:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:31.440707+00:00 (in 9.998686 seconds) 2020-03-31 09:47:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:31 UTC)" (scheduled at 2020-03-31 09:47:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:47:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:31.440707+00:00 (in 9.998686 seconds) 2020-03-31 09:47:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:31 UTC)" (scheduled at 2020-03-31 09:47:21.440707+00:00) 2020-03-31 09:47:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:47:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:47:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:47:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:47:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:47:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:31 UTC)" executed successfully 2020-03-31 09:47:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:47:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:52.113221+00:00 (in 29.999114 seconds) 2020-03-31 09:47:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:52 UTC)" (scheduled at 2020-03-31 09:47:22.113221+00:00) securityworker stdout | 2020-03-31 09:47:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:52.113221+00:00 (in 29.999114 seconds) 2020-03-31 09:47:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:52 UTC)" (scheduled at 2020-03-31 09:47:22.113221+00:00) 2020-03-31 09:47:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:47:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:47:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:47:22,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:47:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:47:22,131 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:47:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:47:22,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:22,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:47:52 UTC)" executed successfully 2020-03-31 09:47:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:24,327 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:31.498397+00:00 (in 7.170313 seconds) 2020-03-31 09:47:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:34 UTC)" (scheduled at 2020-03-31 09:47:24.323352+00:00) 2020-03-31 09:47:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:47:24,328 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:31.498397+00:00 (in 7.170313 seconds) 2020-03-31 09:47:24,328 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:34 UTC)" (scheduled at 2020-03-31 09:47:24.323352+00:00) 2020-03-31 09:47:24,328 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 24, 329059), True, datetime.datetime(2020, 3, 31, 9, 47, 24, 329059), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:47:24,331 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 24, 329059), True, datetime.datetime(2020, 3, 31, 9, 47, 24, 329059), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:47:24,345 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:47:24,345 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:24,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:24,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:24,345 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:24,345 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:34 UTC)" executed successfully 2020-03-31 09:47:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:47:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:33.330112+00:00 (in 8.918414 seconds) 2020-03-31 09:47:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:24 UTC)" (scheduled at 2020-03-31 09:47:24.410927+00:00) security_notification_worker stdout | 2020-03-31 09:47:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:33.330112+00:00 (in 8.918414 seconds) 2020-03-31 09:47:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:24 UTC)" (scheduled at 2020-03-31 09:47:24.410927+00:00) 2020-03-31 09:47:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:47:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:24 UTC)" executed successfully 2020-03-31 09:47:24,446 [518] [DEBUG] [app] Starting request: urn:request:6c1a5a1a-662b-4132-a760-6057dd6feb2b (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:24,446 [518] [DEBUG] [app] Starting request: urn:request:6c1a5a1a-662b-4132-a760-6057dd6feb2b (/health/instance) 2020-03-31 09:47:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:24,454 [554] [DEBUG] [app] Starting request: urn:request:27f8467b-f417-4b6e-9974-8f8e1c1f6152 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:24,454 [554] [DEBUG] [app] Starting request: urn:request:27f8467b-f417-4b6e-9974-8f8e1c1f6152 (/v1/_internal_ping) 2020-03-31 09:47:24,455 [554] [DEBUG] [app] Ending request: urn:request:27f8467b-f417-4b6e-9974-8f8e1c1f6152 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:24,455 [554] [DEBUG] [app] Ending request: urn:request:27f8467b-f417-4b6e-9974-8f8e1c1f6152 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:24,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:47:24,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:47:24,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:24,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:24,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:24,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:24,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:24,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:24,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:24,460 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:24,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:24,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:24,487 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:24,488 [518] [DEBUG] [app] Starting request: urn:request:3c7f2c8f-65b4-4615-b553-799aacc7f547 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:24,488 [518] [DEBUG] [app] Starting request: urn:request:3c7f2c8f-65b4-4615-b553-799aacc7f547 (/_internal_ping) 2020-03-31 09:47:24,489 [518] [DEBUG] [app] Ending request: urn:request:3c7f2c8f-65b4-4615-b553-799aacc7f547 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:24,489 [518] [DEBUG] [app] Ending request: urn:request:3c7f2c8f-65b4-4615-b553-799aacc7f547 (/_internal_ping) 2020-03-31 09:47:24,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:24,489 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:24,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:24,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:24,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:47:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:24,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:24,502 [534] [DEBUG] [app] Starting request: urn:request:8ea3c664-33d0-4e6e-a980-42c8807ae7f0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:24,502 [534] [DEBUG] [app] Starting request: urn:request:8ea3c664-33d0-4e6e-a980-42c8807ae7f0 (/c1/_internal_ping) 2020-03-31 09:47:24,503 [534] [DEBUG] [app] Ending request: urn:request:8ea3c664-33d0-4e6e-a980-42c8807ae7f0 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:24,503 [534] [DEBUG] [app] Ending request: urn:request:8ea3c664-33d0-4e6e-a980-42c8807ae7f0 (/c1/_internal_ping) 2020-03-31 09:47:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:24,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:47:24,504 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:24,505 [518] [DEBUG] [app] Ending request: urn:request:6c1a5a1a-662b-4132-a760-6057dd6feb2b (/health/instance) 2020-03-31 09:47:24,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:24,505 [518] [DEBUG] [app] Ending request: urn:request:6c1a5a1a-662b-4132-a760-6057dd6feb2b (/health/instance) 2020-03-31 09:47:24,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:47:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:47:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:36.924497+00:00 (in 9.999243 seconds) 2020-03-31 09:47:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:36 UTC)" (scheduled at 2020-03-31 09:47:26.924497+00:00) 2020-03-31 09:47:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:47:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 26, 926718), True, datetime.datetime(2020, 3, 31, 9, 47, 26, 926718), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:47:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:36.924497+00:00 (in 9.999243 seconds) 2020-03-31 09:47:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:36 UTC)" (scheduled at 2020-03-31 09:47:26.924497+00:00) 2020-03-31 09:47:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:47:26,929 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 26, 926718), True, datetime.datetime(2020, 3, 31, 9, 47, 26, 926718), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:47:26,948 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:47:26,948 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:47:26,948 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:36 UTC)" executed successfully 2020-03-31 09:47:28,274 [527] [DEBUG] [app] Starting request: urn:request:7f8ccd29-d156-4e12-8f68-b58efa4bbd5b (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:28,274 [527] [DEBUG] [app] Starting request: urn:request:7f8ccd29-d156-4e12-8f68-b58efa4bbd5b (/health/instance) 2020-03-31 09:47:28,277 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:28,277 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:28,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:28,281 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:28,283 [554] [DEBUG] [app] Starting request: urn:request:af0bce0d-9f27-4496-9e99-31b0f886289d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:28,283 [554] [DEBUG] [app] Starting request: urn:request:af0bce0d-9f27-4496-9e99-31b0f886289d (/v1/_internal_ping) 2020-03-31 09:47:28,286 [554] [DEBUG] [app] Ending request: urn:request:af0bce0d-9f27-4496-9e99-31b0f886289d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:28,286 [554] [DEBUG] [app] Ending request: urn:request:af0bce0d-9f27-4496-9e99-31b0f886289d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:47:28,286 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:28,287 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:47:28,286 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:28,287 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:28,288 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:47:28,288 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:28,289 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:28,289 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:28,291 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:28,291 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:28,292 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:28,292 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:28,298 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:28,319 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:28,319 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:28,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:28,323 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:28,325 [526] [DEBUG] [app] Starting request: urn:request:0c5bbe22-7a46-40c8-bf6c-1c15761b5b05 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:28,325 [526] [DEBUG] [app] Starting request: urn:request:0c5bbe22-7a46-40c8-bf6c-1c15761b5b05 (/_internal_ping) 2020-03-31 09:47:28,326 [526] [DEBUG] [app] Ending request: urn:request:0c5bbe22-7a46-40c8-bf6c-1c15761b5b05 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:28,326 [526] [DEBUG] [app] Ending request: urn:request:0c5bbe22-7a46-40c8-bf6c-1c15761b5b05 (/_internal_ping) 2020-03-31 09:47:28,326 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:47:28,326 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:47:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:28,330 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:28,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:28,335 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:47:28,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:28,336 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:28,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:28,340 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:28,341 [534] [DEBUG] [app] Starting request: urn:request:fb7cecb6-611e-4506-b849-1f7c949431bd (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:28,341 [534] [DEBUG] [app] Starting request: urn:request:fb7cecb6-611e-4506-b849-1f7c949431bd (/c1/_internal_ping) 2020-03-31 09:47:28,342 [534] [DEBUG] [app] Ending request: urn:request:fb7cecb6-611e-4506-b849-1f7c949431bd (/c1/_internal_ping) 2020-03-31 09:47:28,343 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:28,343 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:47:28,342 [534] [DEBUG] [app] Ending request: urn:request:fb7cecb6-611e-4506-b849-1f7c949431bd (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:28,343 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:28,344 [527] [DEBUG] [app] Ending request: urn:request:7f8ccd29-d156-4e12-8f68-b58efa4bbd5b (/health/instance) 2020-03-31 09:47:28,345 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:47:28,343 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:28,345 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) gunicorn-web stdout | 2020-03-31 09:47:28,344 [527] [DEBUG] [app] Ending request: urn:request:7f8ccd29-d156-4e12-8f68-b58efa4bbd5b (/health/instance) 2020-03-31 09:47:28,345 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:28,345 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.074 49 0.074) 2020-03-31 09:47:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:47:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:00.337699+00:00 (in 29.998832 seconds) 2020-03-31 09:47:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:00 UTC)" (scheduled at 2020-03-31 09:47:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:47:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:00.337699+00:00 (in 29.998832 seconds) 2020-03-31 09:47:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:00 UTC)" (scheduled at 2020-03-31 09:47:30.337699+00:00) 2020-03-31 09:47:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 47, 30, 339759), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:47:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 47, 30, 339759), False, 50, 1, 0]) 2020-03-31 09:47:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:47:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:47:30,357 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:47:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:00 UTC)" executed successfully 2020-03-31 09:47:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:47:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:41.440707+00:00 (in 9.998881 seconds) 2020-03-31 09:47:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:41 UTC)" (scheduled at 2020-03-31 09:47:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:47:31,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:41.440707+00:00 (in 9.998881 seconds) 2020-03-31 09:47:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:41 UTC)" (scheduled at 2020-03-31 09:47:31.440707+00:00) 2020-03-31 09:47:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:47:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:47:31,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:47:31,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:47:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:47:31,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:31,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:41 UTC)" executed successfully 2020-03-31 09:47:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:31,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:34.323352+00:00 (in 2.824133 seconds) 2020-03-31 09:47:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:31 UTC)" (scheduled at 2020-03-31 09:47:31.498397+00:00) notificationworker stdout | 2020-03-31 09:47:31,499 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:34.323352+00:00 (in 2.824133 seconds) 2020-03-31 09:47:31,499 [175] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:31 UTC)" (scheduled at 2020-03-31 09:47:31.498397+00:00) 2020-03-31 09:47:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), u'notification/%']) notificationworker stdout | 2020-03-31 09:47:31,501 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), u'notification/%']) 2020-03-31 09:47:31,518 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), True, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), 0, u'notification/%']) notificationworker stdout | 2020-03-31 09:47:31,518 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), True, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), 0, u'notification/%']) 2020-03-31 09:47:31,523 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), True, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), u'notification/%']) notificationworker stdout | 2020-03-31 09:47:31,523 [175] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), True, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), 0, u'notification/%', False, datetime.datetime(2020, 3, 31, 9, 47, 31, 500058), u'notification/%']) 2020-03-31 09:47:31,527 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:47:31,527 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:31,530 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:47:31,530 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:31,531 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:31 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:31,531 [175] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:52:31 UTC)" executed successfully 2020-03-31 09:47:31,531 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:47:31,531 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:31,532 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics notificationworker stdout | 2020-03-31 09:47:31,532 [175] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:31,534 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 notificationworker stdout | 2020-03-31 09:47:31,534 [175] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:31,535 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 notificationworker stdout | 2020-03-31 09:47:31,535 [175] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:31,536 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics notificationworker stdout | 2020-03-31 09:47:31,536 [175] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:47:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:47:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:03.330112+00:00 (in 29.998730 seconds) 2020-03-31 09:47:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:03 UTC)" (scheduled at 2020-03-31 09:47:33.330112+00:00) 2020-03-31 09:47:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:47:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:03.330112+00:00 (in 29.998730 seconds) 2020-03-31 09:47:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:03 UTC)" (scheduled at 2020-03-31 09:47:33.330112+00:00) 2020-03-31 09:47:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 33, 332334), True, datetime.datetime(2020, 3, 31, 9, 47, 33, 332334), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:47:33,333 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 33, 332334), True, datetime.datetime(2020, 3, 31, 9, 47, 33, 332334), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:47:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:47:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:47:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:03 UTC)" executed successfully 2020-03-31 09:47:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:47:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:43.539999+00:00 (in 10.172449 seconds) 2020-03-31 09:47:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:33 UTC)" (scheduled at 2020-03-31 09:47:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:47:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:43.539999+00:00 (in 10.172449 seconds) 2020-03-31 09:47:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:33 UTC)" (scheduled at 2020-03-31 09:47:33.366791+00:00) 2020-03-31 09:47:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:47:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 33, 368371), True, datetime.datetime(2020, 3, 31, 9, 47, 33, 368371), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:47:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 33, 368371), True, datetime.datetime(2020, 3, 31, 9, 47, 33, 368371), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:47:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:47:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:47:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:33 UTC)" executed successfully 2020-03-31 09:47:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:44.323352+00:00 (in 9.999097 seconds) 2020-03-31 09:47:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:44 UTC)" (scheduled at 2020-03-31 09:47:34.323352+00:00) 2020-03-31 09:47:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:47:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:44.323352+00:00 (in 9.999097 seconds) 2020-03-31 09:47:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:44 UTC)" (scheduled at 2020-03-31 09:47:34.323352+00:00) 2020-03-31 09:47:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 34, 324873), True, datetime.datetime(2020, 3, 31, 9, 47, 34, 324873), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:47:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 34, 324873), True, datetime.datetime(2020, 3, 31, 9, 47, 34, 324873), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:47:34,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:47:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:44 UTC)" executed successfully 2020-03-31 09:47:34,445 [527] [DEBUG] [app] Starting request: urn:request:f1bcfa64-1f41-46f8-bfc0-062c68f9bbb4 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:34,445 [527] [DEBUG] [app] Starting request: urn:request:f1bcfa64-1f41-46f8-bfc0-062c68f9bbb4 (/health/instance) 2020-03-31 09:47:34,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:34,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:34,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:34,451 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:34,453 [538] [DEBUG] [app] Starting request: urn:request:da8b9ce1-0631-4403-a185-b7bd0ccdc50f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:34,453 [538] [DEBUG] [app] Starting request: urn:request:da8b9ce1-0631-4403-a185-b7bd0ccdc50f (/v1/_internal_ping) 2020-03-31 09:47:34,454 [538] [DEBUG] [app] Ending request: urn:request:da8b9ce1-0631-4403-a185-b7bd0ccdc50f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:34,454 [538] [DEBUG] [app] Ending request: urn:request:da8b9ce1-0631-4403-a185-b7bd0ccdc50f (/v1/_internal_ping) 2020-03-31 09:47:34,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:47:34,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:47:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:34,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:34,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:34,460 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:34,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:34,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:34,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:34,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:34,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:34,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:34,488 [527] [DEBUG] [app] Starting request: urn:request:40748bc3-dc4b-48fe-bf5d-0715eea69dc1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:34,488 [527] [DEBUG] [app] Starting request: urn:request:40748bc3-dc4b-48fe-bf5d-0715eea69dc1 (/_internal_ping) 2020-03-31 09:47:34,489 [527] [DEBUG] [app] Ending request: urn:request:40748bc3-dc4b-48fe-bf5d-0715eea69dc1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:34,489 [527] [DEBUG] [app] Ending request: urn:request:40748bc3-dc4b-48fe-bf5d-0715eea69dc1 (/_internal_ping) 2020-03-31 09:47:34,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:34,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:34,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:34,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:34,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:34,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:34,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:34,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:34,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:47:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:34,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:34,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:34,502 [534] [DEBUG] [app] Starting request: urn:request:fa7aa837-0adc-459f-a56e-205658592d8c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:34,502 [534] [DEBUG] [app] Starting request: urn:request:fa7aa837-0adc-459f-a56e-205658592d8c (/c1/_internal_ping) 2020-03-31 09:47:34,503 [534] [DEBUG] [app] Ending request: urn:request:fa7aa837-0adc-459f-a56e-205658592d8c (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:34,503 [534] [DEBUG] [app] Ending request: urn:request:fa7aa837-0adc-459f-a56e-205658592d8c (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:34,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:34,505 [527] [DEBUG] [app] Ending request: urn:request:f1bcfa64-1f41-46f8-bfc0-062c68f9bbb4 (/health/instance) 2020-03-31 09:47:34,505 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-verbs stdout | 2020-03-31 09:47:34,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:34,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:34,505 [527] [DEBUG] [app] Ending request: urn:request:f1bcfa64-1f41-46f8-bfc0-062c68f9bbb4 (/health/instance) 2020-03-31 09:47:34,505 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:34,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:47:34,507 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:47:36,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:38.010934+00:00 (in 1.085770 seconds) 2020-03-31 09:47:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:46 UTC)" (scheduled at 2020-03-31 09:47:36.924497+00:00) 2020-03-31 09:47:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 36, 925955), True, datetime.datetime(2020, 3, 31, 9, 47, 36, 925955), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:47:36,924 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:47:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:38.010934+00:00 (in 1.085770 seconds) 2020-03-31 09:47:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:46 UTC)" (scheduled at 2020-03-31 09:47:36.924497+00:00) 2020-03-31 09:47:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:47:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 36, 925955), True, datetime.datetime(2020, 3, 31, 9, 47, 36, 925955), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:47:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:47:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:47:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:46 UTC)" executed successfully 2020-03-31 09:47:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:47:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:46.924497+00:00 (in 8.912818 seconds) 2020-03-31 09:47:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:38 UTC)" (scheduled at 2020-03-31 09:47:38.010934+00:00) 2020-03-31 09:47:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:47:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:46.924497+00:00 (in 8.912818 seconds) 2020-03-31 09:47:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:38 UTC)" (scheduled at 2020-03-31 09:47:38.010934+00:00) 2020-03-31 09:47:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:38 UTC)" executed successfully 2020-03-31 09:47:38,273 [525] [DEBUG] [app] Starting request: urn:request:ab4caa8e-bb84-4cd8-ab01-ba7153cd3731 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:38,273 [525] [DEBUG] [app] Starting request: urn:request:ab4caa8e-bb84-4cd8-ab01-ba7153cd3731 (/health/instance) 2020-03-31 09:47:38,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:38,275 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:38,280 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:38,280 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:38,281 [556] [DEBUG] [app] Starting request: urn:request:22454976-53b5-4d10-a5d5-874597cd5603 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:38,281 [556] [DEBUG] [app] Starting request: urn:request:22454976-53b5-4d10-a5d5-874597cd5603 (/v1/_internal_ping) 2020-03-31 09:47:38,282 [556] [DEBUG] [app] Ending request: urn:request:22454976-53b5-4d10-a5d5-874597cd5603 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:38,282 [556] [DEBUG] [app] Ending request: urn:request:22454976-53b5-4d10-a5d5-874597cd5603 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:38,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:38,283 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:47:38,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:38,284 [525] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:38,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:47:38,283 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:38,284 [525] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:47:38,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:38,289 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:38,289 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:38,290 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:38,290 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:38,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:38,293 [525] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:38,309 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:38,309 [525] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:38,313 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:38,313 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:38,314 [525] [DEBUG] [app] Starting request: urn:request:a2547eaa-4643-4f8c-a4a6-7b14260d6743 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:38,314 [525] [DEBUG] [app] Starting request: urn:request:a2547eaa-4643-4f8c-a4a6-7b14260d6743 (/_internal_ping) 2020-03-31 09:47:38,315 [525] [DEBUG] [app] Ending request: urn:request:a2547eaa-4643-4f8c-a4a6-7b14260d6743 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:38,315 [525] [DEBUG] [app] Ending request: urn:request:a2547eaa-4643-4f8c-a4a6-7b14260d6743 (/_internal_ping) 2020-03-31 09:47:38,316 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:47:38,316 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:38,316 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:47:38,316 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:38,319 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:38,319 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:38,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:38,320 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:38,323 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:38,323 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:47:38,324 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:38,324 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:38,327 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:38,327 [525] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:38,328 [534] [DEBUG] [app] Starting request: urn:request:6b15a5c6-f107-473e-b6a4-8a892097335d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:38,328 [534] [DEBUG] [app] Starting request: urn:request:6b15a5c6-f107-473e-b6a4-8a892097335d (/c1/_internal_ping) 2020-03-31 09:47:38,329 [534] [DEBUG] [app] Ending request: urn:request:6b15a5c6-f107-473e-b6a4-8a892097335d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:38,329 [534] [DEBUG] [app] Ending request: urn:request:6b15a5c6-f107-473e-b6a4-8a892097335d (/c1/_internal_ping) 2020-03-31 09:47:38,330 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:38,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:38,330 [525] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:38,332 [525] [DEBUG] [app] Ending request: urn:request:ab4caa8e-bb84-4cd8-ab01-ba7153cd3731 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:47:38,330 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:38,332 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:38,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:47:38,332 [525] [DEBUG] [app] Ending request: urn:request:ab4caa8e-bb84-4cd8-ab01-ba7153cd3731 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:38,332 [525] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:38,333 [525] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:47:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:47:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:51 UTC)" (scheduled at 2020-03-31 09:47:41.440707+00:00) 2020-03-31 09:47:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:51.440707+00:00 (in 9.999134 seconds) repositoryactioncounter stdout | 2020-03-31 09:47:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:51 UTC)" (scheduled at 2020-03-31 09:47:41.440707+00:00) 2020-03-31 09:47:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:51.440707+00:00 (in 9.999134 seconds) 2020-03-31 09:47:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:47:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:47:41,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:47:41,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:47:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:47:41,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:41,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:51 UTC)" executed successfully 2020-03-31 09:47:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:47:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:33.366791+00:00 (in 49.826037 seconds) 2020-03-31 09:47:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:43 UTC)" (scheduled at 2020-03-31 09:47:43.539999+00:00) 2020-03-31 09:47:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:47:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:33.366791+00:00 (in 49.826037 seconds) 2020-03-31 09:47:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:43 UTC)" (scheduled at 2020-03-31 09:47:43.539999+00:00) 2020-03-31 09:47:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:43 UTC)" executed successfully 2020-03-31 09:47:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:49.498966+00:00 (in 5.174900 seconds) 2020-03-31 09:47:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:54 UTC)" (scheduled at 2020-03-31 09:47:44.323352+00:00) 2020-03-31 09:47:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:47:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:49.498966+00:00 (in 5.174900 seconds) 2020-03-31 09:47:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:54 UTC)" (scheduled at 2020-03-31 09:47:44.323352+00:00) 2020-03-31 09:47:44,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 44, 324906), True, datetime.datetime(2020, 3, 31, 9, 47, 44, 324906), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:47:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 44, 324906), True, datetime.datetime(2020, 3, 31, 9, 47, 44, 324906), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:47:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:47:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:54 UTC)" executed successfully 2020-03-31 09:47:44,446 [518] [DEBUG] [app] Starting request: urn:request:2294104f-a107-4e74-942b-bc27d13f5e17 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:44,446 [518] [DEBUG] [app] Starting request: urn:request:2294104f-a107-4e74-942b-bc27d13f5e17 (/health/instance) 2020-03-31 09:47:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:44,453 [548] [DEBUG] [app] Starting request: urn:request:79a8a761-c6ca-40a9-966a-568d3fd784cf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:44,453 [548] [DEBUG] [app] Starting request: urn:request:79a8a761-c6ca-40a9-966a-568d3fd784cf (/v1/_internal_ping) 2020-03-31 09:47:44,455 [548] [DEBUG] [app] Ending request: urn:request:79a8a761-c6ca-40a9-966a-568d3fd784cf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:44,455 [548] [DEBUG] [app] Ending request: urn:request:79a8a761-c6ca-40a9-966a-568d3fd784cf (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:44,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:47:44,455 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:47:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:44,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:44,458 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:44,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:44,459 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:44,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:44,460 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:44,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:44,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:44,486 [518] [DEBUG] [app] Starting request: urn:request:d89d9335-7c33-4ad1-bb5b-72a6caa788a5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:44,486 [518] [DEBUG] [app] Starting request: urn:request:d89d9335-7c33-4ad1-bb5b-72a6caa788a5 (/_internal_ping) 2020-03-31 09:47:44,487 [518] [DEBUG] [app] Ending request: urn:request:d89d9335-7c33-4ad1-bb5b-72a6caa788a5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:44,487 [518] [DEBUG] [app] Ending request: urn:request:d89d9335-7c33-4ad1-bb5b-72a6caa788a5 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:47:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:44,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:44,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:47:44,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:47:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:44,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:47:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:44,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:44,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:44,501 [534] [DEBUG] [app] Starting request: urn:request:8fdfeacd-de29-43db-8fe0-3d74ce93fdaf (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:44,501 [534] [DEBUG] [app] Starting request: urn:request:8fdfeacd-de29-43db-8fe0-3d74ce93fdaf (/c1/_internal_ping) 2020-03-31 09:47:44,502 [534] [DEBUG] [app] Ending request: urn:request:8fdfeacd-de29-43db-8fe0-3d74ce93fdaf (/c1/_internal_ping) 2020-03-31 09:47:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:44,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:47:44,502 [534] [DEBUG] [app] Ending request: urn:request:8fdfeacd-de29-43db-8fe0-3d74ce93fdaf (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:44,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:44,504 [518] [DEBUG] [app] Ending request: urn:request:2294104f-a107-4e74-942b-bc27d13f5e17 (/health/instance) 2020-03-31 09:47:44,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:47:44,502 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:47:44,504 [518] [DEBUG] [app] Ending request: urn:request:2294104f-a107-4e74-942b-bc27d13f5e17 (/health/instance) 2020-03-31 09:47:44,504 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:44,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:47:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:47:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:15.064514+00:00 (in 29.999121 seconds) 2020-03-31 09:47:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:15 UTC)" (scheduled at 2020-03-31 09:47:45.064514+00:00) gcworker stdout | 2020-03-31 09:47:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:15.064514+00:00 (in 29.999121 seconds) 2020-03-31 09:47:45,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:15 UTC)" (scheduled at 2020-03-31 09:47:45.064514+00:00) 2020-03-31 09:47:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438465, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:47:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438465, 1209600, 500, 1, 0]) 2020-03-31 09:47:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:47:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:47:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:47:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:15 UTC)" executed successfully 2020-03-31 09:47:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:56.924497+00:00 (in 9.999276 seconds) 2020-03-31 09:47:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:56 UTC)" (scheduled at 2020-03-31 09:47:46.924497+00:00) 2020-03-31 09:47:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 46, 925924), True, datetime.datetime(2020, 3, 31, 9, 47, 46, 925924), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:47:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:56.924497+00:00 (in 9.999276 seconds) 2020-03-31 09:47:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:56 UTC)" (scheduled at 2020-03-31 09:47:46.924497+00:00) 2020-03-31 09:47:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:46,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 46, 925924), True, datetime.datetime(2020, 3, 31, 9, 47, 46, 925924), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:47:46,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:47:46,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:47:46,944 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:46,944 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:47:56 UTC)" executed successfully 2020-03-31 09:47:48,272 [527] [DEBUG] [app] Starting request: urn:request:ca31e510-c7c4-42e3-9026-c60dc7a5be9c (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:48,272 [527] [DEBUG] [app] Starting request: urn:request:ca31e510-c7c4-42e3-9026-c60dc7a5be9c (/health/instance) 2020-03-31 09:47:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:48,280 [546] [DEBUG] [app] Starting request: urn:request:7d57a0c5-5092-44d1-9666-081e7efa1750 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:48,280 [546] [DEBUG] [app] Starting request: urn:request:7d57a0c5-5092-44d1-9666-081e7efa1750 (/v1/_internal_ping) 2020-03-31 09:47:48,281 [546] [DEBUG] [app] Ending request: urn:request:7d57a0c5-5092-44d1-9666-081e7efa1750 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:48,281 [546] [DEBUG] [app] Ending request: urn:request:7d57a0c5-5092-44d1-9666-081e7efa1750 (/v1/_internal_ping) 2020-03-31 09:47:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:47:48,282 [546] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:47:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:48,285 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:48,285 [546] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:48,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:48,286 [546] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:48,287 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:48,287 [546] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:48,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:48,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:48,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:48,308 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:48,311 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:48,313 [527] [DEBUG] [app] Starting request: urn:request:d8d1778e-b840-4036-9480-a5fe313a4fb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:48,313 [527] [DEBUG] [app] Starting request: urn:request:d8d1778e-b840-4036-9480-a5fe313a4fb2 (/_internal_ping) 2020-03-31 09:47:48,314 [527] [DEBUG] [app] Ending request: urn:request:d8d1778e-b840-4036-9480-a5fe313a4fb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:48,314 [527] [DEBUG] [app] Ending request: urn:request:d8d1778e-b840-4036-9480-a5fe313a4fb2 (/_internal_ping) 2020-03-31 09:47:48,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:48,314 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:47:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:48,319 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:47:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:48,329 [532] [DEBUG] [app] Starting request: urn:request:5009aa43-ffe8-447a-a28a-985ac7eef1a8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:48,329 [532] [DEBUG] [app] Starting request: urn:request:5009aa43-ffe8-447a-a28a-985ac7eef1a8 (/c1/_internal_ping) 2020-03-31 09:47:48,329 [532] [DEBUG] [app] Ending request: urn:request:5009aa43-ffe8-447a-a28a-985ac7eef1a8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:48,329 [532] [DEBUG] [app] Ending request: urn:request:5009aa43-ffe8-447a-a28a-985ac7eef1a8 (/c1/_internal_ping) 2020-03-31 09:47:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:48,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:47:48,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:48,332 [527] [DEBUG] [app] Ending request: urn:request:ca31e510-c7c4-42e3-9026-c60dc7a5be9c (/health/instance) 2020-03-31 09:47:48,332 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:48,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:47:48,332 [527] [DEBUG] [app] Ending request: urn:request:ca31e510-c7c4-42e3-9026-c60dc7a5be9c (/health/instance) 2020-03-31 09:47:48,332 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:48,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:47:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:54.323352+00:00 (in 4.823592 seconds) 2020-03-31 09:47:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:49 UTC)" (scheduled at 2020-03-31 09:47:49.498966+00:00) 2020-03-31 09:47:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:54.323352+00:00 (in 4.823592 seconds) 2020-03-31 09:47:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:49 UTC)" (scheduled at 2020-03-31 09:47:49.498966+00:00) 2020-03-31 09:47:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:49 UTC)" executed successfully 2020-03-31 09:47:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:47:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:59.560326+00:00 (in 8.448446 seconds) 2020-03-31 09:47:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:51 UTC)" (scheduled at 2020-03-31 09:47:51.110610+00:00) 2020-03-31 09:47:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:47:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:47:59.560326+00:00 (in 8.448446 seconds) 2020-03-31 09:47:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:51 UTC)" (scheduled at 2020-03-31 09:47:51.110610+00:00) 2020-03-31 09:47:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 51, 112748), True, datetime.datetime(2020, 3, 31, 9, 47, 51, 112748), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:47:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 51, 112748), True, datetime.datetime(2020, 3, 31, 9, 47, 51, 112748), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:47:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:47:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:47:51,130 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:51,130 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:51 UTC)" executed successfully 2020-03-31 09:47:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:47:51,442 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:01.440707+00:00 (in 9.998031 seconds) 2020-03-31 09:47:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:01 UTC)" (scheduled at 2020-03-31 09:47:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:47:51,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:01.440707+00:00 (in 9.998031 seconds) 2020-03-31 09:47:51,443 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:01 UTC)" (scheduled at 2020-03-31 09:47:51.440707+00:00) 2020-03-31 09:47:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:47:51,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:47:51,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:47:51,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:47:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:47:51,463 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:51,463 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:01 UTC)" executed successfully 2020-03-31 09:47:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:47:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:22.113221+00:00 (in 29.997556 seconds) 2020-03-31 09:47:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:22 UTC)" (scheduled at 2020-03-31 09:47:52.113221+00:00) securityworker stdout | 2020-03-31 09:47:52,115 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:22.113221+00:00 (in 29.997556 seconds) 2020-03-31 09:47:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:22 UTC)" (scheduled at 2020-03-31 09:47:52.113221+00:00) 2020-03-31 09:47:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:47:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:47:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:47:52,128 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:47:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:47:52,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:47:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:47:52,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:52,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:22 UTC)" executed successfully 2020-03-31 09:47:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:47:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:04.323352+00:00 (in 9.999084 seconds) 2020-03-31 09:47:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:04 UTC)" (scheduled at 2020-03-31 09:47:54.323352+00:00) 2020-03-31 09:47:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:47:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:04.323352+00:00 (in 9.999084 seconds) 2020-03-31 09:47:54,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:04 UTC)" (scheduled at 2020-03-31 09:47:54.323352+00:00) 2020-03-31 09:47:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 54, 325221), True, datetime.datetime(2020, 3, 31, 9, 47, 54, 325221), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:47:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 54, 325221), True, datetime.datetime(2020, 3, 31, 9, 47, 54, 325221), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:47:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:54,343 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:47:54,343 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:47:54,343 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:47:54,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:04 UTC)" executed successfully 2020-03-31 09:47:54,445 [518] [DEBUG] [app] Starting request: urn:request:bd995973-a8d8-449a-b658-5101cae6d1d2 (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:54,445 [518] [DEBUG] [app] Starting request: urn:request:bd995973-a8d8-449a-b658-5101cae6d1d2 (/health/instance) 2020-03-31 09:47:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:54,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:54,453 [552] [DEBUG] [app] Starting request: urn:request:dd929d06-211c-46e1-8156-b80282ba3698 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:54,453 [552] [DEBUG] [app] Starting request: urn:request:dd929d06-211c-46e1-8156-b80282ba3698 (/v1/_internal_ping) 2020-03-31 09:47:54,454 [552] [DEBUG] [app] Ending request: urn:request:dd929d06-211c-46e1-8156-b80282ba3698 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:54,454 [552] [DEBUG] [app] Ending request: urn:request:dd929d06-211c-46e1-8156-b80282ba3698 (/v1/_internal_ping) 2020-03-31 09:47:54,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:47:54,455 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:47:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:54,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:54,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:54,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:54,459 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:54,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:54,460 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:54,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:54,484 [518] [DEBUG] [app] Starting request: urn:request:8c5d1e18-c7a1-4441-9881-b66aab5d9d98 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:54,484 [518] [DEBUG] [app] Starting request: urn:request:8c5d1e18-c7a1-4441-9881-b66aab5d9d98 (/_internal_ping) 2020-03-31 09:47:54,485 [518] [DEBUG] [app] Ending request: urn:request:8c5d1e18-c7a1-4441-9881-b66aab5d9d98 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:54,485 [518] [DEBUG] [app] Ending request: urn:request:8c5d1e18-c7a1-4441-9881-b66aab5d9d98 (/_internal_ping) 2020-03-31 09:47:54,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:54,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:47:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:54,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:47:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:54,498 [532] [DEBUG] [app] Starting request: urn:request:249418f0-e7cf-4a93-88f5-8e369efe9167 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:54,498 [532] [DEBUG] [app] Starting request: urn:request:249418f0-e7cf-4a93-88f5-8e369efe9167 (/c1/_internal_ping) 2020-03-31 09:47:54,499 [532] [DEBUG] [app] Ending request: urn:request:249418f0-e7cf-4a93-88f5-8e369efe9167 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:54,499 [532] [DEBUG] [app] Ending request: urn:request:249418f0-e7cf-4a93-88f5-8e369efe9167 (/c1/_internal_ping) 2020-03-31 09:47:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:54,500 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:54,501 [518] [DEBUG] [app] Ending request: urn:request:bd995973-a8d8-449a-b658-5101cae6d1d2 (/health/instance) 2020-03-31 09:47:54,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:47:54,500 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:47:54,501 [518] [DEBUG] [app] Ending request: urn:request:bd995973-a8d8-449a-b658-5101cae6d1d2 (/health/instance) 2020-03-31 09:47:54,501 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:47:54,502 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:47:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:06.924497+00:00 (in 9.999322 seconds) 2020-03-31 09:47:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:06 UTC)" (scheduled at 2020-03-31 09:47:56.924497+00:00) 2020-03-31 09:47:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:47:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:47:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:06.924497+00:00 (in 9.999322 seconds) 2020-03-31 09:47:56,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:06 UTC)" (scheduled at 2020-03-31 09:47:56.924497+00:00) 2020-03-31 09:47:56,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:47:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 56, 925964), True, datetime.datetime(2020, 3, 31, 9, 47, 56, 925964), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:47:56,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 47, 56, 925964), True, datetime.datetime(2020, 3, 31, 9, 47, 56, 925964), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:47:56,942 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 7, 56, 925964), 1, u'2617ba0b-07ac-4247-86f7-fd1b9343fbdd', 3, u'300d19df-aa55-45a0-bfa0-0120629460e0']) storagereplication stdout | 2020-03-31 09:47:56,942 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 7, 56, 925964), 1, u'2617ba0b-07ac-4247-86f7-fd1b9343fbdd', 3, u'300d19df-aa55-45a0-bfa0-0120629460e0']) 2020-03-31 09:47:56,949 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} storagereplication stdout | 2020-03-31 09:47:56,949 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:47:56,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:56,949 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 storagereplication stdout | 2020-03-31 09:47:56,949 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:56,949 [191] [DEBUG] [__main__] Starting replication of image storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace 1 2020-03-31 09:47:56,951 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:47:56,951 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:47:56,967 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:47:56,967 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:47:56,971 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) storagereplication stdout | 2020-03-31 09:47:56,971 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8']) 2020-03-31 09:47:56,976 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:47:56,976 [191] [DEBUG] [__main__] For replication of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:47:56,976 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b32cb180-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:47:56 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:47:56,976 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b32cb180-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b32cb180-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gCG0PjfDae9HKvRPgirms7EbhuhJ2KIMobdYQ3HI4pQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:56 GMT'}. storagereplication stdout | 2020-03-31 09:47:56,976 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b32cb180-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:47:56 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:47:56,976 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b32cb180-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b32cb180-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gCG0PjfDae9HKvRPgirms7EbhuhJ2KIMobdYQ3HI4pQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:56 GMT'}. 2020-03-31 09:47:56,978 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:47:56,978 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:47:57,098 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:47:57,098 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:47:57,099 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b32cb180-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=dd737649-b01e-000c-2a41-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b32cb180-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gCG0PjfDae9HKvRPgirms7EbhuhJ2KIMobdYQ3HI4pQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:56 GMT'}. storagereplication stdout | 2020-03-31 09:47:57,099 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b32cb180-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=dd737649-b01e-000c-2a41-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b32cb180-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:gCG0PjfDae9HKvRPgirms7EbhuhJ2KIMobdYQ3HI4pQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:56 GMT'}. 2020-03-31 09:47:57,101 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:47:57,101 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:57,101 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:47:57,101 [191] [DEBUG] [__main__] Starting copy of storage 0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8 to location azure-uk from azure-us 2020-03-31 09:47:57,101 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:57,101 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:47:57,102 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:b33fdeb8-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=Ixj6SvJcWL2RAEHGn8sOulDX0kKSB9/LcLtFSlln03U%3D&sv=2017-07-29&se=2020-03-31T09%3A48%3A57Z x-ms-date:Tue, 31 Mar 2020 09:47:57 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:47:57,102 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b33fdeb8-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=Ixj6SvJcWL2RAEHGn8sOulDX0kKSB9/LcLtFSlln03U%3D&sv=2017-07-29&se=2020-03-31T09%3A48%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SjORJ7I0pcuMGsoo3PEDsTKwYM98qkVO5lxOTMqnQY0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'b33fdeb8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. storagereplication stdout | 2020-03-31 09:47:57,102 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:b33fdeb8-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=Ixj6SvJcWL2RAEHGn8sOulDX0kKSB9/LcLtFSlln03U%3D&sv=2017-07-29&se=2020-03-31T09%3A48%3A57Z x-ms-date:Tue, 31 Mar 2020 09:47:57 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 2020-03-31 09:47:57,102 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b33fdeb8-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=Ixj6SvJcWL2RAEHGn8sOulDX0kKSB9/LcLtFSlln03U%3D&sv=2017-07-29&se=2020-03-31T09%3A48%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SjORJ7I0pcuMGsoo3PEDsTKwYM98qkVO5lxOTMqnQY0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'b33fdeb8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. 2020-03-31 09:47:57,164 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:47:57,164 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 HTTP/1.1" 202 0 2020-03-31 09:47:57,165 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b33fdeb8-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=dd73764c-b01e-000c-2b41-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=Ixj6SvJcWL2RAEHGn8sOulDX0kKSB9/LcLtFSlln03U%3D&sv=2017-07-29&se=2020-03-31T09%3A48%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SjORJ7I0pcuMGsoo3PEDsTKwYM98qkVO5lxOTMqnQY0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'b33fdeb8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. storagereplication stdout | 2020-03-31 09:47:57,165 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b33fdeb8-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=dd73764c-b01e-000c-2b41-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?sr=b&sp=r&sig=Ixj6SvJcWL2RAEHGn8sOulDX0kKSB9/LcLtFSlln03U%3D&sv=2017-07-29&se=2020-03-31T09%3A48%3A57Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:SjORJ7I0pcuMGsoo3PEDsTKwYM98qkVO5lxOTMqnQY0=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'b33fdeb8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. 2020-03-31 09:47:57,166 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:47:57,166 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:47:57,166 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b349b028-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:47:57 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:47:57,166 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b349b028-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AGnSw6fD/kRErnK43OwBGLqFsUprlgJfToyHkVagkLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. storagereplication stdout | 2020-03-31 09:47:57,166 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b349b028-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:47:57 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:47:57,166 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b349b028-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AGnSw6fD/kRErnK43OwBGLqFsUprlgJfToyHkVagkLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. 2020-03-31 09:47:57,169 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net storagereplication stdout | 2020-03-31 09:47:57,169 [191] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: ukquaybackendbmwuskq7s7.blob.core.windows.net 2020-03-31 09:47:57,605 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:47:57,605 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:47:57,606 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=47778ee8-501e-0041-7a41-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b349b028-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AGnSw6fD/kRErnK43OwBGLqFsUprlgJfToyHkVagkLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. 2020-03-31 09:47:57,606 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=47778ee8-501e-0041-7a41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:57,606 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=47778ee8-501e-0041-7a41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:57,606 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:47:57,606 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=47778ee8-501e-0041-7a41-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b349b028-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AGnSw6fD/kRErnK43OwBGLqFsUprlgJfToyHkVagkLQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:57 GMT'}. 2020-03-31 09:47:57,606 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=47778ee8-501e-0041-7a41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:57,606 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b349b028-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:47:57 GMT, Server-Request-ID=47778ee8-501e-0041-7a41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:57,606 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:47:58,273 [518] [DEBUG] [app] Starting request: urn:request:1bd7e322-370e-4316-baf4-b713a9f4654f (/health/instance) gunicorn-web stdout | 2020-03-31 09:47:58,273 [518] [DEBUG] [app] Starting request: urn:request:1bd7e322-370e-4316-baf4-b713a9f4654f (/health/instance) 2020-03-31 09:47:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:58,278 [538] [DEBUG] [app] Starting request: urn:request:c10a6bb9-eabc-4597-8388-2d675a202a1f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:58,278 [538] [DEBUG] [app] Starting request: urn:request:c10a6bb9-eabc-4597-8388-2d675a202a1f (/v1/_internal_ping) 2020-03-31 09:47:58,278 [538] [DEBUG] [app] Ending request: urn:request:c10a6bb9-eabc-4597-8388-2d675a202a1f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:47:58,278 [538] [DEBUG] [app] Ending request: urn:request:c10a6bb9-eabc-4597-8388-2d675a202a1f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:47:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:47:58,279 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:47:58,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:58,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:47:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:47:58,282 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:47:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:47:58,283 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:47:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:47:58,284 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:47:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:47:58,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:47:58,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:47:58,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:47:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:58,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:58,311 [518] [DEBUG] [app] Starting request: urn:request:bccf26a2-d8f1-48ae-b04c-97b07e898c85 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:58,311 [518] [DEBUG] [app] Starting request: urn:request:bccf26a2-d8f1-48ae-b04c-97b07e898c85 (/_internal_ping) 2020-03-31 09:47:58,312 [518] [DEBUG] [app] Ending request: urn:request:bccf26a2-d8f1-48ae-b04c-97b07e898c85 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:47:58,312 [518] [DEBUG] [app] Ending request: urn:request:bccf26a2-d8f1-48ae-b04c-97b07e898c85 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:47:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:58,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:47:58,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:58,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:47:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:58,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:47:58,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:47:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:58,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.002 153 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.002 153 0.002) 2020-03-31 09:47:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:47:58,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:47:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:47:58,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:47:58,326 [532] [DEBUG] [app] Starting request: urn:request:5e262e76-a8e9-4457-8147-e7df3a1d2dec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:58,326 [532] [DEBUG] [app] Starting request: urn:request:5e262e76-a8e9-4457-8147-e7df3a1d2dec (/c1/_internal_ping) 2020-03-31 09:47:58,327 [532] [DEBUG] [app] Ending request: urn:request:5e262e76-a8e9-4457-8147-e7df3a1d2dec (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:47:58,327 [532] [DEBUG] [app] Ending request: urn:request:5e262e76-a8e9-4457-8147-e7df3a1d2dec (/c1/_internal_ping) 2020-03-31 09:47:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:47:58,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:47:58,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:47:58,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:47:58,329 [518] [DEBUG] [app] Ending request: urn:request:1bd7e322-370e-4316-baf4-b713a9f4654f (/health/instance) 2020-03-31 09:47:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:47:58,329 [518] [DEBUG] [app] Ending request: urn:request:1bd7e322-370e-4316-baf4-b713a9f4654f (/health/instance) 2020-03-31 09:47:58,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:47:58,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:47:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:47:58,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:47:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:47:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:47:59,560 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:47:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:51.110610+00:00 (in 51.549533 seconds) 2020-03-31 09:47:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:59 UTC)" (scheduled at 2020-03-31 09:47:59.560326+00:00) 2020-03-31 09:47:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:47:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:51.110610+00:00 (in 51.549533 seconds) 2020-03-31 09:47:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:59 UTC)" (scheduled at 2020-03-31 09:47:59.560326+00:00) 2020-03-31 09:47:59,561 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:47:59,561 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:47:59,561 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:48:59 UTC)" executed successfully 2020-03-31 09:47:59,608 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b4be4fc2-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:47:59 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:47:59,608 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b4be4fc2-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qbugxVVEUlc1JgAuhJbeKmo0usGkRReIeR+uXH+tSSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:59 GMT'}. storagereplication stdout | 2020-03-31 09:47:59,608 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b4be4fc2-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:47:59 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:47:59,608 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b4be4fc2-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qbugxVVEUlc1JgAuhJbeKmo0usGkRReIeR+uXH+tSSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:59 GMT'}. 2020-03-31 09:47:59,688 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:47:59,688 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:47:59,689 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:59 GMT, Server-Request-ID=47779739-501e-0041-6c41-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b4be4fc2-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qbugxVVEUlc1JgAuhJbeKmo0usGkRReIeR+uXH+tSSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:59 GMT'}. 2020-03-31 09:47:59,689 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:47:59 GMT, Server-Request-ID=47779739-501e-0041-6c41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:59,689 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:47:59 GMT, Server-Request-ID=47779739-501e-0041-6c41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:59,689 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:47:59,689 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:47:59 GMT, Server-Request-ID=47779739-501e-0041-6c41-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b4be4fc2-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qbugxVVEUlc1JgAuhJbeKmo0usGkRReIeR+uXH+tSSk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:47:59 GMT'}. 2020-03-31 09:47:59,689 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:47:59 GMT, Server-Request-ID=47779739-501e-0041-6c41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:59,689 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b4be4fc2-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:47:59 GMT, Server-Request-ID=47779739-501e-0041-6c41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:47:59,689 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:48:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:48:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:30.337699+00:00 (in 29.999296 seconds) 2020-03-31 09:48:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:30 UTC)" (scheduled at 2020-03-31 09:48:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:48:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:30.337699+00:00 (in 29.999296 seconds) 2020-03-31 09:48:00,338 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:30 UTC)" (scheduled at 2020-03-31 09:48:00.337699+00:00) 2020-03-31 09:48:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 48, 0, 339159), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:48:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 48, 0, 339159), False, 50, 1, 0]) 2020-03-31 09:48:00,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:48:00,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:48:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:48:00,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:00,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:30 UTC)" executed successfully 2020-03-31 09:48:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:48:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:11.440707+00:00 (in 9.999064 seconds) 2020-03-31 09:48:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:11 UTC)" (scheduled at 2020-03-31 09:48:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:48:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:11.440707+00:00 (in 9.999064 seconds) 2020-03-31 09:48:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:11 UTC)" (scheduled at 2020-03-31 09:48:01.440707+00:00) 2020-03-31 09:48:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:48:01,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:48:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:48:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:48:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:48:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:11 UTC)" executed successfully 2020-03-31 09:48:03,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:48:03,333 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:03,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:03 UTC)" (scheduled at 2020-03-31 09:48:03.330112+00:00) 2020-03-31 09:48:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:48:03,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:03 UTC)" (scheduled at 2020-03-31 09:48:03.330112+00:00) 2020-03-31 09:48:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:03,336 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:24.410927+00:00 (in 21.077655 seconds) security_notification_worker stdout | 2020-03-31 09:48:03,336 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:24.410927+00:00 (in 21.077655 seconds) 2020-03-31 09:48:03,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 3, 334047), True, datetime.datetime(2020, 3, 31, 9, 48, 3, 334047), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:48:03,337 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 3, 334047), True, datetime.datetime(2020, 3, 31, 9, 48, 3, 334047), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:48:03,356 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:48:03,356 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:03,356 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:03,357 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:48:03,356 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:03,357 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:33 UTC)" executed successfully 2020-03-31 09:48:03,693 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b72d9f24-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:48:03,693 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:b72d9f24-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:03 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:48:03,694 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b72d9f24-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:b1OLcgArXZ79Pa1KkrMyKgIU0JSEBEbPGFlxZ47GZq4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:03 GMT'}. storagereplication stdout | 2020-03-31 09:48:03,694 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b72d9f24-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:b1OLcgArXZ79Pa1KkrMyKgIU0JSEBEbPGFlxZ47GZq4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:03 GMT'}. 2020-03-31 09:48:03,773 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:48:03,773 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:48:03,774 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:03 GMT, Server-Request-ID=4777a829-501e-0041-1541-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b72d9f24-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:b1OLcgArXZ79Pa1KkrMyKgIU0JSEBEbPGFlxZ47GZq4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:03 GMT'}. 2020-03-31 09:48:03,775 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:03 GMT, Server-Request-ID=4777a829-501e-0041-1541-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:48:03,774 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:03 GMT, Server-Request-ID=4777a829-501e-0041-1541-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'b72d9f24-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:b1OLcgArXZ79Pa1KkrMyKgIU0JSEBEbPGFlxZ47GZq4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:03 GMT'}. 2020-03-31 09:48:03,775 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:03 GMT, Server-Request-ID=4777a829-501e-0041-1541-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:03,775 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:03 GMT, Server-Request-ID=4777a829-501e-0041-1541-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:03,775 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:48:03,775 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=b72d9f24-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:03 GMT, Server-Request-ID=4777a829-501e-0041-1541-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:03,775 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:48:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:48:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:14.323352+00:00 (in 9.998880 seconds) 2020-03-31 09:48:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:14 UTC)" (scheduled at 2020-03-31 09:48:04.323352+00:00) 2020-03-31 09:48:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:48:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:14.323352+00:00 (in 9.998880 seconds) 2020-03-31 09:48:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:14 UTC)" (scheduled at 2020-03-31 09:48:04.323352+00:00) 2020-03-31 09:48:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 4, 325571), True, datetime.datetime(2020, 3, 31, 9, 48, 4, 325571), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:48:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 4, 325571), True, datetime.datetime(2020, 3, 31, 9, 48, 4, 325571), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:48:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:48:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:48:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:14 UTC)" executed successfully 2020-03-31 09:48:04,446 [518] [DEBUG] [app] Starting request: urn:request:14348e5f-059e-4bb7-9c0d-0eab359ccb22 (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:04,446 [518] [DEBUG] [app] Starting request: urn:request:14348e5f-059e-4bb7-9c0d-0eab359ccb22 (/health/instance) 2020-03-31 09:48:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:04,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:04,459 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:04,459 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:04,460 [538] [DEBUG] [app] Starting request: urn:request:5e698256-fef8-4dcc-a41b-ac324eff646d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:04,460 [538] [DEBUG] [app] Starting request: urn:request:5e698256-fef8-4dcc-a41b-ac324eff646d (/v1/_internal_ping) 2020-03-31 09:48:04,461 [538] [DEBUG] [app] Ending request: urn:request:5e698256-fef8-4dcc-a41b-ac324eff646d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:04,461 [538] [DEBUG] [app] Ending request: urn:request:5e698256-fef8-4dcc-a41b-ac324eff646d (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:04,462 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:04,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:04,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:04,464 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:04,462 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:04,462 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:04,465 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:48:04,463 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:04,466 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:04,464 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:04,465 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:04,466 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:04,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:04,475 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:04,493 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:04,493 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:04,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:04,497 [518] [DEBUG] [app] Starting request: urn:request:bb14bab3-f4c0-4bbd-8a98-8f67fc24f624 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:04,497 [518] [DEBUG] [app] Starting request: urn:request:bb14bab3-f4c0-4bbd-8a98-8f67fc24f624 (/_internal_ping) 2020-03-31 09:48:04,498 [518] [DEBUG] [app] Ending request: urn:request:bb14bab3-f4c0-4bbd-8a98-8f67fc24f624 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:04,498 [518] [DEBUG] [app] Ending request: urn:request:bb14bab3-f4c0-4bbd-8a98-8f67fc24f624 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:48:04,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:04,499 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:48:04,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:04,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:48:04,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:48:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:04,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:48:04,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:48:04,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:48:04,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:04,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:04,514 [537] [DEBUG] [app] Starting request: urn:request:b7f2834b-3315-4236-af3e-d46f7d0503c3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:04,514 [537] [DEBUG] [app] Starting request: urn:request:b7f2834b-3315-4236-af3e-d46f7d0503c3 (/c1/_internal_ping) 2020-03-31 09:48:04,515 [537] [DEBUG] [app] Ending request: urn:request:b7f2834b-3315-4236-af3e-d46f7d0503c3 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:04,515 [537] [DEBUG] [app] Ending request: urn:request:b7f2834b-3315-4236-af3e-d46f7d0503c3 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:04,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:04,516 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:04,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:48:04,516 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:04,517 [518] [DEBUG] [app] Ending request: urn:request:14348e5f-059e-4bb7-9c0d-0eab359ccb22 (/health/instance) 2020-03-31 09:48:04,517 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:04,517 [518] [DEBUG] [app] Ending request: urn:request:14348e5f-059e-4bb7-9c0d-0eab359ccb22 (/health/instance) 2020-03-31 09:48:04,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:04,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:48:04,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:48:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:16.924497+00:00 (in 9.998995 seconds) storagereplication stdout | 2020-03-31 09:48:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:48:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:16.924497+00:00 (in 9.998995 seconds) 2020-03-31 09:48:08,273 [518] [DEBUG] [app] Starting request: urn:request:a07c7cf3-30ce-476c-ade4-56dbc935cb1f (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:08,273 [518] [DEBUG] [app] Starting request: urn:request:a07c7cf3-30ce-476c-ade4-56dbc935cb1f (/health/instance) 2020-03-31 09:48:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:08,278 [548] [DEBUG] [app] Starting request: urn:request:057e57bc-5b99-4a90-89c7-11ecea9347de (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:08,278 [548] [DEBUG] [app] Starting request: urn:request:057e57bc-5b99-4a90-89c7-11ecea9347de (/v1/_internal_ping) 2020-03-31 09:48:08,279 [548] [DEBUG] [app] Ending request: urn:request:057e57bc-5b99-4a90-89c7-11ecea9347de (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:08,279 [548] [DEBUG] [app] Ending request: urn:request:057e57bc-5b99-4a90-89c7-11ecea9347de (/v1/_internal_ping) 2020-03-31 09:48:08,280 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:48:08,280 [548] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:08,283 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:48:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:08,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:08,283 [548] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:08,284 [548] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:08,285 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:08,285 [548] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:08,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:08,307 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:08,313 [526] [DEBUG] [app] Starting request: urn:request:e147e5c6-e73a-4f46-b720-c86030d655c6 (/_internal_ping) 2020-03-31 09:48:08,314 [526] [DEBUG] [app] Ending request: urn:request:e147e5c6-e73a-4f46-b720-c86030d655c6 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:08,311 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:08,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:48:08,313 [526] [DEBUG] [app] Starting request: urn:request:e147e5c6-e73a-4f46-b720-c86030d655c6 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:08,314 [526] [DEBUG] [app] Ending request: urn:request:e147e5c6-e73a-4f46-b720-c86030d655c6 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:08,314 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:08,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:48:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:48:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:48:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:08,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:08,329 [537] [DEBUG] [app] Starting request: urn:request:1a30c026-2e38-4e6f-a5f9-0a200071d9e8 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:08,329 [537] [DEBUG] [app] Starting request: urn:request:1a30c026-2e38-4e6f-a5f9-0a200071d9e8 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:08,330 [537] [DEBUG] [app] Ending request: urn:request:1a30c026-2e38-4e6f-a5f9-0a200071d9e8 (/c1/_internal_ping) 2020-03-31 09:48:08,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:48:08,330 [537] [DEBUG] [app] Ending request: urn:request:1a30c026-2e38-4e6f-a5f9-0a200071d9e8 (/c1/_internal_ping) 2020-03-31 09:48:08,331 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:08,331 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:08,333 [518] [DEBUG] [app] Ending request: urn:request:a07c7cf3-30ce-476c-ade4-56dbc935cb1f (/health/instance) 2020-03-31 09:48:08,333 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:08,333 [518] [DEBUG] [app] Ending request: urn:request:a07c7cf3-30ce-476c-ade4-56dbc935cb1f (/health/instance) 2020-03-31 09:48:08,333 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:48:08,333 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:48:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:48:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:21.440707+00:00 (in 9.999259 seconds) 2020-03-31 09:48:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:21 UTC)" (scheduled at 2020-03-31 09:48:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:48:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:21.440707+00:00 (in 9.999259 seconds) 2020-03-31 09:48:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:21 UTC)" (scheduled at 2020-03-31 09:48:11.440707+00:00) 2020-03-31 09:48:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:48:11,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:48:11,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:48:11,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:11,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:48:11,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:48:11,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:11,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:21 UTC)" executed successfully 2020-03-31 09:48:11,779 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:bbff56dc-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:11 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 storagereplication stdout | 2020-03-31 09:48:11,779 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:bbff56dc-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:11 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29 timeout:20 2020-03-31 09:48:11,779 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bbff56dc-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:96KYxZCqK6Rq6CUgJBVSKMru54rKmXc7tDaxZxvjT38=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:11 GMT'}. storagereplication stdout | 2020-03-31 09:48:11,779 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bbff56dc-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:96KYxZCqK6Rq6CUgJBVSKMru54rKmXc7tDaxZxvjT38=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:11 GMT'}. 2020-03-31 09:48:11,860 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:48:11,860 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:48:11,860 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:11 GMT, Server-Request-ID=4777c88f-501e-0041-7641-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bbff56dc-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:96KYxZCqK6Rq6CUgJBVSKMru54rKmXc7tDaxZxvjT38=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:11 GMT'}. 2020-03-31 09:48:11,861 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:11 GMT, Server-Request-ID=4777c88f-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:11,861 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:11 GMT, Server-Request-ID=4777c88f-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:48:11,860 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:11 GMT, Server-Request-ID=4777c88f-501e-0041-7641-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'bbff56dc-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:96KYxZCqK6Rq6CUgJBVSKMru54rKmXc7tDaxZxvjT38=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:11 GMT'}. 2020-03-31 09:48:11,861 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:11 GMT, Server-Request-ID=4777c88f-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:11,861 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=bbff56dc-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:11 GMT, Server-Request-ID=4777c88f-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:11,861 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:48:11,861 [191] [DEBUG] [__main__] Cannot find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:48:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:48:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:24.323352+00:00 (in 9.998605 seconds) 2020-03-31 09:48:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:24 UTC)" (scheduled at 2020-03-31 09:48:14.323352+00:00) 2020-03-31 09:48:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:48:14,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:24.323352+00:00 (in 9.998605 seconds) 2020-03-31 09:48:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:24 UTC)" (scheduled at 2020-03-31 09:48:14.323352+00:00) 2020-03-31 09:48:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 14, 325536), True, datetime.datetime(2020, 3, 31, 9, 48, 14, 325536), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:48:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 14, 325536), True, datetime.datetime(2020, 3, 31, 9, 48, 14, 325536), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:48:14,349 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:48:14,349 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:14,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:14,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:48:14,349 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:14,349 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:24 UTC)" executed successfully 2020-03-31 09:48:14,446 [527] [DEBUG] [app] Starting request: urn:request:0f6ad4bd-ecf5-405e-9fd1-1df31be8f396 (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:14,446 [527] [DEBUG] [app] Starting request: urn:request:0f6ad4bd-ecf5-405e-9fd1-1df31be8f396 (/health/instance) 2020-03-31 09:48:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:14,453 [553] [DEBUG] [app] Starting request: urn:request:767d3eef-7dd1-4994-acac-d65947c39c4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:14,453 [553] [DEBUG] [app] Starting request: urn:request:767d3eef-7dd1-4994-acac-d65947c39c4c (/v1/_internal_ping) 2020-03-31 09:48:14,454 [553] [DEBUG] [app] Ending request: urn:request:767d3eef-7dd1-4994-acac-d65947c39c4c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:14,454 [553] [DEBUG] [app] Ending request: urn:request:767d3eef-7dd1-4994-acac-d65947c39c4c (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:48:14,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:48:14,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:48:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:14,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:14,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:14,459 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:14,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:14,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:14,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:14,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:14,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:14,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:14,487 [527] [DEBUG] [app] Starting request: urn:request:ba94753b-b620-4a0b-8007-ac16c0f9d693 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:14,487 [527] [DEBUG] [app] Starting request: urn:request:ba94753b-b620-4a0b-8007-ac16c0f9d693 (/_internal_ping) 2020-03-31 09:48:14,488 [527] [DEBUG] [app] Ending request: urn:request:ba94753b-b620-4a0b-8007-ac16c0f9d693 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:14,488 [527] [DEBUG] [app] Ending request: urn:request:ba94753b-b620-4a0b-8007-ac16c0f9d693 (/_internal_ping) 2020-03-31 09:48:14,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:14,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:14,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:14,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:14,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:14,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:14,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:48:14,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:48:14,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:14,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:48:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:48:14,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:48:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:14,499 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:14,500 [537] [DEBUG] [app] Starting request: urn:request:237fd93f-341a-435c-b5ab-d19fd5fd827d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:14,500 [537] [DEBUG] [app] Starting request: urn:request:237fd93f-341a-435c-b5ab-d19fd5fd827d (/c1/_internal_ping) 2020-03-31 09:48:14,501 [537] [DEBUG] [app] Ending request: urn:request:237fd93f-341a-435c-b5ab-d19fd5fd827d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:14,501 [537] [DEBUG] [app] Ending request: urn:request:237fd93f-341a-435c-b5ab-d19fd5fd827d (/c1/_internal_ping) 2020-03-31 09:48:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:48:14,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:14,501 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:48:14,503 [527] [DEBUG] [app] Ending request: urn:request:0f6ad4bd-ecf5-405e-9fd1-1df31be8f396 (/health/instance) 2020-03-31 09:48:14,504 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:48:14,501 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:48:14,503 [527] [DEBUG] [app] Ending request: urn:request:0f6ad4bd-ecf5-405e-9fd1-1df31be8f396 (/health/instance) 2020-03-31 09:48:14,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:14,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:48:14,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:48:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:48:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:45.064514+00:00 (in 29.999075 seconds) 2020-03-31 09:48:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:45 UTC)" (scheduled at 2020-03-31 09:48:15.064514+00:00) gcworker stdout | 2020-03-31 09:48:15,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:45.064514+00:00 (in 29.999075 seconds) 2020-03-31 09:48:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:45 UTC)" (scheduled at 2020-03-31 09:48:15.064514+00:00) 2020-03-31 09:48:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438495, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:48:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438495, 1209600, 500, 1, 0]) 2020-03-31 09:48:15,086 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:48:15,086 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:48:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:48:15,086 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:15,087 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:45 UTC)" executed successfully 2020-03-31 09:48:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:48:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:26.924497+00:00 (in 9.999234 seconds) storagereplication stdout | 2020-03-31 09:48:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:26.924497+00:00 (in 9.999234 seconds) 2020-03-31 09:48:18,273 [518] [DEBUG] [app] Starting request: urn:request:f1a27943-a0f5-4127-9703-6a85f64255bd (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:18,273 [518] [DEBUG] [app] Starting request: urn:request:f1a27943-a0f5-4127-9703-6a85f64255bd (/health/instance) 2020-03-31 09:48:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:18,280 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:18,281 [556] [DEBUG] [app] Starting request: urn:request:26515469-36f3-492f-afb9-7b8bcbcafd73 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:18,281 [556] [DEBUG] [app] Starting request: urn:request:26515469-36f3-492f-afb9-7b8bcbcafd73 (/v1/_internal_ping) 2020-03-31 09:48:18,283 [556] [DEBUG] [app] Ending request: urn:request:26515469-36f3-492f-afb9-7b8bcbcafd73 (/v1/_internal_ping) 2020-03-31 09:48:18,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:48:18,283 [556] [DEBUG] [app] Ending request: urn:request:26515469-36f3-492f-afb9-7b8bcbcafd73 (/v1/_internal_ping) 2020-03-31 09:48:18,283 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:48:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:18,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:18,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:18,286 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:18,288 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:18,288 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:18,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:18,288 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:18,296 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:18,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:18,315 [518] [DEBUG] [app] Starting request: urn:request:b8a2b486-71b8-4963-a519-a6c23c0c8643 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:18,315 [518] [DEBUG] [app] Starting request: urn:request:b8a2b486-71b8-4963-a519-a6c23c0c8643 (/_internal_ping) 2020-03-31 09:48:18,316 [518] [DEBUG] [app] Ending request: urn:request:b8a2b486-71b8-4963-a519-a6c23c0c8643 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:18,316 [518] [DEBUG] [app] Ending request: urn:request:b8a2b486-71b8-4963-a519-a6c23c0c8643 (/_internal_ping) 2020-03-31 09:48:18,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:18,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:18,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:18,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:18,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:48:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:48:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:48:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:18,330 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:18,332 [543] [DEBUG] [app] Starting request: urn:request:435ca2ab-25a8-4275-893f-b35d8f3af688 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:18,332 [543] [DEBUG] [app] Starting request: urn:request:435ca2ab-25a8-4275-893f-b35d8f3af688 (/c1/_internal_ping) 2020-03-31 09:48:18,333 [543] [DEBUG] [app] Ending request: urn:request:435ca2ab-25a8-4275-893f-b35d8f3af688 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:18,333 [543] [DEBUG] [app] Ending request: urn:request:435ca2ab-25a8-4275-893f-b35d8f3af688 (/c1/_internal_ping) 2020-03-31 09:48:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:18,334 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:18,334 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:18,335 [518] [DEBUG] [app] Ending request: urn:request:f1a27943-a0f5-4127-9703-6a85f64255bd (/health/instance) 2020-03-31 09:48:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:48:18,334 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:48:18,335 [518] [DEBUG] [app] Ending request: urn:request:f1a27943-a0f5-4127-9703-6a85f64255bd (/health/instance) 2020-03-31 09:48:18,335 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:18,336 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:48:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:48:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:31 UTC)" (scheduled at 2020-03-31 09:48:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:48:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:31 UTC)" (scheduled at 2020-03-31 09:48:21.440707+00:00) 2020-03-31 09:48:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:31.440707+00:00 (in 9.998841 seconds) repositoryactioncounter stdout | 2020-03-31 09:48:21,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:31.440707+00:00 (in 9.998841 seconds) 2020-03-31 09:48:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:48:21,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:48:21,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:48:21,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:48:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:48:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:31 UTC)" executed successfully 2020-03-31 09:48:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:48:22,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:52.113221+00:00 (in 29.999156 seconds) 2020-03-31 09:48:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:52 UTC)" (scheduled at 2020-03-31 09:48:22.113221+00:00) securityworker stdout | 2020-03-31 09:48:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:52.113221+00:00 (in 29.999156 seconds) 2020-03-31 09:48:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:52 UTC)" (scheduled at 2020-03-31 09:48:22.113221+00:00) 2020-03-31 09:48:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:48:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:48:22,126 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:48:22,126 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:48:22,128 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:48:22,128 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:48:22,128 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:22,128 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:48:22,128 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:22,128 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:48:52 UTC)" executed successfully 2020-03-31 09:48:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:48:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:34.323352+00:00 (in 9.998938 seconds) 2020-03-31 09:48:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:34 UTC)" (scheduled at 2020-03-31 09:48:24.323352+00:00) 2020-03-31 09:48:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:48:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:34.323352+00:00 (in 9.998938 seconds) 2020-03-31 09:48:24,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:34 UTC)" (scheduled at 2020-03-31 09:48:24.323352+00:00) 2020-03-31 09:48:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 24, 325277), True, datetime.datetime(2020, 3, 31, 9, 48, 24, 325277), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:48:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 24, 325277), True, datetime.datetime(2020, 3, 31, 9, 48, 24, 325277), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:48:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:24,341 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:48:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:48:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:34 UTC)" executed successfully 2020-03-31 09:48:24,415 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:48:24,415 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:24,415 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:24 UTC)" (scheduled at 2020-03-31 09:48:24.410927+00:00) 2020-03-31 09:48:24,416 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:24,416 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:48:24,415 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:24 UTC)" (scheduled at 2020-03-31 09:48:24.410927+00:00) 2020-03-31 09:48:24,416 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:24,416 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:24 UTC)" executed successfully 2020-03-31 09:48:24,416 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:33.330112+00:00 (in 8.914565 seconds) security_notification_worker stdout | 2020-03-31 09:48:24,416 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:33.330112+00:00 (in 8.914565 seconds) 2020-03-31 09:48:24,446 [518] [DEBUG] [app] Starting request: urn:request:08c47168-ad80-497c-9c64-6f6ad3d55852 (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:24,446 [518] [DEBUG] [app] Starting request: urn:request:08c47168-ad80-497c-9c64-6f6ad3d55852 (/health/instance) 2020-03-31 09:48:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:24,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:24,453 [545] [DEBUG] [app] Starting request: urn:request:9bc64e36-353f-4652-9204-62ca989e3883 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:24,453 [545] [DEBUG] [app] Starting request: urn:request:9bc64e36-353f-4652-9204-62ca989e3883 (/v1/_internal_ping) 2020-03-31 09:48:24,454 [545] [DEBUG] [app] Ending request: urn:request:9bc64e36-353f-4652-9204-62ca989e3883 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:24,454 [545] [DEBUG] [app] Ending request: urn:request:9bc64e36-353f-4652-9204-62ca989e3883 (/v1/_internal_ping) 2020-03-31 09:48:24,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:48:24,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:48:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:48:24,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:48:24,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:24,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:24,457 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:24,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:24,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:24,459 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:24,459 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:24,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:24,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:24,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:24,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:24,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:24,486 [518] [DEBUG] [app] Starting request: urn:request:c0b901c0-a739-40a0-9dd1-894f02ca0bb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:24,486 [518] [DEBUG] [app] Starting request: urn:request:c0b901c0-a739-40a0-9dd1-894f02ca0bb2 (/_internal_ping) 2020-03-31 09:48:24,487 [518] [DEBUG] [app] Ending request: urn:request:c0b901c0-a739-40a0-9dd1-894f02ca0bb2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:24,487 [518] [DEBUG] [app] Ending request: urn:request:c0b901c0-a739-40a0-9dd1-894f02ca0bb2 (/_internal_ping) 2020-03-31 09:48:24,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:24,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:24,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:24,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:24,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:48:24,492 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:48:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:48:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:48:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:24,502 [543] [DEBUG] [app] Starting request: urn:request:f6c0eada-e943-48f4-89b0-ecb43ffa2b84 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:24,502 [543] [DEBUG] [app] Starting request: urn:request:f6c0eada-e943-48f4-89b0-ecb43ffa2b84 (/c1/_internal_ping) 2020-03-31 09:48:24,503 [543] [DEBUG] [app] Ending request: urn:request:f6c0eada-e943-48f4-89b0-ecb43ffa2b84 (/c1/_internal_ping) 2020-03-31 09:48:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:24,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:48:24,503 [543] [DEBUG] [app] Ending request: urn:request:f6c0eada-e943-48f4-89b0-ecb43ffa2b84 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:24,505 [518] [DEBUG] [app] Ending request: urn:request:08c47168-ad80-497c-9c64-6f6ad3d55852 (/health/instance) 2020-03-31 09:48:24,506 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-verbs stdout | 2020-03-31 09:48:24,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:48:24,505 [518] [DEBUG] [app] Ending request: urn:request:08c47168-ad80-497c-9c64-6f6ad3d55852 (/health/instance) 2020-03-31 09:48:24,506 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:24,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:48:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:48:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:36.924497+00:00 (in 9.999229 seconds) storagereplication stdout | 2020-03-31 09:48:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:36.924497+00:00 (in 9.999229 seconds) 2020-03-31 09:48:27,869 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:48:27,869 [191] [WARNING] [__main__] Failed to find path `sha256/e4/e4c9c050b32779acbfd4dca6bb92e55e60f1ff16e3d088ff5998d952b75b4c29` in location `azure-uk` after copy 2020-03-31 09:48:27,869 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:48:27,869 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:48:27,869 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "0cb8b4be-420f-4dcf-8b5a-179b7c2b13d8"} 2020-03-31 09:48:27,869 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:48:27,878 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) storagereplication stdout | 2020-03-31 09:48:27,878 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [3, 1, 0]) 2020-03-31 09:48:27,882 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 53, 27, 877658), True, u'86847241-9322-4aab-af70-a3bbaca894aa', 3]) storagereplication stdout | 2020-03-31 09:48:27,882 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 53, 27, 877658), True, u'86847241-9322-4aab-af70-a3bbaca894aa', 3]) 2020-03-31 09:48:27,888 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 27, 886856), True, datetime.datetime(2020, 3, 31, 9, 48, 27, 886856), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:48:27,888 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 27, 886856), True, datetime.datetime(2020, 3, 31, 9, 48, 27, 886856), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:48:27,892 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 8, 27, 886856), 1, u'a3c9096a-fa5f-4112-92b9-7470ea34b71b', 4, u'1ce6d63c-596b-4a6b-b342-736d881feb7c']) storagereplication stdout | 2020-03-31 09:48:27,892 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 8, 27, 886856), 1, u'a3c9096a-fa5f-4112-92b9-7470ea34b71b', 4, u'1ce6d63c-596b-4a6b-b342-736d881feb7c']) 2020-03-31 09:48:27,898 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} storagereplication stdout | 2020-03-31 09:48:27,898 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:48:27,898 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:27,898 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 storagereplication stdout | 2020-03-31 09:48:27,898 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:27,898 [191] [DEBUG] [__main__] Starting replication of image storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace 1 2020-03-31 09:48:27,900 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:48:27,900 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:48:27,912 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:48:27,912 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:48:27,915 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) storagereplication stdout | 2020-03-31 09:48:27,915 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'229ce905-1640-4594-aae8-fb66fcc5e51f']) 2020-03-31 09:48:27,919 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:48:27,919 [191] [DEBUG] [__main__] For replication of storage 229ce905-1640-4594-aae8-fb66fcc5e51f under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:48:27,919 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c59e3d8e-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:27 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:48:27,920 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c59e3d8e-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c59e3d8e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cj0eiLC4iZ1TkM1r/SlotaV3cM3SVphe4wTImaBPrQc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. storagereplication stdout | 2020-03-31 09:48:27,919 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c59e3d8e-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:27 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:48:27,920 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c59e3d8e-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c59e3d8e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cj0eiLC4iZ1TkM1r/SlotaV3cM3SVphe4wTImaBPrQc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. 2020-03-31 09:48:27,929 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:48:27,929 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:48:27,930 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c59e3d8e-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:27 GMT, Server-Request-ID=dd737fc9-b01e-000c-3141-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c59e3d8e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cj0eiLC4iZ1TkM1r/SlotaV3cM3SVphe4wTImaBPrQc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. storagereplication stdout | 2020-03-31 09:48:27,930 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c59e3d8e-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:27 GMT, Server-Request-ID=dd737fc9-b01e-000c-3141-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c59e3d8e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:cj0eiLC4iZ1TkM1r/SlotaV3cM3SVphe4wTImaBPrQc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. 2020-03-31 09:48:27,932 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us 2020-03-31 09:48:27,932 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:48:27,932 [191] [DEBUG] [__main__] Starting copy of storage 229ce905-1640-4594-aae8-fb66fcc5e51f to location azure-uk from azure-us 2020-03-31 09:48:27,932 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:27,932 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:48:27,933 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:c5a04e58-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=ETOgkFKnnEcm5D15iHixEy3dJHI5U3ajDmoxXSF1Kew%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A27Z x-ms-date:Tue, 31 Mar 2020 09:48:27 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 storagereplication stdout | 2020-03-31 09:48:27,932 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:48:27,933 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:c5a04e58-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=ETOgkFKnnEcm5D15iHixEy3dJHI5U3ajDmoxXSF1Kew%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A27Z x-ms-date:Tue, 31 Mar 2020 09:48:27 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 2020-03-31 09:48:27,933 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a04e58-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=ETOgkFKnnEcm5D15iHixEy3dJHI5U3ajDmoxXSF1Kew%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A27Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:V7QEt5jz3MkEXBu5xO7CS4Y7U9cfRkDHm7fMp9qpIzk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'c5a04e58-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. storagereplication stdout | 2020-03-31 09:48:27,933 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a04e58-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=ETOgkFKnnEcm5D15iHixEy3dJHI5U3ajDmoxXSF1Kew%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A27Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:V7QEt5jz3MkEXBu5xO7CS4Y7U9cfRkDHm7fMp9qpIzk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'c5a04e58-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. 2020-03-31 09:48:27,954 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:48:27,954 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 HTTP/1.1" 202 0 2020-03-31 09:48:27,955 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a04e58-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:27 GMT, Server-Request-ID=dd737fce-b01e-000c-3341-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=ETOgkFKnnEcm5D15iHixEy3dJHI5U3ajDmoxXSF1Kew%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A27Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:V7QEt5jz3MkEXBu5xO7CS4Y7U9cfRkDHm7fMp9qpIzk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'c5a04e58-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. storagereplication stdout | 2020-03-31 09:48:27,955 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a04e58-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:27 GMT, Server-Request-ID=dd737fce-b01e-000c-3341-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?sr=b&sp=r&sig=ETOgkFKnnEcm5D15iHixEy3dJHI5U3ajDmoxXSF1Kew%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A27Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:V7QEt5jz3MkEXBu5xO7CS4Y7U9cfRkDHm7fMp9qpIzk=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'c5a04e58-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. 2020-03-31 09:48:27,956 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:48:27,956 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c5a3da3c-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:27 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:48:27,956 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c5a3da3c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:52ElBnvpkJB9ptcUdCmVMVO03vIxkIm6m28iQjc1TYY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. storagereplication stdout | 2020-03-31 09:48:27,956 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:48:27,956 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c5a3da3c-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:27 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:48:27,956 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c5a3da3c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:52ElBnvpkJB9ptcUdCmVMVO03vIxkIm6m28iQjc1TYY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. 2020-03-31 09:48:28,038 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:48:28,038 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:48:28,039 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:28 GMT, Server-Request-ID=4778029e-501e-0041-7741-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c5a3da3c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:52ElBnvpkJB9ptcUdCmVMVO03vIxkIm6m28iQjc1TYY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. 2020-03-31 09:48:28,040 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:28 GMT, Server-Request-ID=4778029e-501e-0041-7741-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:28,040 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:28 GMT, Server-Request-ID=4778029e-501e-0041-7741-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:48:28,039 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:28 GMT, Server-Request-ID=4778029e-501e-0041-7741-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c5a3da3c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:52ElBnvpkJB9ptcUdCmVMVO03vIxkIm6m28iQjc1TYY=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:27 GMT'}. 2020-03-31 09:48:28,040 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:28 GMT, Server-Request-ID=4778029e-501e-0041-7741-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:28,040 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c5a3da3c-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:28 GMT, Server-Request-ID=4778029e-501e-0041-7741-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:28,040 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:48:28,040 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:48:28,273 [518] [DEBUG] [app] Starting request: urn:request:b0d4422e-c93c-43a4-848c-fbb3f6ea90de (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:28,273 [518] [DEBUG] [app] Starting request: urn:request:b0d4422e-c93c-43a4-848c-fbb3f6ea90de (/health/instance) 2020-03-31 09:48:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:28,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:28,278 [552] [DEBUG] [app] Starting request: urn:request:2f6134d3-f6b2-4a2d-928b-30e13686cea2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:28,278 [552] [DEBUG] [app] Starting request: urn:request:2f6134d3-f6b2-4a2d-928b-30e13686cea2 (/v1/_internal_ping) 2020-03-31 09:48:28,280 [552] [DEBUG] [app] Ending request: urn:request:2f6134d3-f6b2-4a2d-928b-30e13686cea2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:28,280 [552] [DEBUG] [app] Ending request: urn:request:2f6134d3-f6b2-4a2d-928b-30e13686cea2 (/v1/_internal_ping) 2020-03-31 09:48:28,280 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:48:28,280 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:28,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:48:28,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:48:28,283 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:48:28,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:48:28,283 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:28,285 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:28,285 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:28,286 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:28,286 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:28,299 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:28,316 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:28,316 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:28,320 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:28,322 [518] [DEBUG] [app] Starting request: urn:request:ce59e0e0-502d-42f0-831c-750c2e6ae2bf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:28,322 [518] [DEBUG] [app] Starting request: urn:request:ce59e0e0-502d-42f0-831c-750c2e6ae2bf (/_internal_ping) 2020-03-31 09:48:28,323 [518] [DEBUG] [app] Ending request: urn:request:ce59e0e0-502d-42f0-831c-750c2e6ae2bf (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:28,323 [518] [DEBUG] [app] Ending request: urn:request:ce59e0e0-502d-42f0-831c-750c2e6ae2bf (/_internal_ping) 2020-03-31 09:48:28,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:28,324 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:28,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:28,328 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:28,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:28,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:28,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:28,332 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:48:28,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:48:28,333 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:48:28,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:28,336 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:28,338 [543] [DEBUG] [app] Starting request: urn:request:7d5a712d-09bf-4b54-b1b0-b15c0cdf0b9a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:28,338 [543] [DEBUG] [app] Starting request: urn:request:7d5a712d-09bf-4b54-b1b0-b15c0cdf0b9a (/c1/_internal_ping) 2020-03-31 09:48:28,339 [543] [DEBUG] [app] Ending request: urn:request:7d5a712d-09bf-4b54-b1b0-b15c0cdf0b9a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:28,339 [543] [DEBUG] [app] Ending request: urn:request:7d5a712d-09bf-4b54-b1b0-b15c0cdf0b9a (/c1/_internal_ping) 2020-03-31 09:48:28,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:28,342 [518] [DEBUG] [app] Ending request: urn:request:b0d4422e-c93c-43a4-848c-fbb3f6ea90de (/health/instance) 2020-03-31 09:48:28,342 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:28,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 10.131.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:48:28,340 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:28,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:28,342 [518] [DEBUG] [app] Ending request: urn:request:b0d4422e-c93c-43a4-848c-fbb3f6ea90de (/health/instance) 2020-03-31 09:48:28,342 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:28,343 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 10.131.0.1 () - - [31/Mar/2020:09:48:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-verbs stdout | 2020-03-31 09:48:28,340 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:30,043 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c6e2313c-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:30 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:48:30,043 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c6e2313c-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:30 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:48:30,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c6e2313c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:c6ZHOm8RuVYj5Earw/C41EhqoRVCc26pSvmBt0SLjSw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:30 GMT'}. storagereplication stdout | 2020-03-31 09:48:30,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c6e2313c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:c6ZHOm8RuVYj5Earw/C41EhqoRVCc26pSvmBt0SLjSw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:30 GMT'}. 2020-03-31 09:48:30,125 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:48:30,125 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:48:30,126 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:30 GMT, Server-Request-ID=47780ac4-501e-0041-1041-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c6e2313c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:c6ZHOm8RuVYj5Earw/C41EhqoRVCc26pSvmBt0SLjSw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:30 GMT'}. 2020-03-31 09:48:30,126 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:30 GMT, Server-Request-ID=47780ac4-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:30,126 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:30 GMT, Server-Request-ID=47780ac4-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:48:30,126 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:30 GMT, Server-Request-ID=47780ac4-501e-0041-1041-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c6e2313c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:c6ZHOm8RuVYj5Earw/C41EhqoRVCc26pSvmBt0SLjSw=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:30 GMT'}. 2020-03-31 09:48:30,126 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:30 GMT, Server-Request-ID=47780ac4-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:30,126 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c6e2313c-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:30 GMT, Server-Request-ID=47780ac4-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:30,126 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:48:30,126 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:48:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:48:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:00.337699+00:00 (in 29.998954 seconds) 2020-03-31 09:48:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:00 UTC)" (scheduled at 2020-03-31 09:48:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:48:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:00.337699+00:00 (in 29.998954 seconds) 2020-03-31 09:48:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:00 UTC)" (scheduled at 2020-03-31 09:48:30.337699+00:00) 2020-03-31 09:48:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 48, 30, 339522), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:48:30,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 48, 30, 339522), False, 50, 1, 0]) 2020-03-31 09:48:30,355 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:48:30,355 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:48:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:48:30,355 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:30,355 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:00 UTC)" executed successfully 2020-03-31 09:48:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:48:31,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:41.440707+00:00 (in 9.996220 seconds) 2020-03-31 09:48:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:41 UTC)" (scheduled at 2020-03-31 09:48:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:48:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:41.440707+00:00 (in 9.996220 seconds) 2020-03-31 09:48:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:41 UTC)" (scheduled at 2020-03-31 09:48:31.440707+00:00) 2020-03-31 09:48:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:48:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:48:31,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:48:31,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:48:31,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:48:31,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:31,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:41 UTC)" executed successfully 2020-03-31 09:48:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:48:33,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:03 UTC)" (scheduled at 2020-03-31 09:48:33.330112+00:00) 2020-03-31 09:48:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:48:33,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:03 UTC)" (scheduled at 2020-03-31 09:48:33.330112+00:00) 2020-03-31 09:48:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:03.330112+00:00 (in 29.998370 seconds) 2020-03-31 09:48:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 33, 332415), True, datetime.datetime(2020, 3, 31, 9, 48, 33, 332415), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:48:33,332 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:03.330112+00:00 (in 29.998370 seconds) security_notification_worker stdout | 2020-03-31 09:48:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 33, 332415), True, datetime.datetime(2020, 3, 31, 9, 48, 33, 332415), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:48:33,348 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:48:33,348 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:33,348 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:48:33,348 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:33,348 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:03 UTC)" executed successfully 2020-03-31 09:48:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:48:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:43.539999+00:00 (in 10.172414 seconds) 2020-03-31 09:48:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:33 UTC)" (scheduled at 2020-03-31 09:48:33.366791+00:00) 2020-03-31 09:48:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:48:33,367 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:43.539999+00:00 (in 10.172414 seconds) 2020-03-31 09:48:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:33 UTC)" (scheduled at 2020-03-31 09:48:33.366791+00:00) 2020-03-31 09:48:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 33, 368222), True, datetime.datetime(2020, 3, 31, 9, 48, 33, 368222), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:48:33,369 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 33, 368222), True, datetime.datetime(2020, 3, 31, 9, 48, 33, 368222), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:48:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:48:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:48:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:33 UTC)" executed successfully 2020-03-31 09:48:34,130 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c951f060-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:34 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:48:34,130 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:c951f060-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:34 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:48:34,130 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c951f060-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Tu5gfpFSZVQnBitlNOTa4XWhFUvMjiQ9DicxY/ZjBx4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:34 GMT'}. storagereplication stdout | 2020-03-31 09:48:34,130 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c951f060-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Tu5gfpFSZVQnBitlNOTa4XWhFUvMjiQ9DicxY/ZjBx4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:34 GMT'}. 2020-03-31 09:48:34,210 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:48:34,210 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:48:34,211 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:34 GMT, Server-Request-ID=477818bd-501e-0041-3141-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c951f060-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Tu5gfpFSZVQnBitlNOTa4XWhFUvMjiQ9DicxY/ZjBx4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:34 GMT'}. 2020-03-31 09:48:34,211 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:34 GMT, Server-Request-ID=477818bd-501e-0041-3141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:34,211 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:34 GMT, Server-Request-ID=477818bd-501e-0041-3141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:34,211 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:48:34,211 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:34 GMT, Server-Request-ID=477818bd-501e-0041-3141-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'c951f060-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:Tu5gfpFSZVQnBitlNOTa4XWhFUvMjiQ9DicxY/ZjBx4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:34 GMT'}. 2020-03-31 09:48:34,211 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:34 GMT, Server-Request-ID=477818bd-501e-0041-3141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:34,211 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=c951f060-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:34 GMT, Server-Request-ID=477818bd-501e-0041-3141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:34,211 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:48:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:48:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:44.323352+00:00 (in 9.999087 seconds) 2020-03-31 09:48:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:44 UTC)" (scheduled at 2020-03-31 09:48:34.323352+00:00) 2020-03-31 09:48:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:48:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:44.323352+00:00 (in 9.999087 seconds) 2020-03-31 09:48:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:44 UTC)" (scheduled at 2020-03-31 09:48:34.323352+00:00) 2020-03-31 09:48:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 34, 325211), True, datetime.datetime(2020, 3, 31, 9, 48, 34, 325211), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:48:34,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 34, 325211), True, datetime.datetime(2020, 3, 31, 9, 48, 34, 325211), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:48:34,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:48:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:48:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:34,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:44 UTC)" executed successfully 2020-03-31 09:48:34,446 [518] [DEBUG] [app] Starting request: urn:request:3c325acc-5ac0-4d71-8517-179f6a429c0f (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:34,446 [518] [DEBUG] [app] Starting request: urn:request:3c325acc-5ac0-4d71-8517-179f6a429c0f (/health/instance) 2020-03-31 09:48:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:34,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:34,454 [538] [DEBUG] [app] Starting request: urn:request:5bf502cb-8f90-4a76-a601-f086ed2b5638 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:34,454 [538] [DEBUG] [app] Starting request: urn:request:5bf502cb-8f90-4a76-a601-f086ed2b5638 (/v1/_internal_ping) 2020-03-31 09:48:34,455 [538] [DEBUG] [app] Ending request: urn:request:5bf502cb-8f90-4a76-a601-f086ed2b5638 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:34,455 [538] [DEBUG] [app] Ending request: urn:request:5bf502cb-8f90-4a76-a601-f086ed2b5638 (/v1/_internal_ping) 2020-03-31 09:48:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:48:34,456 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:48:34,456 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:48:34,457 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:34,458 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:34,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:34,460 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:34,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:34,461 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:34,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:34,480 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:34,484 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:34,485 [518] [DEBUG] [app] Starting request: urn:request:872be6a8-2a09-40f2-9faf-ccb1f0cb0ee0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:34,485 [518] [DEBUG] [app] Starting request: urn:request:872be6a8-2a09-40f2-9faf-ccb1f0cb0ee0 (/_internal_ping) 2020-03-31 09:48:34,486 [518] [DEBUG] [app] Ending request: urn:request:872be6a8-2a09-40f2-9faf-ccb1f0cb0ee0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:34,486 [518] [DEBUG] [app] Ending request: urn:request:872be6a8-2a09-40f2-9faf-ccb1f0cb0ee0 (/_internal_ping) 2020-03-31 09:48:34,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:34,486 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:34,487 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:34,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:34,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:34,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:48:34,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:48:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:34,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:34,499 [543] [DEBUG] [app] Starting request: urn:request:3a18baba-cc5d-4bc4-99b4-37868666b304 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:34,499 [543] [DEBUG] [app] Starting request: urn:request:3a18baba-cc5d-4bc4-99b4-37868666b304 (/c1/_internal_ping) 2020-03-31 09:48:34,500 [543] [DEBUG] [app] Ending request: urn:request:3a18baba-cc5d-4bc4-99b4-37868666b304 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:34,500 [543] [DEBUG] [app] Ending request: urn:request:3a18baba-cc5d-4bc4-99b4-37868666b304 (/c1/_internal_ping) 2020-03-31 09:48:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:34,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:34,501 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:34,502 [518] [DEBUG] [app] Ending request: urn:request:3c325acc-5ac0-4d71-8517-179f6a429c0f (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:48:34,501 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:34,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:34,502 [518] [DEBUG] [app] Ending request: urn:request:3c325acc-5ac0-4d71-8517-179f6a429c0f (/health/instance) 2020-03-31 09:48:34,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:48:34,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:34,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:48:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:48:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:38.010934+00:00 (in 1.085645 seconds) storagereplication stdout | 2020-03-31 09:48:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:38.010934+00:00 (in 1.085645 seconds) 2020-03-31 09:48:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:48:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:46.924497+00:00 (in 8.912873 seconds) 2020-03-31 09:48:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:38 UTC)" (scheduled at 2020-03-31 09:48:38.010934+00:00) 2020-03-31 09:48:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:48:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:46.924497+00:00 (in 8.912873 seconds) 2020-03-31 09:48:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:38 UTC)" (scheduled at 2020-03-31 09:48:38.010934+00:00) 2020-03-31 09:48:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:38 UTC)" executed successfully 2020-03-31 09:48:38,272 [518] [DEBUG] [app] Starting request: urn:request:591f0d12-bcc5-489a-8848-d8476ea6b391 (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:38,272 [518] [DEBUG] [app] Starting request: urn:request:591f0d12-bcc5-489a-8848-d8476ea6b391 (/health/instance) 2020-03-31 09:48:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:38,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:38,277 [540] [DEBUG] [app] Starting request: urn:request:c8d3b3b7-baea-45d7-bc9b-f977b25b744e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:38,277 [540] [DEBUG] [app] Starting request: urn:request:c8d3b3b7-baea-45d7-bc9b-f977b25b744e (/v1/_internal_ping) 2020-03-31 09:48:38,278 [540] [DEBUG] [app] Ending request: urn:request:c8d3b3b7-baea-45d7-bc9b-f977b25b744e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:38,278 [540] [DEBUG] [app] Ending request: urn:request:c8d3b3b7-baea-45d7-bc9b-f977b25b744e (/v1/_internal_ping) 2020-03-31 09:48:38,279 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:38,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:48:38,279 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:38,279 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:38,281 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:38,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:48:38,281 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:48:38,280 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:38,283 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:38,283 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:38,283 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:38,283 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:38,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:38,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:38,302 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:38,302 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:38,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:38,306 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:38,308 [518] [DEBUG] [app] Starting request: urn:request:ff3ccc68-59d5-4b4d-8b37-90740ccd8fb5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:38,308 [518] [DEBUG] [app] Starting request: urn:request:ff3ccc68-59d5-4b4d-8b37-90740ccd8fb5 (/_internal_ping) 2020-03-31 09:48:38,309 [518] [DEBUG] [app] Ending request: urn:request:ff3ccc68-59d5-4b4d-8b37-90740ccd8fb5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:38,309 [518] [DEBUG] [app] Ending request: urn:request:ff3ccc68-59d5-4b4d-8b37-90740ccd8fb5 (/_internal_ping) 2020-03-31 09:48:38,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:38,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:38,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:38,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:38,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:38,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:38,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:38,318 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:48:38,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:48:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:38,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:38,325 [543] [DEBUG] [app] Starting request: urn:request:1375b077-6b39-43ed-beeb-5f7c08d84f44 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:38,325 [543] [DEBUG] [app] Starting request: urn:request:1375b077-6b39-43ed-beeb-5f7c08d84f44 (/c1/_internal_ping) 2020-03-31 09:48:38,326 [543] [DEBUG] [app] Ending request: urn:request:1375b077-6b39-43ed-beeb-5f7c08d84f44 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:38,326 [543] [DEBUG] [app] Ending request: urn:request:1375b077-6b39-43ed-beeb-5f7c08d84f44 (/c1/_internal_ping) 2020-03-31 09:48:38,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:48:38,327 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:48:38,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:38,330 [518] [DEBUG] [app] Ending request: urn:request:591f0d12-bcc5-489a-8848-d8476ea6b391 (/health/instance) 2020-03-31 09:48:38,330 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:38,330 [518] [DEBUG] [app] Ending request: urn:request:591f0d12-bcc5-489a-8848-d8476ea6b391 (/health/instance) 2020-03-31 09:48:38,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:38,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:48:38,330 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:48:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:48:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:51 UTC)" (scheduled at 2020-03-31 09:48:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:48:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:51 UTC)" (scheduled at 2020-03-31 09:48:41.440707+00:00) 2020-03-31 09:48:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:51.440707+00:00 (in 9.998774 seconds) repositoryactioncounter stdout | 2020-03-31 09:48:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:51.440707+00:00 (in 9.998774 seconds) 2020-03-31 09:48:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:48:41,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:48:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:48:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:48:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:48:41,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:41,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:51 UTC)" executed successfully 2020-03-31 09:48:42,220 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ce244692-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:42 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 storagereplication stdout | 2020-03-31 09:48:42,220 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:ce244692-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:42 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438 timeout:20 2020-03-31 09:48:42,220 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ce244692-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xNBb7kLCdTvwZiHHeq4ThbgGxt8AgOYeRbm6P0dbsfo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:42 GMT'}. storagereplication stdout | 2020-03-31 09:48:42,220 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ce244692-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xNBb7kLCdTvwZiHHeq4ThbgGxt8AgOYeRbm6P0dbsfo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:42 GMT'}. 2020-03-31 09:48:42,300 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:48:42,300 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:48:42,301 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:42 GMT, Server-Request-ID=477834f8-501e-0041-7341-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ce244692-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xNBb7kLCdTvwZiHHeq4ThbgGxt8AgOYeRbm6P0dbsfo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:42 GMT'}. 2020-03-31 09:48:42,301 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:42 GMT, Server-Request-ID=477834f8-501e-0041-7341-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:48:42,301 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:42 GMT, Server-Request-ID=477834f8-501e-0041-7341-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'ce244692-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:xNBb7kLCdTvwZiHHeq4ThbgGxt8AgOYeRbm6P0dbsfo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:42 GMT'}. 2020-03-31 09:48:42,301 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:42 GMT, Server-Request-ID=477834f8-501e-0041-7341-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:42,301 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:42 GMT, Server-Request-ID=477834f8-501e-0041-7341-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:42,302 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:48:42,301 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=ce244692-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:42 GMT, Server-Request-ID=477834f8-501e-0041-7341-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:42,302 [191] [DEBUG] [__main__] Cannot find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:48:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:48:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:33.366791+00:00 (in 49.826044 seconds) 2020-03-31 09:48:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:43 UTC)" (scheduled at 2020-03-31 09:48:43.539999+00:00) 2020-03-31 09:48:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:48:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:33.366791+00:00 (in 49.826044 seconds) 2020-03-31 09:48:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:43 UTC)" (scheduled at 2020-03-31 09:48:43.539999+00:00) 2020-03-31 09:48:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:43 UTC)" executed successfully 2020-03-31 09:48:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:48:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:49.498966+00:00 (in 5.174337 seconds) 2020-03-31 09:48:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:54 UTC)" (scheduled at 2020-03-31 09:48:44.323352+00:00) 2020-03-31 09:48:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:48:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:49.498966+00:00 (in 5.174337 seconds) 2020-03-31 09:48:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:54 UTC)" (scheduled at 2020-03-31 09:48:44.323352+00:00) 2020-03-31 09:48:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 44, 325675), True, datetime.datetime(2020, 3, 31, 9, 48, 44, 325675), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:48:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 44, 325675), True, datetime.datetime(2020, 3, 31, 9, 48, 44, 325675), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:48:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:44,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:48:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:48:44,343 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:54 UTC)" executed successfully 2020-03-31 09:48:44,446 [518] [DEBUG] [app] Starting request: urn:request:494e0102-7f8f-42d4-9bd6-fe78584b4276 (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:44,446 [518] [DEBUG] [app] Starting request: urn:request:494e0102-7f8f-42d4-9bd6-fe78584b4276 (/health/instance) 2020-03-31 09:48:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:44,453 [551] [DEBUG] [app] Starting request: urn:request:660bb42c-270a-41b0-8213-324ae910633f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:44,453 [551] [DEBUG] [app] Starting request: urn:request:660bb42c-270a-41b0-8213-324ae910633f (/v1/_internal_ping) 2020-03-31 09:48:44,454 [551] [DEBUG] [app] Ending request: urn:request:660bb42c-270a-41b0-8213-324ae910633f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:44,454 [551] [DEBUG] [app] Ending request: urn:request:660bb42c-270a-41b0-8213-324ae910633f (/v1/_internal_ping) 2020-03-31 09:48:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:48:44,455 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:48:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:44,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:48:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:48:44,457 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:44,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:44,458 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:44,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:44,459 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:44,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:44,486 [518] [DEBUG] [app] Starting request: urn:request:42528556-5f58-4a20-b832-2446004f8a31 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:44,486 [518] [DEBUG] [app] Starting request: urn:request:42528556-5f58-4a20-b832-2446004f8a31 (/_internal_ping) 2020-03-31 09:48:44,487 [518] [DEBUG] [app] Ending request: urn:request:42528556-5f58-4a20-b832-2446004f8a31 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:44,487 [518] [DEBUG] [app] Ending request: urn:request:42528556-5f58-4a20-b832-2446004f8a31 (/_internal_ping) 2020-03-31 09:48:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:44,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:44,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:44,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:44,496 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:48:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:48:44,497 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:48:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:44,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:44,502 [543] [DEBUG] [app] Starting request: urn:request:cfb4d3c0-bf55-4c62-b43c-fb7348124820 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:44,502 [543] [DEBUG] [app] Starting request: urn:request:cfb4d3c0-bf55-4c62-b43c-fb7348124820 (/c1/_internal_ping) 2020-03-31 09:48:44,503 [543] [DEBUG] [app] Ending request: urn:request:cfb4d3c0-bf55-4c62-b43c-fb7348124820 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:44,503 [543] [DEBUG] [app] Ending request: urn:request:cfb4d3c0-bf55-4c62-b43c-fb7348124820 (/c1/_internal_ping) 2020-03-31 09:48:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:44,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:44,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:44,505 [518] [DEBUG] [app] Ending request: urn:request:494e0102-7f8f-42d4-9bd6-fe78584b4276 (/health/instance) 2020-03-31 09:48:44,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:48:44,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:44,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:48:44,505 [518] [DEBUG] [app] Ending request: urn:request:494e0102-7f8f-42d4-9bd6-fe78584b4276 (/health/instance) 2020-03-31 09:48:44,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:44,506 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:48:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:48:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:15.064514+00:00 (in 29.999113 seconds) 2020-03-31 09:48:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:15 UTC)" (scheduled at 2020-03-31 09:48:45.064514+00:00) gcworker stdout | 2020-03-31 09:48:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:15.064514+00:00 (in 29.999113 seconds) 2020-03-31 09:48:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:15 UTC)" (scheduled at 2020-03-31 09:48:45.064514+00:00) 2020-03-31 09:48:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438525, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:48:45,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438525, 1209600, 500, 1, 0]) 2020-03-31 09:48:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:48:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:48:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:48:45,082 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:15 UTC)" executed successfully 2020-03-31 09:48:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:48:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:56.924497+00:00 (in 9.999189 seconds) storagereplication stdout | 2020-03-31 09:48:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:56.924497+00:00 (in 9.999189 seconds) 2020-03-31 09:48:48,272 [527] [DEBUG] [app] Starting request: urn:request:0924d040-b8ac-409f-ba60-ff89b3e5a576 (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:48,272 [527] [DEBUG] [app] Starting request: urn:request:0924d040-b8ac-409f-ba60-ff89b3e5a576 (/health/instance) 2020-03-31 09:48:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:48,274 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:48,278 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:48,280 [552] [DEBUG] [app] Starting request: urn:request:a907bb67-e939-4c9d-9ec7-52dfabe33222 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:48,280 [552] [DEBUG] [app] Starting request: urn:request:a907bb67-e939-4c9d-9ec7-52dfabe33222 (/v1/_internal_ping) 2020-03-31 09:48:48,280 [552] [DEBUG] [app] Ending request: urn:request:a907bb67-e939-4c9d-9ec7-52dfabe33222 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:48,280 [552] [DEBUG] [app] Ending request: urn:request:a907bb67-e939-4c9d-9ec7-52dfabe33222 (/v1/_internal_ping) 2020-03-31 09:48:48,281 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:48:48,281 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:48:48,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:48:48,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:48,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:48,285 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:48,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:48,286 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:48,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:48,287 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:48,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:48,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:48,309 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:48,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:48,312 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:48,314 [527] [DEBUG] [app] Starting request: urn:request:3af698e1-eebd-4d87-b78b-0c5c3e59f8ef (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:48,314 [527] [DEBUG] [app] Starting request: urn:request:3af698e1-eebd-4d87-b78b-0c5c3e59f8ef (/_internal_ping) 2020-03-31 09:48:48,314 [527] [DEBUG] [app] Ending request: urn:request:3af698e1-eebd-4d87-b78b-0c5c3e59f8ef (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:48,314 [527] [DEBUG] [app] Ending request: urn:request:3af698e1-eebd-4d87-b78b-0c5c3e59f8ef (/_internal_ping) 2020-03-31 09:48:48,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:48,315 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:48:48,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:48,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:48,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:48,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:48:48,323 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:48:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:48,327 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:48,328 [532] [DEBUG] [app] Starting request: urn:request:a1d6459f-5c3c-425a-90ce-d39e87421dff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:48,328 [532] [DEBUG] [app] Starting request: urn:request:a1d6459f-5c3c-425a-90ce-d39e87421dff (/c1/_internal_ping) 2020-03-31 09:48:48,329 [532] [DEBUG] [app] Ending request: urn:request:a1d6459f-5c3c-425a-90ce-d39e87421dff (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:48,329 [532] [DEBUG] [app] Ending request: urn:request:a1d6459f-5c3c-425a-90ce-d39e87421dff (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:48,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:48,330 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:48:48,330 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:48,331 [527] [DEBUG] [app] Ending request: urn:request:0924d040-b8ac-409f-ba60-ff89b3e5a576 (/health/instance) 2020-03-31 09:48:48,332 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:48,331 [527] [DEBUG] [app] Ending request: urn:request:0924d040-b8ac-409f-ba60-ff89b3e5a576 (/health/instance) 2020-03-31 09:48:48,332 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:48:48,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:48:48,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:48:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:48:49,499 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:54.323352+00:00 (in 4.823335 seconds) 2020-03-31 09:48:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:49 UTC)" (scheduled at 2020-03-31 09:48:49.498966+00:00) 2020-03-31 09:48:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2020-03-31 09:48:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:54.323352+00:00 (in 4.823335 seconds) 2020-03-31 09:48:49,500 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:49 UTC)" (scheduled at 2020-03-31 09:48:49.498966+00:00) 2020-03-31 09:48:49,500 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:48:49,500 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:49 UTC)" executed successfully 2020-03-31 09:48:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:48:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:59.560326+00:00 (in 8.448625 seconds) 2020-03-31 09:48:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:51 UTC)" (scheduled at 2020-03-31 09:48:51.110610+00:00) 2020-03-31 09:48:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:48:51,112 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:48:59.560326+00:00 (in 8.448625 seconds) 2020-03-31 09:48:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:51 UTC)" (scheduled at 2020-03-31 09:48:51.110610+00:00) 2020-03-31 09:48:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 51, 112809), True, datetime.datetime(2020, 3, 31, 9, 48, 51, 112809), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:48:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 51, 112809), True, datetime.datetime(2020, 3, 31, 9, 48, 51, 112809), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:48:51,131 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:48:51,131 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:48:51,131 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:51,131 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:51 UTC)" executed successfully 2020-03-31 09:48:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:48:51,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:01.440707+00:00 (in 9.995728 seconds) 2020-03-31 09:48:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:01 UTC)" (scheduled at 2020-03-31 09:48:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:48:51,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:01.440707+00:00 (in 9.995728 seconds) 2020-03-31 09:48:51,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:01 UTC)" (scheduled at 2020-03-31 09:48:51.440707+00:00) 2020-03-31 09:48:51,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:48:51,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:48:51,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:48:51,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:48:51,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:48:51,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:51,464 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:01 UTC)" executed successfully 2020-03-31 09:48:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:48:52,113 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:22.113221+00:00 (in 29.999132 seconds) 2020-03-31 09:48:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:22 UTC)" (scheduled at 2020-03-31 09:48:52.113221+00:00) securityworker stdout | 2020-03-31 09:48:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:22.113221+00:00 (in 29.999132 seconds) 2020-03-31 09:48:52,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:22 UTC)" (scheduled at 2020-03-31 09:48:52.113221+00:00) 2020-03-31 09:48:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:48:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:48:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:48:52,129 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:48:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:48:52,132 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:48:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:48:52,132 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:52,132 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:22 UTC)" executed successfully 2020-03-31 09:48:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:48:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:04.323352+00:00 (in 9.998749 seconds) 2020-03-31 09:48:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:04 UTC)" (scheduled at 2020-03-31 09:48:54.323352+00:00) 2020-03-31 09:48:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:48:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:04.323352+00:00 (in 9.998749 seconds) 2020-03-31 09:48:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:04 UTC)" (scheduled at 2020-03-31 09:48:54.323352+00:00) 2020-03-31 09:48:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:48:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 54, 325524), True, datetime.datetime(2020, 3, 31, 9, 48, 54, 325524), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:48:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 54, 325524), True, datetime.datetime(2020, 3, 31, 9, 48, 54, 325524), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:48:54,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:48:54,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:48:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:48:54,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:54,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:04 UTC)" executed successfully 2020-03-31 09:48:54,446 [518] [DEBUG] [app] Starting request: urn:request:c83b217c-dad3-41ad-8010-12b4cfa5cb2d (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:54,446 [518] [DEBUG] [app] Starting request: urn:request:c83b217c-dad3-41ad-8010-12b4cfa5cb2d (/health/instance) 2020-03-31 09:48:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:54,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:54,454 [550] [DEBUG] [app] Starting request: urn:request:ff274955-8acd-4146-a817-94766474f1d1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:54,454 [550] [DEBUG] [app] Starting request: urn:request:ff274955-8acd-4146-a817-94766474f1d1 (/v1/_internal_ping) 2020-03-31 09:48:54,455 [550] [DEBUG] [app] Ending request: urn:request:ff274955-8acd-4146-a817-94766474f1d1 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:54,455 [550] [DEBUG] [app] Ending request: urn:request:ff274955-8acd-4146-a817-94766474f1d1 (/v1/_internal_ping) 2020-03-31 09:48:54,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:48:54,456 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:48:54,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:48:54,458 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:54,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:48:54,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:54,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:54,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:54,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:54,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:54,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:54,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:54,486 [518] [DEBUG] [app] Starting request: urn:request:caf80436-1dca-4fcf-930e-fe1c6b10651e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:54,486 [518] [DEBUG] [app] Starting request: urn:request:caf80436-1dca-4fcf-930e-fe1c6b10651e (/_internal_ping) 2020-03-31 09:48:54,487 [518] [DEBUG] [app] Ending request: urn:request:caf80436-1dca-4fcf-930e-fe1c6b10651e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:54,487 [518] [DEBUG] [app] Ending request: urn:request:caf80436-1dca-4fcf-930e-fe1c6b10651e (/_internal_ping) 2020-03-31 09:48:54,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:54,487 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:48:54,488 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:54,491 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:48:54,491 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:54,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:48:54,495 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:48:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:54,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:54,500 [541] [DEBUG] [app] Starting request: urn:request:0809d922-ba4e-4cdb-bcca-bdbd2502a71b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:54,500 [541] [DEBUG] [app] Starting request: urn:request:0809d922-ba4e-4cdb-bcca-bdbd2502a71b (/c1/_internal_ping) 2020-03-31 09:48:54,501 [541] [DEBUG] [app] Ending request: urn:request:0809d922-ba4e-4cdb-bcca-bdbd2502a71b (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:54,501 [541] [DEBUG] [app] Ending request: urn:request:0809d922-ba4e-4cdb-bcca-bdbd2502a71b (/c1/_internal_ping) 2020-03-31 09:48:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:48:54,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:54,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:48:54,501 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:54,503 [518] [DEBUG] [app] Ending request: urn:request:c83b217c-dad3-41ad-8010-12b4cfa5cb2d (/health/instance) 2020-03-31 09:48:54,503 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:54,503 [518] [DEBUG] [app] Ending request: urn:request:c83b217c-dad3-41ad-8010-12b4cfa5cb2d (/health/instance) 2020-03-31 09:48:54,503 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:48:54,504 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:48:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:48:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:06.924497+00:00 (in 9.999198 seconds) storagereplication stdout | 2020-03-31 09:48:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:48:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:48:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:06.924497+00:00 (in 9.999198 seconds) 2020-03-31 09:48:57,639 [188] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2020-03-31 09:48:57,639 [188] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:57,640 [188] [INFO] [apscheduler.executors.default] Running job "_try_report_stats (trigger: interval[1:00:00], next run at: 2020-03-31 09:48:57 UTC)" (scheduled at 2020-03-31 09:48:57.639223+00:00) globalpromstats stdout | 2020-03-31 09:48:57,640 [188] [INFO] [apscheduler.executors.default] Running job "_try_report_stats (trigger: interval[1:00:00], next run at: 2020-03-31 09:48:57 UTC)" (scheduled at 2020-03-31 09:48:57.639223+00:00) 2020-03-31 09:48:57,640 [188] [DEBUG] [__main__] Attempting to report stats globalpromstats stdout | 2020-03-31 09:48:57,640 [188] [DEBUG] [__main__] Attempting to report stats 2020-03-31 09:48:57,640 [188] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:48:57.639223+00:00 (in 3599.999507 seconds) 2020-03-31 09:48:57,640 [188] [DEBUG] [util.locking] Acquiring global lock GLOBAL_PROM_STATS globalpromstats stdout | 2020-03-31 09:48:57,640 [188] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:48:57.639223+00:00 (in 3599.999507 seconds) 2020-03-31 09:48:57,640 [188] [DEBUG] [util.locking] Acquiring global lock GLOBAL_PROM_STATS 2020-03-31 09:48:57,649 [188] [DEBUG] [util.locking] Acquired lock GLOBAL_PROM_STATS globalpromstats stdout | 2020-03-31 09:48:57,649 [188] [DEBUG] [util.locking] Acquired lock GLOBAL_PROM_STATS 2020-03-31 09:48:57,650 [188] [DEBUG] [__main__] Reporting global stats globalpromstats stdout | 2020-03-31 09:48:57,650 [188] [DEBUG] [__main__] Reporting global stats 2020-03-31 09:48:57,650 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "repository" AS "t1") AS "_wrapped"', []) globalpromstats stdout | 2020-03-31 09:48:57,650 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "repository" AS "t1") AS "_wrapped"', []) 2020-03-31 09:48:57,665 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."id" NOT IN (SELECT "t2"."namespace_id" FROM "deletednamespace" AS "t2")))) AS "_wrapped"', [False, False]) globalpromstats stdout | 2020-03-31 09:48:57,665 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."id" NOT IN (SELECT "t2"."namespace_id" FROM "deletednamespace" AS "t2")))) AS "_wrapped"', [False, False]) 2020-03-31 09:48:57,668 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 globalpromstats stdout | 2020-03-31 09:48:57,668 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:57,669 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 globalpromstats stdout | 2020-03-31 09:48:57,669 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:57,670 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics globalpromstats stdout | 2020-03-31 09:48:57,670 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:48:57,672 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."id" NOT IN (SELECT "t2"."namespace_id" FROM "deletednamespace" AS "t2")))) AS "_wrapped"', [True, False]) globalpromstats stdout | 2020-03-31 09:48:57,672 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."id" NOT IN (SELECT "t2"."namespace_id" FROM "deletednamespace" AS "t2")))) AS "_wrapped"', [True, False]) 2020-03-31 09:48:57,674 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 globalpromstats stdout | 2020-03-31 09:48:57,674 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:57,675 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 globalpromstats stdout | 2020-03-31 09:48:57,675 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:57,675 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics globalpromstats stdout | 2020-03-31 09:48:57,675 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:48:57,678 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ("t1"."robot" = %s)) AS "_wrapped"', [True]) globalpromstats stdout | 2020-03-31 09:48:57,678 [188] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ("t1"."robot" = %s)) AS "_wrapped"', [True]) 2020-03-31 09:48:57,679 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 globalpromstats stdout | 2020-03-31 09:48:57,679 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:57,680 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 globalpromstats stdout | 2020-03-31 09:48:57,680 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:57,681 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics globalpromstats stdout | 2020-03-31 09:48:57,681 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:48:57,682 [188] [DEBUG] [data.database] Disconnecting from database. globalpromstats stdout | 2020-03-31 09:48:57,682 [188] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:57,682 [188] [DEBUG] [util.locking] Releasing lock GLOBAL_PROM_STATS globalpromstats stdout | 2020-03-31 09:48:57,682 [188] [DEBUG] [util.locking] Releasing lock GLOBAL_PROM_STATS 2020-03-31 09:48:57,685 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 globalpromstats stdout | 2020-03-31 09:48:57,685 [188] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:57,686 [188] [DEBUG] [util.locking] Released lock GLOBAL_PROM_STATS globalpromstats stdout | 2020-03-31 09:48:57,686 [188] [DEBUG] [util.locking] Released lock GLOBAL_PROM_STATS 2020-03-31 09:48:57,686 [188] [INFO] [apscheduler.executors.default] Job "_try_report_stats (trigger: interval[1:00:00], next run at: 2020-03-31 10:48:57 UTC)" executed successfully globalpromstats stdout | 2020-03-31 09:48:57,686 [188] [INFO] [apscheduler.executors.default] Job "_try_report_stats (trigger: interval[1:00:00], next run at: 2020-03-31 10:48:57 UTC)" executed successfully 2020-03-31 09:48:57,687 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 globalpromstats stdout | 2020-03-31 09:48:57,687 [188] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:57,687 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics globalpromstats stdout | 2020-03-31 09:48:57,687 [188] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:48:58,272 [518] [DEBUG] [app] Starting request: urn:request:4d902c14-b37a-49f2-82a6-cbadd268ca59 (/health/instance) gunicorn-web stdout | 2020-03-31 09:48:58,272 [518] [DEBUG] [app] Starting request: urn:request:4d902c14-b37a-49f2-82a6-cbadd268ca59 (/health/instance) 2020-03-31 09:48:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:58,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:58,278 [538] [DEBUG] [app] Starting request: urn:request:f4ca94a5-8c48-4333-ace6-4a742ceabb64 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:58,278 [538] [DEBUG] [app] Starting request: urn:request:f4ca94a5-8c48-4333-ace6-4a742ceabb64 (/v1/_internal_ping) 2020-03-31 09:48:58,279 [538] [DEBUG] [app] Ending request: urn:request:f4ca94a5-8c48-4333-ace6-4a742ceabb64 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:48:58,279 [538] [DEBUG] [app] Ending request: urn:request:f4ca94a5-8c48-4333-ace6-4a742ceabb64 (/v1/_internal_ping) 2020-03-31 09:48:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:48:58,280 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:48:58,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:48:58,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:48:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:48:58,283 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:48:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:48:58,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:48:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:48:58,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:48:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:48:58,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:48:58,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:48:58,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:48:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:58,315 [518] [DEBUG] [app] Starting request: urn:request:ffafa10e-9ca8-4313-90e5-6e944663d48f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:48:58,315 [518] [DEBUG] [app] Starting request: urn:request:ffafa10e-9ca8-4313-90e5-6e944663d48f (/_internal_ping) 2020-03-31 09:48:58,317 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy 2020-03-31 09:48:58,316 [518] [DEBUG] [app] Ending request: urn:request:ffafa10e-9ca8-4313-90e5-6e944663d48f (/_internal_ping) storagereplication stdout | 2020-03-31 09:48:58,317 [191] [WARNING] [__main__] Failed to find path `sha256/07/070a739142cf01a7c485b65233471f7d89c8dc6fb7276771601a31673714d438` in location `azure-uk` after copy 2020-03-31 09:48:58,317 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:48:58,317 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:48:58,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:48:58,316 [518] [DEBUG] [app] Ending request: urn:request:ffafa10e-9ca8-4313-90e5-6e944663d48f (/_internal_ping) storagereplication stdout | 2020-03-31 09:48:58,317 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "229ce905-1640-4594-aae8-fb66fcc5e51f"} 2020-03-31 09:48:58,317 [191] [WARNING] [workers.queueworker] Job exception: gunicorn-web stdout | 2020-03-31 09:48:58,317 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) 2020-03-31 09:48:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:58,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:48:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:48:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:48:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:58,327 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:58,328 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) storagereplication stdout | 2020-03-31 09:48:58,328 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [4, 1, 0]) 2020-03-31 09:48:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:48:58,328 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:48:58,332 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 53, 58, 327464), True, u'5b62781d-3e64-4a0b-929f-e1c9e6f602b4', 4]) storagereplication stdout | 2020-03-31 09:48:58,332 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 53, 58, 327464), True, u'5b62781d-3e64-4a0b-929f-e1c9e6f602b4', 4]) 2020-03-31 09:48:58,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:48:58,333 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:48:58,334 [537] [DEBUG] [app] Starting request: urn:request:d6e1edaa-d6d2-458c-8ccf-aa823a783087 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:58,334 [537] [DEBUG] [app] Starting request: urn:request:d6e1edaa-d6d2-458c-8ccf-aa823a783087 (/c1/_internal_ping) 2020-03-31 09:48:58,335 [537] [DEBUG] [app] Ending request: urn:request:d6e1edaa-d6d2-458c-8ccf-aa823a783087 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:48:58,335 [537] [DEBUG] [app] Ending request: urn:request:d6e1edaa-d6d2-458c-8ccf-aa823a783087 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:48:58,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:48:58,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:48:58,336 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:48:58,335 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:48:58,337 [518] [DEBUG] [app] Ending request: urn:request:4d902c14-b37a-49f2-82a6-cbadd268ca59 (/health/instance) 2020-03-31 09:48:58,337 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:48:58,337 [518] [DEBUG] [app] Ending request: urn:request:4d902c14-b37a-49f2-82a6-cbadd268ca59 (/health/instance) 2020-03-31 09:48:58,337 [518] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:48:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.067 49 0.067) 2020-03-31 09:48:58,338 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:48:58,338 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:48:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:48:58,339 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 58, 337502), True, datetime.datetime(2020, 3, 31, 9, 48, 58, 337502), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:48:58,339 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 48, 58, 337502), True, datetime.datetime(2020, 3, 31, 9, 48, 58, 337502), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:48:58,343 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 8, 58, 337502), 1, u'1503c2c0-36ac-47dd-a69f-0a4ad7459fb7', 2, u'93e98dd4-693f-4354-83aa-01ff072767fa']) storagereplication stdout | 2020-03-31 09:48:58,343 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 8, 58, 337502), 1, u'1503c2c0-36ac-47dd-a69f-0a4ad7459fb7', 2, u'93e98dd4-693f-4354-83aa-01ff072767fa']) 2020-03-31 09:48:58,351 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} storagereplication stdout | 2020-03-31 09:48:58,351 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:48:58,352 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:58,352 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 storagereplication stdout | 2020-03-31 09:48:58,352 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:58,352 [191] [DEBUG] [__main__] Starting replication of image storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace 1 2020-03-31 09:48:58,353 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:48:58,353 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:48:58,367 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:48:58,367 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:48:58,370 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) storagereplication stdout | 2020-03-31 09:48:58,370 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'd910da7a-e575-40de-8965-a8fbd51c3b85']) 2020-03-31 09:48:58,374 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:48:58,374 [191] [DEBUG] [__main__] For replication of storage d910da7a-e575-40de-8965-a8fbd51c3b85 under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:48:58,374 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:d7c5534e-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:48:58,375 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c5534e-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7c5534e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:O+utu3EBawICj0ilfN0tZ7dMhiN3mZFO+ZIo0Vs3hmE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. storagereplication stdout | 2020-03-31 09:48:58,374 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:d7c5534e-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:48:58,375 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c5534e-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7c5534e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:O+utu3EBawICj0ilfN0tZ7dMhiN3mZFO+ZIo0Vs3hmE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. 2020-03-31 09:48:58,383 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:48:58,383 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:48:58,383 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c5534e-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=dd7388d5-b01e-000c-4641-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7c5534e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:O+utu3EBawICj0ilfN0tZ7dMhiN3mZFO+ZIo0Vs3hmE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. storagereplication stdout | 2020-03-31 09:48:58,383 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c5534e-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=dd7388d5-b01e-000c-4641-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7c5534e-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:O+utu3EBawICj0ilfN0tZ7dMhiN3mZFO+ZIo0Vs3hmE=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. 2020-03-31 09:48:58,384 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us 2020-03-31 09:48:58,384 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:48:58,384 [191] [DEBUG] [__main__] Starting copy of storage d910da7a-e575-40de-8965-a8fbd51c3b85 to location azure-uk from azure-us 2020-03-31 09:48:58,384 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:48:58,385 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:48:58,385 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:d7c6ef06-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=b/Bn4lgiTqc7byFJTQBJeXNwSkbcFdYDxrXInVtXWec%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A58Z x-ms-date:Tue, 31 Mar 2020 09:48:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 storagereplication stdout | 2020-03-31 09:48:58,385 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:48:58,385 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:d7c6ef06-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=b/Bn4lgiTqc7byFJTQBJeXNwSkbcFdYDxrXInVtXWec%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A58Z x-ms-date:Tue, 31 Mar 2020 09:48:58 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 2020-03-31 09:48:58,385 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c6ef06-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=b/Bn4lgiTqc7byFJTQBJeXNwSkbcFdYDxrXInVtXWec%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:as7S/PWuEieFu8rAJ43AtFN0TMkl3ERvTr8sYZZ+OPQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'd7c6ef06-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. storagereplication stdout | 2020-03-31 09:48:58,385 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c6ef06-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=b/Bn4lgiTqc7byFJTQBJeXNwSkbcFdYDxrXInVtXWec%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:as7S/PWuEieFu8rAJ43AtFN0TMkl3ERvTr8sYZZ+OPQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'd7c6ef06-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. 2020-03-31 09:48:58,412 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:48:58,412 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 HTTP/1.1" 202 0 2020-03-31 09:48:58,413 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c6ef06-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=dd7388da-b01e-000c-4841-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=b/Bn4lgiTqc7byFJTQBJeXNwSkbcFdYDxrXInVtXWec%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:as7S/PWuEieFu8rAJ43AtFN0TMkl3ERvTr8sYZZ+OPQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'd7c6ef06-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. storagereplication stdout | 2020-03-31 09:48:58,413 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7c6ef06-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=dd7388da-b01e-000c-4841-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?sr=b&sp=r&sig=b/Bn4lgiTqc7byFJTQBJeXNwSkbcFdYDxrXInVtXWec%3D&sv=2017-07-29&se=2020-03-31T09%3A49%3A58Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:as7S/PWuEieFu8rAJ43AtFN0TMkl3ERvTr8sYZZ+OPQ=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'd7c6ef06-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. 2020-03-31 09:48:58,414 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:48:58,414 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:48:58,414 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:d7cb6086-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:58 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:48:58,414 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7cb6086-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:QvSNNH71k0iIoMWJ+fgHqjLW7EXTDuMOCKIBLWIvtQ4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. storagereplication stdout | 2020-03-31 09:48:58,414 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:d7cb6086-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:48:58 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:48:58,414 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7cb6086-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:QvSNNH71k0iIoMWJ+fgHqjLW7EXTDuMOCKIBLWIvtQ4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. 2020-03-31 09:48:58,496 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:48:58,496 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:48:58,497 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=47787011-501e-0041-5141-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7cb6086-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:QvSNNH71k0iIoMWJ+fgHqjLW7EXTDuMOCKIBLWIvtQ4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. 2020-03-31 09:48:58,497 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=47787011-501e-0041-5141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:58,497 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=47787011-501e-0041-5141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:58,497 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:48:58,497 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=47787011-501e-0041-5141-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd7cb6086-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:QvSNNH71k0iIoMWJ+fgHqjLW7EXTDuMOCKIBLWIvtQ4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:48:58 GMT'}. 2020-03-31 09:48:58,497 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=47787011-501e-0041-5141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:58,497 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=d7cb6086-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:48:58 GMT, Server-Request-ID=47787011-501e-0041-5141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:48:58,497 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:48:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:48:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:48:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:51.110610+00:00 (in 51.549090 seconds) 2020-03-31 09:48:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:59 UTC)" (scheduled at 2020-03-31 09:48:59.560326+00:00) 2020-03-31 09:48:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:48:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:51.110610+00:00 (in 51.549090 seconds) 2020-03-31 09:48:59,562 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:59 UTC)" (scheduled at 2020-03-31 09:48:59.560326+00:00) 2020-03-31 09:48:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:48:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:49:59 UTC)" executed successfully 2020-03-31 09:49:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:49:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:30.337699+00:00 (in 29.998934 seconds) 2020-03-31 09:49:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:30 UTC)" (scheduled at 2020-03-31 09:49:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:49:00,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:30.337699+00:00 (in 29.998934 seconds) 2020-03-31 09:49:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:30 UTC)" (scheduled at 2020-03-31 09:49:00.337699+00:00) 2020-03-31 09:49:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 49, 0, 339489), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:49:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 49, 0, 339489), False, 50, 1, 0]) 2020-03-31 09:49:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:49:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:49:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:49:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:30 UTC)" executed successfully 2020-03-31 09:49:00,500 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:d9099670-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:00 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:49:00,500 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:d9099670-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:00 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:49:00,500 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd9099670-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AMAtEiMJz3/IDRd8vuyyMUj3x+uZhL2W3Lfh/YZn4Sc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:00 GMT'}. storagereplication stdout | 2020-03-31 09:49:00,500 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd9099670-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AMAtEiMJz3/IDRd8vuyyMUj3x+uZhL2W3Lfh/YZn4Sc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:00 GMT'}. 2020-03-31 09:49:00,580 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:49:00,580 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:49:00,581 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:00 GMT, Server-Request-ID=47787818-501e-0041-6241-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd9099670-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AMAtEiMJz3/IDRd8vuyyMUj3x+uZhL2W3Lfh/YZn4Sc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:00 GMT'}. 2020-03-31 09:49:00,581 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:00 GMT, Server-Request-ID=47787818-501e-0041-6241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:00,581 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:00 GMT, Server-Request-ID=47787818-501e-0041-6241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:00,581 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:49:00,581 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:00 GMT, Server-Request-ID=47787818-501e-0041-6241-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'd9099670-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:AMAtEiMJz3/IDRd8vuyyMUj3x+uZhL2W3Lfh/YZn4Sc=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:00 GMT'}. 2020-03-31 09:49:00,581 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:00 GMT, Server-Request-ID=47787818-501e-0041-6241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:00,581 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=d9099670-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:00 GMT, Server-Request-ID=47787818-501e-0041-6241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:00,581 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:49:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:49:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:11 UTC)" (scheduled at 2020-03-31 09:49:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:49:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:11 UTC)" (scheduled at 2020-03-31 09:49:01.440707+00:00) 2020-03-31 09:49:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:11.440707+00:00 (in 9.999056 seconds) repositoryactioncounter stdout | 2020-03-31 09:49:01,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:11.440707+00:00 (in 9.999056 seconds) 2020-03-31 09:49:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:49:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:49:01,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:49:01,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:49:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:49:01,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:01,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:11 UTC)" executed successfully 2020-03-31 09:49:03,332 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:49:03,332 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:03,333 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:23.410371+00:00 (in 20.077378 seconds) 2020-03-31 09:49:03,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:33 UTC)" (scheduled at 2020-03-31 09:49:03.330112+00:00) 2020-03-31 09:49:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:49:03,333 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:23.410371+00:00 (in 20.077378 seconds) 2020-03-31 09:49:03,333 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:33 UTC)" (scheduled at 2020-03-31 09:49:03.330112+00:00) 2020-03-31 09:49:03,333 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 3, 333942), True, datetime.datetime(2020, 3, 31, 9, 49, 3, 333942), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:49:03,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 3, 333942), True, datetime.datetime(2020, 3, 31, 9, 49, 3, 333942), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:49:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:49:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:49:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:33 UTC)" executed successfully 2020-03-31 09:49:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:49:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:14.323352+00:00 (in 9.998671 seconds) 2020-03-31 09:49:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:14 UTC)" (scheduled at 2020-03-31 09:49:04.323352+00:00) 2020-03-31 09:49:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:49:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:14.323352+00:00 (in 9.998671 seconds) 2020-03-31 09:49:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:14 UTC)" (scheduled at 2020-03-31 09:49:04.323352+00:00) 2020-03-31 09:49:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 4, 325388), True, datetime.datetime(2020, 3, 31, 9, 49, 4, 325388), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:49:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 4, 325388), True, datetime.datetime(2020, 3, 31, 9, 49, 4, 325388), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:49:04,344 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:49:04,344 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:49:04,344 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:04,344 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:14 UTC)" executed successfully 2020-03-31 09:49:04,445 [518] [DEBUG] [app] Starting request: urn:request:3018daf1-a30c-4b82-9a17-ef2d5e4ce504 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:04,445 [518] [DEBUG] [app] Starting request: urn:request:3018daf1-a30c-4b82-9a17-ef2d5e4ce504 (/health/instance) 2020-03-31 09:49:04,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:04,447 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:04,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:04,452 [540] [DEBUG] [app] Starting request: urn:request:e4a069e0-6be9-4961-913b-4f0506a9f195 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:04,452 [540] [DEBUG] [app] Starting request: urn:request:e4a069e0-6be9-4961-913b-4f0506a9f195 (/v1/_internal_ping) 2020-03-31 09:49:04,453 [540] [DEBUG] [app] Ending request: urn:request:e4a069e0-6be9-4961-913b-4f0506a9f195 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:04,453 [540] [DEBUG] [app] Ending request: urn:request:e4a069e0-6be9-4961-913b-4f0506a9f195 (/v1/_internal_ping) 2020-03-31 09:49:04,454 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:49:04,454 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:04,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:04,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:04,454 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:04,456 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:49:04,455 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:49:04,457 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:04,456 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:49:04,457 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:04,458 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:04,458 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:04,469 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:04,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:04,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:04,498 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:04,500 [527] [DEBUG] [app] Starting request: urn:request:3241b959-c81a-4ccb-afff-72c44c97aca8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:04,500 [527] [DEBUG] [app] Starting request: urn:request:3241b959-c81a-4ccb-afff-72c44c97aca8 (/_internal_ping) 2020-03-31 09:49:04,501 [527] [DEBUG] [app] Ending request: urn:request:3241b959-c81a-4ccb-afff-72c44c97aca8 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:04,501 [527] [DEBUG] [app] Ending request: urn:request:3241b959-c81a-4ccb-afff-72c44c97aca8 (/_internal_ping) 2020-03-31 09:49:04,502 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:49:04,502 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:04,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:49:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:04,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:49:04,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:49:04,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:04,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:49:04,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:04,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:04,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:04,514 [541] [DEBUG] [app] Starting request: urn:request:ffd34657-3b23-467f-96f0-284c7b98b463 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:04,514 [541] [DEBUG] [app] Starting request: urn:request:ffd34657-3b23-467f-96f0-284c7b98b463 (/c1/_internal_ping) 2020-03-31 09:49:04,514 [541] [DEBUG] [app] Ending request: urn:request:ffd34657-3b23-467f-96f0-284c7b98b463 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:04,514 [541] [DEBUG] [app] Ending request: urn:request:ffd34657-3b23-467f-96f0-284c7b98b463 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:04,515 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:04,515 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:49:04,515 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:04,515 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:04,517 [518] [DEBUG] [app] Ending request: urn:request:3018daf1-a30c-4b82-9a17-ef2d5e4ce504 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:04,517 [518] [DEBUG] [app] Ending request: urn:request:3018daf1-a30c-4b82-9a17-ef2d5e4ce504 (/health/instance) 2020-03-31 09:49:04,517 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:49:04,517 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:04,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) gunicorn-web stdout | 2020-03-31 09:49:04,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.073 49 0.073) 2020-03-31 09:49:04,586 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:db7913f4-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:04 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:49:04,586 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:db7913f4-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:04 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:49:04,586 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'db7913f4-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:pATrW8dFZTtEP8qlAKseyR57RfAa2oQGTI8MjlubFHo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:04 GMT'}. storagereplication stdout | 2020-03-31 09:49:04,586 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'db7913f4-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:pATrW8dFZTtEP8qlAKseyR57RfAa2oQGTI8MjlubFHo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:04 GMT'}. 2020-03-31 09:49:04,666 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:49:04,666 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:49:04,667 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:04 GMT, Server-Request-ID=47788758-501e-0041-3e41-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'db7913f4-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:pATrW8dFZTtEP8qlAKseyR57RfAa2oQGTI8MjlubFHo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:04 GMT'}. 2020-03-31 09:49:04,667 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:04 GMT, Server-Request-ID=47788758-501e-0041-3e41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:04,667 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:04 GMT, Server-Request-ID=47788758-501e-0041-3e41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:04,667 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:49:04,667 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:04 GMT, Server-Request-ID=47788758-501e-0041-3e41-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'db7913f4-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:pATrW8dFZTtEP8qlAKseyR57RfAa2oQGTI8MjlubFHo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:04 GMT'}. 2020-03-31 09:49:04,667 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:04 GMT, Server-Request-ID=47788758-501e-0041-3e41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:04,667 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=db7913f4-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:04 GMT, Server-Request-ID=47788758-501e-0041-3e41-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:04,667 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:49:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:49:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:16.924497+00:00 (in 9.999327 seconds) storagereplication stdout | 2020-03-31 09:49:06,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:06 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:16.924497+00:00 (in 9.999327 seconds) 2020-03-31 09:49:08,272 [518] [DEBUG] [app] Starting request: urn:request:eaa5fc63-f851-430f-a18d-18c7721709e6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:08,272 [518] [DEBUG] [app] Starting request: urn:request:eaa5fc63-f851-430f-a18d-18c7721709e6 (/health/instance) 2020-03-31 09:49:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:08,276 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:08,278 [553] [DEBUG] [app] Starting request: urn:request:dd8a80c5-080a-4452-9a44-5f3a69d1a25c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:08,278 [553] [DEBUG] [app] Starting request: urn:request:dd8a80c5-080a-4452-9a44-5f3a69d1a25c (/v1/_internal_ping) 2020-03-31 09:49:08,279 [553] [DEBUG] [app] Ending request: urn:request:dd8a80c5-080a-4452-9a44-5f3a69d1a25c (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:08,279 [553] [DEBUG] [app] Ending request: urn:request:dd8a80c5-080a-4452-9a44-5f3a69d1a25c (/v1/_internal_ping) 2020-03-31 09:49:08,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:49:08,280 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:08,280 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:49:08,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:49:08,281 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:49:08,282 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:08,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:08,284 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:08,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:08,285 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:08,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:08,289 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:08,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:08,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:08,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:08,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:08,311 [518] [DEBUG] [app] Starting request: urn:request:8eee03eb-d904-4c7a-9387-8c66f205dddf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:08,311 [518] [DEBUG] [app] Starting request: urn:request:8eee03eb-d904-4c7a-9387-8c66f205dddf (/_internal_ping) 2020-03-31 09:49:08,312 [518] [DEBUG] [app] Ending request: urn:request:8eee03eb-d904-4c7a-9387-8c66f205dddf (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:08,312 [518] [DEBUG] [app] Ending request: urn:request:8eee03eb-d904-4c7a-9387-8c66f205dddf (/_internal_ping) 2020-03-31 09:49:08,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:49:08,312 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:49:08,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:08,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:08,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:08,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:08,322 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:08,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:08,327 [537] [DEBUG] [app] Starting request: urn:request:574eb764-f9da-4e5f-9907-24813d5e2f53 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:08,327 [537] [DEBUG] [app] Starting request: urn:request:574eb764-f9da-4e5f-9907-24813d5e2f53 (/c1/_internal_ping) 2020-03-31 09:49:08,328 [537] [DEBUG] [app] Ending request: urn:request:574eb764-f9da-4e5f-9907-24813d5e2f53 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:08,328 [537] [DEBUG] [app] Ending request: urn:request:574eb764-f9da-4e5f-9907-24813d5e2f53 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:08,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:49:08,329 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:08,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:08,330 [518] [DEBUG] [app] Ending request: urn:request:eaa5fc63-f851-430f-a18d-18c7721709e6 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:08,330 [518] [DEBUG] [app] Ending request: urn:request:eaa5fc63-f851-430f-a18d-18c7721709e6 (/health/instance) 2020-03-31 09:49:08,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:49:08,330 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:08,331 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:49:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:49:11,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:21.440707+00:00 (in 9.995485 seconds) 2020-03-31 09:49:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:21 UTC)" (scheduled at 2020-03-31 09:49:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:49:11,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:21.440707+00:00 (in 9.995485 seconds) 2020-03-31 09:49:11,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:21 UTC)" (scheduled at 2020-03-31 09:49:11.440707+00:00) 2020-03-31 09:49:11,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:49:11,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:49:11,462 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:49:11,462 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:49:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:49:11,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:21 UTC)" executed successfully 2020-03-31 09:49:12,675 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e04b6aa8-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:12 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 storagereplication stdout | 2020-03-31 09:49:12,675 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e04b6aa8-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:12 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4 timeout:20 2020-03-31 09:49:12,675 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e04b6aa8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:9SAonkDRDVyVf/2v2g+UTnaf+RrTgzPHDU9tP3Dh0Jk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:12 GMT'}. storagereplication stdout | 2020-03-31 09:49:12,675 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e04b6aa8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:9SAonkDRDVyVf/2v2g+UTnaf+RrTgzPHDU9tP3Dh0Jk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:12 GMT'}. 2020-03-31 09:49:12,755 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:49:12,755 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:49:12,756 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:12 GMT, Server-Request-ID=4778a6d8-501e-0041-3241-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e04b6aa8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:9SAonkDRDVyVf/2v2g+UTnaf+RrTgzPHDU9tP3Dh0Jk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:12 GMT'}. 2020-03-31 09:49:12,757 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:12 GMT, Server-Request-ID=4778a6d8-501e-0041-3241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:12,757 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:12 GMT, Server-Request-ID=4778a6d8-501e-0041-3241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:12,757 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:49:12,756 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:12 GMT, Server-Request-ID=4778a6d8-501e-0041-3241-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e04b6aa8-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:9SAonkDRDVyVf/2v2g+UTnaf+RrTgzPHDU9tP3Dh0Jk=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:12 GMT'}. 2020-03-31 09:49:12,757 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:12 GMT, Server-Request-ID=4778a6d8-501e-0041-3241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:12,757 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=e04b6aa8-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:12 GMT, Server-Request-ID=4778a6d8-501e-0041-3241-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:12,757 [191] [DEBUG] [__main__] Cannot find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:49:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:49:14,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:24.323352+00:00 (in 9.999249 seconds) 2020-03-31 09:49:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:24 UTC)" (scheduled at 2020-03-31 09:49:14.323352+00:00) 2020-03-31 09:49:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:49:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:24.323352+00:00 (in 9.999249 seconds) 2020-03-31 09:49:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:24 UTC)" (scheduled at 2020-03-31 09:49:14.323352+00:00) 2020-03-31 09:49:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 14, 325136), True, datetime.datetime(2020, 3, 31, 9, 49, 14, 325136), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:49:14,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 14, 325136), True, datetime.datetime(2020, 3, 31, 9, 49, 14, 325136), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:49:14,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:49:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:49:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:24 UTC)" executed successfully 2020-03-31 09:49:14,446 [527] [DEBUG] [app] Starting request: urn:request:62e70cb0-caf4-4eea-8712-0698024a2d91 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:14,446 [527] [DEBUG] [app] Starting request: urn:request:62e70cb0-caf4-4eea-8712-0698024a2d91 (/health/instance) 2020-03-31 09:49:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:14,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:14,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:14,453 [552] [DEBUG] [app] Starting request: urn:request:054f6e39-24c4-4784-8fdf-d0f3459fc519 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:14,453 [552] [DEBUG] [app] Starting request: urn:request:054f6e39-24c4-4784-8fdf-d0f3459fc519 (/v1/_internal_ping) 2020-03-31 09:49:14,455 [552] [DEBUG] [app] Ending request: urn:request:054f6e39-24c4-4784-8fdf-d0f3459fc519 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:14,455 [552] [DEBUG] [app] Ending request: urn:request:054f6e39-24c4-4784-8fdf-d0f3459fc519 (/v1/_internal_ping) 2020-03-31 09:49:14,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:49:14,456 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:14,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:14,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:49:14,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:49:14,458 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:14,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:14,460 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:14,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:14,461 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:14,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:14,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:14,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:14,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:14,486 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:14,488 [527] [DEBUG] [app] Starting request: urn:request:469abfbc-df5b-4f01-bd43-168dc7932fa0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:14,488 [527] [DEBUG] [app] Starting request: urn:request:469abfbc-df5b-4f01-bd43-168dc7932fa0 (/_internal_ping) 2020-03-31 09:49:14,488 [527] [DEBUG] [app] Ending request: urn:request:469abfbc-df5b-4f01-bd43-168dc7932fa0 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:14,488 [527] [DEBUG] [app] Ending request: urn:request:469abfbc-df5b-4f01-bd43-168dc7932fa0 (/_internal_ping) 2020-03-31 09:49:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:49:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:49:14,489 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:14,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:49:14,493 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:49:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:14,496 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:14,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:14,500 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:14,501 [543] [DEBUG] [app] Starting request: urn:request:95d0b3dc-9910-4aaa-9b81-d5a8830039bb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:14,501 [543] [DEBUG] [app] Starting request: urn:request:95d0b3dc-9910-4aaa-9b81-d5a8830039bb (/c1/_internal_ping) 2020-03-31 09:49:14,502 [543] [DEBUG] [app] Ending request: urn:request:95d0b3dc-9910-4aaa-9b81-d5a8830039bb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:14,502 [543] [DEBUG] [app] Ending request: urn:request:95d0b3dc-9910-4aaa-9b81-d5a8830039bb (/c1/_internal_ping) 2020-03-31 09:49:14,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:14,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:14,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:14,504 [527] [DEBUG] [app] Ending request: urn:request:62e70cb0-caf4-4eea-8712-0698024a2d91 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:49:14,502 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:14,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:14,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:49:14,504 [527] [DEBUG] [app] Ending request: urn:request:62e70cb0-caf4-4eea-8712-0698024a2d91 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:14,504 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:14,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:49:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:49:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:45 UTC)" (scheduled at 2020-03-31 09:49:15.064514+00:00) gcworker stdout | 2020-03-31 09:49:15,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:45 UTC)" (scheduled at 2020-03-31 09:49:15.064514+00:00) 2020-03-31 09:49:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:45.064514+00:00 (in 29.999092 seconds) gcworker stdout | 2020-03-31 09:49:15,068 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:45.064514+00:00 (in 29.999092 seconds) 2020-03-31 09:49:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438555, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:49:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438555, 1209600, 500, 1, 0]) 2020-03-31 09:49:15,084 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:49:15,084 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:49:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:49:15,084 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:15,084 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:45 UTC)" executed successfully 2020-03-31 09:49:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:49:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:26.924497+00:00 (in 9.999181 seconds) storagereplication stdout | 2020-03-31 09:49:16,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:16 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:26.924497+00:00 (in 9.999181 seconds) 2020-03-31 09:49:18,273 [518] [DEBUG] [app] Starting request: urn:request:b426efa7-f492-4832-bb82-baa29a59226f (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:18,273 [518] [DEBUG] [app] Starting request: urn:request:b426efa7-f492-4832-bb82-baa29a59226f (/health/instance) 2020-03-31 09:49:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:18,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:18,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:18,280 [545] [DEBUG] [app] Starting request: urn:request:4bc431bb-e54b-4494-864b-e5e8b7f1ca17 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:18,280 [545] [DEBUG] [app] Starting request: urn:request:4bc431bb-e54b-4494-864b-e5e8b7f1ca17 (/v1/_internal_ping) 2020-03-31 09:49:18,281 [545] [DEBUG] [app] Ending request: urn:request:4bc431bb-e54b-4494-864b-e5e8b7f1ca17 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:18,281 [545] [DEBUG] [app] Ending request: urn:request:4bc431bb-e54b-4494-864b-e5e8b7f1ca17 (/v1/_internal_ping) 2020-03-31 09:49:18,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:49:18,281 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:18,283 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:49:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:49:18,283 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:18,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:18,286 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:18,286 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:18,286 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:18,291 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:18,311 [518] [DEBUG] [app] Starting request: urn:request:f082cbcc-9414-4d1f-9501-4e5161067920 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:18,311 [518] [DEBUG] [app] Starting request: urn:request:f082cbcc-9414-4d1f-9501-4e5161067920 (/_internal_ping) 2020-03-31 09:49:18,312 [518] [DEBUG] [app] Ending request: urn:request:f082cbcc-9414-4d1f-9501-4e5161067920 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:18,312 [518] [DEBUG] [app] Ending request: urn:request:f082cbcc-9414-4d1f-9501-4e5161067920 (/_internal_ping) 2020-03-31 09:49:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:18,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:18,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:18,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:18,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:18,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:18,325 [532] [DEBUG] [app] Starting request: urn:request:ada4fb43-b25f-4cf0-aae1-f36047103ad1 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:18,324 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:49:18,325 [532] [DEBUG] [app] Starting request: urn:request:ada4fb43-b25f-4cf0-aae1-f36047103ad1 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:18,326 [532] [DEBUG] [app] Ending request: urn:request:ada4fb43-b25f-4cf0-aae1-f36047103ad1 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:18,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:49:18,326 [532] [DEBUG] [app] Ending request: urn:request:ada4fb43-b25f-4cf0-aae1-f36047103ad1 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:18,327 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:18,328 [518] [DEBUG] [app] Ending request: urn:request:b426efa7-f492-4832-bb82-baa29a59226f (/health/instance) 2020-03-31 09:49:18,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:49:18,327 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:18,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:49:18,328 [518] [DEBUG] [app] Ending request: urn:request:b426efa7-f492-4832-bb82-baa29a59226f (/health/instance) 2020-03-31 09:49:18,328 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:49:18,329 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:49:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:49:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:31 UTC)" (scheduled at 2020-03-31 09:49:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:49:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:31 UTC)" (scheduled at 2020-03-31 09:49:21.440707+00:00) 2020-03-31 09:49:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:31.440707+00:00 (in 9.999084 seconds) repositoryactioncounter stdout | 2020-03-31 09:49:21,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:31.440707+00:00 (in 9.999084 seconds) 2020-03-31 09:49:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:49:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:49:21,457 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:49:21,457 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:49:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:49:21,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:21,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:31 UTC)" executed successfully 2020-03-31 09:49:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:49:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:52.113221+00:00 (in 29.999005 seconds) 2020-03-31 09:49:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:52 UTC)" (scheduled at 2020-03-31 09:49:22.113221+00:00) securityworker stdout | 2020-03-31 09:49:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:52.113221+00:00 (in 29.999005 seconds) 2020-03-31 09:49:22,114 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:52 UTC)" (scheduled at 2020-03-31 09:49:22.113221+00:00) 2020-03-31 09:49:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:49:22,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:49:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:49:22,127 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:49:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:49:22,130 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:49:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:49:22,130 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:22,130 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:52 UTC)" executed successfully 2020-03-31 09:49:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:49:23,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:24.410927+00:00 (in 0.999429 seconds) 2020-03-31 09:49:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:23 UTC)" (scheduled at 2020-03-31 09:49:23.410371+00:00) security_notification_worker stdout | 2020-03-31 09:49:23,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:24.410927+00:00 (in 0.999429 seconds) 2020-03-31 09:49:23,411 [168] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:23 UTC)" (scheduled at 2020-03-31 09:49:23.410371+00:00) 2020-03-31 09:49:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:49:23,413 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), u'security_notification/%']) 2020-03-31 09:49:23,427 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), True, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), 0, u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:49:23,427 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), True, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), 0, u'security_notification/%']) 2020-03-31 09:49:23,431 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), True, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), u'security_notification/%']) security_notification_worker stdout | 2020-03-31 09:49:23,431 [168] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), True, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), 0, u'security_notification/%', False, datetime.datetime(2020, 3, 31, 9, 49, 23, 412123), u'security_notification/%']) 2020-03-31 09:49:23,435 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:23,437 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 security_notification_worker stdout | 2020-03-31 09:49:23,435 [168] [DEBUG] [data.database] Disconnecting from database. security_notification_worker stdout | 2020-03-31 09:49:23,437 [168] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:23,438 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:23 UTC)" executed successfully 2020-03-31 09:49:23,438 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 security_notification_worker stdout | 2020-03-31 09:49:23,438 [168] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:23 UTC)" executed successfully 2020-03-31 09:49:23,438 [168] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:23,439 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics security_notification_worker stdout | 2020-03-31 09:49:23,439 [168] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:49:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:49:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:34.323352+00:00 (in 9.998603 seconds) 2020-03-31 09:49:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:34 UTC)" (scheduled at 2020-03-31 09:49:24.323352+00:00) 2020-03-31 09:49:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:49:24,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:34.323352+00:00 (in 9.998603 seconds) 2020-03-31 09:49:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:34 UTC)" (scheduled at 2020-03-31 09:49:24.323352+00:00) 2020-03-31 09:49:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 24, 325565), True, datetime.datetime(2020, 3, 31, 9, 49, 24, 325565), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:49:24,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 24, 325565), True, datetime.datetime(2020, 3, 31, 9, 49, 24, 325565), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:49:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:49:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:49:24,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:34 UTC)" executed successfully 2020-03-31 09:49:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:49:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:33.330112+00:00 (in 8.918480 seconds) 2020-03-31 09:49:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:24 UTC)" (scheduled at 2020-03-31 09:49:24.410927+00:00) 2020-03-31 09:49:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. security_notification_worker stdout | 2020-03-31 09:49:24,411 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:33.330112+00:00 (in 8.918480 seconds) 2020-03-31 09:49:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:24 UTC)" (scheduled at 2020-03-31 09:49:24.410927+00:00) 2020-03-31 09:49:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:49:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:24 UTC)" executed successfully 2020-03-31 09:49:24,446 [518] [DEBUG] [app] Starting request: urn:request:c2693af0-f73e-4d40-8a75-541d3e225e36 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:24,446 [518] [DEBUG] [app] Starting request: urn:request:c2693af0-f73e-4d40-8a75-541d3e225e36 (/health/instance) 2020-03-31 09:49:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:24,453 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:24,454 [554] [DEBUG] [app] Starting request: urn:request:b8d62819-7327-4310-8dc5-3462d5b02c5b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:24,454 [554] [DEBUG] [app] Starting request: urn:request:b8d62819-7327-4310-8dc5-3462d5b02c5b (/v1/_internal_ping) 2020-03-31 09:49:24,455 [554] [DEBUG] [app] Ending request: urn:request:b8d62819-7327-4310-8dc5-3462d5b02c5b (/v1/_internal_ping) 2020-03-31 09:49:24,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:49:24,455 [554] [DEBUG] [app] Ending request: urn:request:b8d62819-7327-4310-8dc5-3462d5b02c5b (/v1/_internal_ping) 2020-03-31 09:49:24,456 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:24,457 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:24,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:49:24,458 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:49:24,458 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:49:24,461 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:24,461 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:24,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:24,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:24,470 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:24,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:24,482 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:24,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:24,487 [518] [DEBUG] [app] Starting request: urn:request:00551984-ceae-4d3c-a9a2-b375a48eebe2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:24,487 [518] [DEBUG] [app] Starting request: urn:request:00551984-ceae-4d3c-a9a2-b375a48eebe2 (/_internal_ping) 2020-03-31 09:49:24,488 [518] [DEBUG] [app] Ending request: urn:request:00551984-ceae-4d3c-a9a2-b375a48eebe2 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:24,488 [518] [DEBUG] [app] Ending request: urn:request:00551984-ceae-4d3c-a9a2-b375a48eebe2 (/_internal_ping) 2020-03-31 09:49:24,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:24,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:24,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:24,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:24,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:24,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:49:24,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:49:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:24,495 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:49:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:24,496 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:24,500 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:24,501 [534] [DEBUG] [app] Starting request: urn:request:20887cb6-d28e-4041-be34-05f0ae83e584 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:24,501 [534] [DEBUG] [app] Starting request: urn:request:20887cb6-d28e-4041-be34-05f0ae83e584 (/c1/_internal_ping) 2020-03-31 09:49:24,503 [534] [DEBUG] [app] Ending request: urn:request:20887cb6-d28e-4041-be34-05f0ae83e584 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:24,503 [534] [DEBUG] [app] Ending request: urn:request:20887cb6-d28e-4041-be34-05f0ae83e584 (/c1/_internal_ping) 2020-03-31 09:49:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:24,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:24,503 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:24,505 [518] [DEBUG] [app] Ending request: urn:request:c2693af0-f73e-4d40-8a75-541d3e225e36 (/health/instance) 2020-03-31 09:49:24,505 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:49:24,503 [534] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 10.131.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:49:24,505 [518] [DEBUG] [app] Ending request: urn:request:c2693af0-f73e-4d40-8a75-541d3e225e36 (/health/instance) 2020-03-31 09:49:24,505 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:24,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:49:24,505 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:49:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:49:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:36.924497+00:00 (in 9.999274 seconds) storagereplication stdout | 2020-03-31 09:49:26,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:26 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:36.924497+00:00 (in 9.999274 seconds) 2020-03-31 09:49:28,273 [523] [DEBUG] [app] Starting request: urn:request:7a299257-2d63-4efb-8c96-f25b4db0694a (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:28,273 [523] [DEBUG] [app] Starting request: urn:request:7a299257-2d63-4efb-8c96-f25b4db0694a (/health/instance) 2020-03-31 09:49:28,276 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:28,276 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:28,282 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:28,282 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:28,284 [538] [DEBUG] [app] Starting request: urn:request:4785a0fd-2485-4ef5-a21f-219d519825e6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:28,284 [538] [DEBUG] [app] Starting request: urn:request:4785a0fd-2485-4ef5-a21f-219d519825e6 (/v1/_internal_ping) 2020-03-31 09:49:28,285 [538] [DEBUG] [app] Ending request: urn:request:4785a0fd-2485-4ef5-a21f-219d519825e6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:28,285 [538] [DEBUG] [app] Ending request: urn:request:4785a0fd-2485-4ef5-a21f-219d519825e6 (/v1/_internal_ping) 2020-03-31 09:49:28,286 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:28,288 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:28,286 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:28,287 [523] [INFO] [data.database] Connection pooling disabled for postgresql 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:49:28,286 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:28,288 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:28,290 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:28,291 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-web stdout | 2020-03-31 09:49:28,286 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:28,287 [523] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:49:28,290 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:28,291 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:28,299 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:28,299 [523] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:28,315 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:28,315 [523] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:28,319 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:28,319 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:28,321 [526] [DEBUG] [app] Starting request: urn:request:57343a8f-993a-41a6-b11f-487fe8f45d1e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:28,321 [526] [DEBUG] [app] Starting request: urn:request:57343a8f-993a-41a6-b11f-487fe8f45d1e (/_internal_ping) 2020-03-31 09:49:28,323 [526] [DEBUG] [app] Ending request: urn:request:57343a8f-993a-41a6-b11f-487fe8f45d1e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:28,323 [526] [DEBUG] [app] Ending request: urn:request:57343a8f-993a-41a6-b11f-487fe8f45d1e (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:49:28,323 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:28,323 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:49:28,323 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:28,323 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:28,328 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:28,328 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:28,329 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:28,329 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:49:28,332 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:28,332 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:28,336 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:28,336 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:28,340 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:28,344 [537] [DEBUG] [app] Starting request: urn:request:390cced9-ee12-49e4-979a-70644bd3b080 (/c1/_internal_ping) 2020-03-31 09:49:28,346 [537] [DEBUG] [app] Ending request: urn:request:390cced9-ee12-49e4-979a-70644bd3b080 (/c1/_internal_ping) 2020-03-31 09:49:28,346 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:28,347 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) 2020-03-31 09:49:28,350 [523] [DEBUG] [app] Ending request: urn:request:7a299257-2d63-4efb-8c96-f25b4db0694a (/health/instance) 2020-03-31 09:49:28,350 [523] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.079 49 0.079) 2020-03-31 09:49:28,352 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:49:28,340 [523] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-verbs stdout | 2020-03-31 09:49:28,344 [537] [DEBUG] [app] Starting request: urn:request:390cced9-ee12-49e4-979a-70644bd3b080 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:28,346 [537] [DEBUG] [app] Ending request: urn:request:390cced9-ee12-49e4-979a-70644bd3b080 (/c1/_internal_ping) 2020-03-31 09:49:28,346 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:28,347 [523] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-web stdout | 2020-03-31 09:49:28,350 [523] [DEBUG] [app] Ending request: urn:request:7a299257-2d63-4efb-8c96-f25b4db0694a (/health/instance) 2020-03-31 09:49:28,350 [523] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.079 49 0.079) gunicorn-web stdout | 2020-03-31 09:49:28,352 [523] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:49:28,773 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:49:28,773 [191] [WARNING] [__main__] Failed to find path `sha256/22/224a78d184bb6077a1a6ce92e84993712ed1fc0dc979daa7865c249c24b97ed4` in location `azure-uk` after copy 2020-03-31 09:49:28,773 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:49:28,773 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:49:28,773 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "d910da7a-e575-40de-8965-a8fbd51c3b85"} 2020-03-31 09:49:28,773 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:49:28,783 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) storagereplication stdout | 2020-03-31 09:49:28,783 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [2, 1, 0]) 2020-03-31 09:49:28,787 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 54, 28, 782749), True, u'05feaa4d-f4fc-45ce-bf2f-ad8f988c50ba', 2]) storagereplication stdout | 2020-03-31 09:49:28,787 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 54, 28, 782749), True, u'05feaa4d-f4fc-45ce-bf2f-ad8f988c50ba', 2]) 2020-03-31 09:49:28,794 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 28, 792304), True, datetime.datetime(2020, 3, 31, 9, 49, 28, 792304), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:49:28,794 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 28, 792304), True, datetime.datetime(2020, 3, 31, 9, 49, 28, 792304), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:49:28,799 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 9, 28, 792304), 1, u'742b8fac-f8e7-46d5-b026-a47ddcecd14c', 1, u'5a59868b-ad0f-4f19-8f8f-2a4de8685fec']) storagereplication stdout | 2020-03-31 09:49:28,799 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available" = %s, "processing_expires" = %s, "retries_remaining" = ("queueitem"."retries_remaining" - %s), "state_id" = %s WHERE (("queueitem"."id" = %s) AND ("queueitem"."state_id" = %s))', [False, datetime.datetime(2020, 3, 31, 10, 9, 28, 792304), 1, u'742b8fac-f8e7-46d5-b026-a47ddcecd14c', 1, u'5a59868b-ad0f-4f19-8f8f-2a4de8685fec']) 2020-03-31 09:49:28,805 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} storagereplication stdout | 2020-03-31 09:49:28,805 [191] [DEBUG] [workers.queueworker] Queue gave us some work: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:49:28,805 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:28,805 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 storagereplication stdout | 2020-03-31 09:49:28,805 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:28,805 [191] [DEBUG] [__main__] Starting replication of image storage 7a10c766-d963-4234-89c4-998a563408cc under namespace 1 2020-03-31 09:49:28,808 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) storagereplication stdout | 2020-03-31 09:49:28,808 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."id" = %s) AND ("t1"."robot" = %s)) LIMIT %s OFFSET %s', [1, False, 1, 0]) 2020-03-31 09:49:28,822 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) storagereplication stdout | 2020-03-31 09:49:28,822 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."location_id" FROM "userregion" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."user_id" = %s)', [1]) 2020-03-31 09:49:28,827 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) storagereplication stdout | 2020-03-31 09:49:28,827 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', [u'7a10c766-d963-4234-89c4-998a563408cc']) 2020-03-31 09:49:28,831 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing storagereplication stdout | 2020-03-31 09:49:28,831 [191] [DEBUG] [__main__] For replication of storage 7a10c766-d963-4234-89c4-998a563408cc under namespace admin: set(['azure-us', 'azure-uk']) required; set(['azure-uk']) missing 2020-03-31 09:49:28,831 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e9eca784-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:49:28,831 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9eca784-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9eca784-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ey/vv8OUawuVn1N71Dtfowfir9qKDc8rFLxXwu1YgQo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. storagereplication stdout | 2020-03-31 09:49:28,831 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e9eca784-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:49:28,831 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9eca784-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9eca784-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ey/vv8OUawuVn1N71Dtfowfir9qKDc8rFLxXwu1YgQo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. 2020-03-31 09:49:28,840 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 storagereplication stdout | 2020-03-31 09:49:28,840 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 200 0 2020-03-31 09:49:28,841 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9eca784-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=dd73920e-b01e-000c-7d41-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9eca784-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ey/vv8OUawuVn1N71Dtfowfir9qKDc8rFLxXwu1YgQo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. storagereplication stdout | 2020-03-31 09:49:28,841 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9eca784-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=dd73920e-b01e-000c-7d41-074867000000, HTTP Status Code=200, Message=OK, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9eca784-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:Ey/vv8OUawuVn1N71Dtfowfir9qKDc8rFLxXwu1YgQo=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. 2020-03-31 09:49:28,843 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:49:28,843 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:49:28,843 [191] [DEBUG] [__main__] Starting copy of storage 7a10c766-d963-4234-89c4-998a563408cc to location azure-uk from azure-us 2020-03-31 09:49:28,843 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:28,843 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy storagereplication stdout | 2020-03-31 09:49:28,843 [191] [DEBUG] [storage.azurestorage] Starting copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:49:28,844 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:e9ee97ba-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=PSCTHVWVT%2BqO7tFNGUuYSl95DYdiXWSPs44JYFLaoJ8%3D&sv=2017-07-29&se=2020-03-31T09%3A50%3A28Z x-ms-date:Tue, 31 Mar 2020 09:49:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:49:28,844 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9ee97ba-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=PSCTHVWVT%2BqO7tFNGUuYSl95DYdiXWSPs44JYFLaoJ8%3D&sv=2017-07-29&se=2020-03-31T09%3A50%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BK7mVe3SdQHngr1CerZBOeTgsGhXZKzN3acnr3dAxNI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e9ee97ba-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. storagereplication stdout | 2020-03-31 09:49:28,844 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=PUT x-ms-client-request-id:e9ee97ba-7334-11ea-a15c-0a580a830015 x-ms-copy-source:https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=PSCTHVWVT%2BqO7tFNGUuYSl95DYdiXWSPs44JYFLaoJ8%3D&sv=2017-07-29&se=2020-03-31T09%3A50%3A28Z x-ms-date:Tue, 31 Mar 2020 09:49:28 GMT x-ms-version:2017-07-29 /quaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 2020-03-31 09:49:28,844 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9ee97ba-7334-11ea-a15c-0a580a830015 Outgoing request: Method=PUT, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'timeout': None}, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=PSCTHVWVT%2BqO7tFNGUuYSl95DYdiXWSPs44JYFLaoJ8%3D&sv=2017-07-29&se=2020-03-31T09%3A50%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BK7mVe3SdQHngr1CerZBOeTgsGhXZKzN3acnr3dAxNI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e9ee97ba-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. 2020-03-31 09:49:28,874 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 storagereplication stdout | 2020-03-31 09:49:28,874 [191] [DEBUG] [urllib3.connectionpool] https://quaybackendbmwuskq7s7.blob.core.windows.net:443 "PUT /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 HTTP/1.1" 202 0 2020-03-31 09:49:28,875 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9ee97ba-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=dd739212-b01e-000c-7f41-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=PSCTHVWVT%2BqO7tFNGUuYSl95DYdiXWSPs44JYFLaoJ8%3D&sv=2017-07-29&se=2020-03-31T09%3A50%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BK7mVe3SdQHngr1CerZBOeTgsGhXZKzN3acnr3dAxNI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e9ee97ba-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. storagereplication stdout | 2020-03-31 09:49:28,875 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9ee97ba-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=dd739212-b01e-000c-7f41-074867000000, HTTP Status Code=202, Message=Accepted, Headers={'x-ms-version': '2017-07-29', 'x-ms-source-if-unmodified-since': None, 'x-ms-copy-source': 'https://quaybackendbmwuskq7s7.blob.core.windows.net/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?sr=b&sp=r&sig=PSCTHVWVT%2BqO7tFNGUuYSl95DYdiXWSPs44JYFLaoJ8%3D&sv=2017-07-29&se=2020-03-31T09%3A50%3A28Z', 'x-ms-source-if-match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'x-ms-access-tier': None, 'Authorization': u'SharedKey quaybackendbmwuskq7s7:BK7mVe3SdQHngr1CerZBOeTgsGhXZKzN3acnr3dAxNI=', 'x-ms-source-if-modified-since': None, 'x-ms-client-request-id': 'e9ee97ba-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'x-ms-source-lease-id': None, 'x-ms-source-if-none-match': None, 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. 2020-03-31 09:49:28,875 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:49:28,876 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e9f3787a-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:28 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:49:28,876 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9f3787a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qymz9Y41huHlqw3ueseg2gZbbHNL8KsrCqPcHgvHy8M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. storagereplication stdout | 2020-03-31 09:49:28,875 [191] [DEBUG] [storage.azurestorage] Finished copying file from Azure quayregistry to Azure via an Azure copy 2020-03-31 09:49:28,876 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:e9f3787a-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:28 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:49:28,876 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9f3787a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qymz9Y41huHlqw3ueseg2gZbbHNL8KsrCqPcHgvHy8M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. 2020-03-31 09:49:28,959 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:49:28,959 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:49:28,960 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=4778e325-501e-0041-1041-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9f3787a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qymz9Y41huHlqw3ueseg2gZbbHNL8KsrCqPcHgvHy8M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. 2020-03-31 09:49:28,960 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=4778e325-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:28,960 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=4778e325-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:28,960 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds storagereplication stdout | 2020-03-31 09:49:28,960 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=4778e325-501e-0041-1041-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'e9f3787a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:qymz9Y41huHlqw3ueseg2gZbbHNL8KsrCqPcHgvHy8M=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:28 GMT'}. 2020-03-31 09:49:28,960 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=4778e325-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:28,960 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=e9f3787a-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:28 GMT, Server-Request-ID=4778e325-501e-0041-1041-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:28,960 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #0). Sleeping for 2 seconds 2020-03-31 09:49:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:49:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:00.337699+00:00 (in 29.998847 seconds) 2020-03-31 09:49:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:00 UTC)" (scheduled at 2020-03-31 09:49:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:49:30,339 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:00.337699+00:00 (in 29.998847 seconds) 2020-03-31 09:49:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:00 UTC)" (scheduled at 2020-03-31 09:49:30.337699+00:00) 2020-03-31 09:49:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 49, 30, 339602), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:49:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 49, 30, 339602), False, 50, 1, 0]) 2020-03-31 09:49:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:49:30,358 [173] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2020-03-31 09:49:30,357 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:49:30,358 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:49:30,358 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:00 UTC)" executed successfully 2020-03-31 09:49:30,963 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:eb31e71c-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:30 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:49:30,963 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:eb31e71c-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:30 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:49:30,963 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eb31e71c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RzIPYQxXWC9CNyuK0sSNT4BxRlsppJ1r2zRlDE8qnn4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:30 GMT'}. storagereplication stdout | 2020-03-31 09:49:30,963 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eb31e71c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RzIPYQxXWC9CNyuK0sSNT4BxRlsppJ1r2zRlDE8qnn4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:30 GMT'}. 2020-03-31 09:49:31,042 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:49:31,042 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:49:31,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:30 GMT, Server-Request-ID=4778eb39-501e-0041-3941-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eb31e71c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RzIPYQxXWC9CNyuK0sSNT4BxRlsppJ1r2zRlDE8qnn4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:30 GMT'}. 2020-03-31 09:49:31,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:30 GMT, Server-Request-ID=4778eb39-501e-0041-3941-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:31,043 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:30 GMT, Server-Request-ID=4778eb39-501e-0041-3941-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:31,043 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds storagereplication stdout | 2020-03-31 09:49:31,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:30 GMT, Server-Request-ID=4778eb39-501e-0041-3941-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eb31e71c-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:RzIPYQxXWC9CNyuK0sSNT4BxRlsppJ1r2zRlDE8qnn4=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:30 GMT'}. 2020-03-31 09:49:31,043 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:30 GMT, Server-Request-ID=4778eb39-501e-0041-3941-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:31,043 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=eb31e71c-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:30 GMT, Server-Request-ID=4778eb39-501e-0041-3941-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:31,043 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #1). Sleeping for 4 seconds 2020-03-31 09:49:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:49:31,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:41.440707+00:00 (in 9.996756 seconds) 2020-03-31 09:49:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:41 UTC)" (scheduled at 2020-03-31 09:49:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:49:31,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:41.440707+00:00 (in 9.996756 seconds) 2020-03-31 09:49:31,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:41 UTC)" (scheduled at 2020-03-31 09:49:31.440707+00:00) 2020-03-31 09:49:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:49:31,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:49:31,464 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:49:31,464 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:49:31,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:31,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:49:31,464 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:31,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:41 UTC)" executed successfully 2020-03-31 09:49:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:49:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:03.330112+00:00 (in 29.999049 seconds) 2020-03-31 09:49:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:03 UTC)" (scheduled at 2020-03-31 09:49:33.330112+00:00) 2020-03-31 09:49:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:49:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:03.330112+00:00 (in 29.999049 seconds) 2020-03-31 09:49:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:03 UTC)" (scheduled at 2020-03-31 09:49:33.330112+00:00) 2020-03-31 09:49:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 33, 332249), True, datetime.datetime(2020, 3, 31, 9, 49, 33, 332249), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:49:33,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 33, 332249), True, datetime.datetime(2020, 3, 31, 9, 49, 33, 332249), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:49:33,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:49:33,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:49:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:03 UTC)" executed successfully 2020-03-31 09:49:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:49:33,367 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:43.539999+00:00 (in 10.172167 seconds) 2020-03-31 09:49:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:33 UTC)" (scheduled at 2020-03-31 09:49:33.366791+00:00) 2020-03-31 09:49:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:49:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:43.539999+00:00 (in 10.172167 seconds) 2020-03-31 09:49:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:33 UTC)" (scheduled at 2020-03-31 09:49:33.366791+00:00) 2020-03-31 09:49:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 33, 368702), True, datetime.datetime(2020, 3, 31, 9, 49, 33, 368702), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:49:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 33, 368702), True, datetime.datetime(2020, 3, 31, 9, 49, 33, 368702), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:49:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:49:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:49:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:33 UTC)" executed successfully 2020-03-31 09:49:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:49:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:44.323352+00:00 (in 9.999318 seconds) 2020-03-31 09:49:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:44 UTC)" (scheduled at 2020-03-31 09:49:34.323352+00:00) 2020-03-31 09:49:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:49:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:44.323352+00:00 (in 9.999318 seconds) 2020-03-31 09:49:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:44 UTC)" (scheduled at 2020-03-31 09:49:34.323352+00:00) 2020-03-31 09:49:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 34, 324980), True, datetime.datetime(2020, 3, 31, 9, 49, 34, 324980), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:49:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 34, 324980), True, datetime.datetime(2020, 3, 31, 9, 49, 34, 324980), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:49:34,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:49:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:49:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:44 UTC)" executed successfully 2020-03-31 09:49:34,446 [527] [DEBUG] [app] Starting request: urn:request:48e7a8b4-c088-4f56-9f03-0360f7f861e0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:34,446 [527] [DEBUG] [app] Starting request: urn:request:48e7a8b4-c088-4f56-9f03-0360f7f861e0 (/health/instance) 2020-03-31 09:49:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:34,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:34,453 [549] [DEBUG] [app] Starting request: urn:request:35a25f56-b17a-4de7-9292-d944fcf084dd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:34,453 [549] [DEBUG] [app] Starting request: urn:request:35a25f56-b17a-4de7-9292-d944fcf084dd (/v1/_internal_ping) 2020-03-31 09:49:34,454 [549] [DEBUG] [app] Ending request: urn:request:35a25f56-b17a-4de7-9292-d944fcf084dd (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:34,454 [549] [DEBUG] [app] Ending request: urn:request:35a25f56-b17a-4de7-9292-d944fcf084dd (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:34,454 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:49:34,454 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:49:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:49:34,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:49:34,457 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:34,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:34,460 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:34,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:34,460 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:34,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:34,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:34,482 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:34,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:34,485 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:34,486 [527] [DEBUG] [app] Starting request: urn:request:e335f069-6b3e-4171-9be0-3b91f69d7c56 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:34,486 [527] [DEBUG] [app] Starting request: urn:request:e335f069-6b3e-4171-9be0-3b91f69d7c56 (/_internal_ping) 2020-03-31 09:49:34,487 [527] [DEBUG] [app] Ending request: urn:request:e335f069-6b3e-4171-9be0-3b91f69d7c56 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:34,487 [527] [DEBUG] [app] Ending request: urn:request:e335f069-6b3e-4171-9be0-3b91f69d7c56 (/_internal_ping) 2020-03-31 09:49:34,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:34,488 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:34,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:34,488 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:34,491 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:34,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:49:34,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:49:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:34,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:34,500 [537] [DEBUG] [app] Starting request: urn:request:f833bb99-5f4b-4130-ba51-120a098707fb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:34,500 [537] [DEBUG] [app] Starting request: urn:request:f833bb99-5f4b-4130-ba51-120a098707fb (/c1/_internal_ping) 2020-03-31 09:49:34,501 [537] [DEBUG] [app] Ending request: urn:request:f833bb99-5f4b-4130-ba51-120a098707fb (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:34,501 [537] [DEBUG] [app] Ending request: urn:request:f833bb99-5f4b-4130-ba51-120a098707fb (/c1/_internal_ping) 2020-03-31 09:49:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:34,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:34,502 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:49:34,502 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:34,503 [527] [DEBUG] [app] Ending request: urn:request:48e7a8b4-c088-4f56-9f03-0360f7f861e0 (/health/instance) 2020-03-31 09:49:34,503 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:49:34,503 [527] [DEBUG] [app] Ending request: urn:request:48e7a8b4-c088-4f56-9f03-0360f7f861e0 (/health/instance) 2020-03-31 09:49:34,503 [527] [DEBUG] [data.database] Disconnecting from database. 10.131.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:49:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:49:34,504 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:49:35,048 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:eda13656-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:35 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:49:35,048 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:eda13656-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:35 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:49:35,048 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eda13656-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hsdijq7+ecASjuzReuipfrem0jRAAGiA28A+C40dbMg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:35 GMT'}. storagereplication stdout | 2020-03-31 09:49:35,048 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eda13656-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hsdijq7+ecASjuzReuipfrem0jRAAGiA28A+C40dbMg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:35 GMT'}. 2020-03-31 09:49:35,128 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:49:35,128 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:49:35,129 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:35 GMT, Server-Request-ID=4778fcf3-501e-0041-7641-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eda13656-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hsdijq7+ecASjuzReuipfrem0jRAAGiA28A+C40dbMg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:35 GMT'}. 2020-03-31 09:49:35,129 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:35 GMT, Server-Request-ID=4778fcf3-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:35,130 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:35 GMT, Server-Request-ID=4778fcf3-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:49:35,129 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:35 GMT, Server-Request-ID=4778fcf3-501e-0041-7641-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'eda13656-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:hsdijq7+ecASjuzReuipfrem0jRAAGiA28A+C40dbMg=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:35 GMT'}. 2020-03-31 09:49:35,129 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:35 GMT, Server-Request-ID=4778fcf3-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:35,130 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=eda13656-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:35 GMT, Server-Request-ID=4778fcf3-501e-0041-7641-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:35,130 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds storagereplication stdout | 2020-03-31 09:49:35,130 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #2). Sleeping for 8 seconds 2020-03-31 09:49:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:49:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:38.010934+00:00 (in 1.085658 seconds) storagereplication stdout | 2020-03-31 09:49:36,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:36 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:38.010934+00:00 (in 1.085658 seconds) 2020-03-31 09:49:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:49:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:46.924497+00:00 (in 8.912778 seconds) 2020-03-31 09:49:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:38 UTC)" (scheduled at 2020-03-31 09:49:38.010934+00:00) 2020-03-31 09:49:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:49:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:46.924497+00:00 (in 8.912778 seconds) 2020-03-31 09:49:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:38 UTC)" (scheduled at 2020-03-31 09:49:38.010934+00:00) 2020-03-31 09:49:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:38 UTC)" executed successfully 2020-03-31 09:49:38,273 [527] [DEBUG] [app] Starting request: urn:request:63767b2f-2aee-4a8d-89fe-d9bec4de1044 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:38,273 [527] [DEBUG] [app] Starting request: urn:request:63767b2f-2aee-4a8d-89fe-d9bec4de1044 (/health/instance) 2020-03-31 09:49:38,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:38,277 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:38,279 [555] [DEBUG] [app] Starting request: urn:request:6e4610e5-2afb-4624-914c-dd03a026655d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:38,279 [555] [DEBUG] [app] Starting request: urn:request:6e4610e5-2afb-4624-914c-dd03a026655d (/v1/_internal_ping) 2020-03-31 09:49:38,280 [555] [DEBUG] [app] Ending request: urn:request:6e4610e5-2afb-4624-914c-dd03a026655d (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:38,280 [555] [DEBUG] [app] Ending request: urn:request:6e4610e5-2afb-4624-914c-dd03a026655d (/v1/_internal_ping) 2020-03-31 09:49:38,281 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:38,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:49:38,281 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:38,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:38,281 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:38,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:49:38,282 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:49:38,283 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:38,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:38,284 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:38,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:38,285 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:38,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:38,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:38,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:38,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:38,308 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:38,309 [527] [DEBUG] [app] Starting request: urn:request:fc3d2a31-9b71-48ec-8aaa-6bb7b3ed1c6b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:38,309 [527] [DEBUG] [app] Starting request: urn:request:fc3d2a31-9b71-48ec-8aaa-6bb7b3ed1c6b (/_internal_ping) 2020-03-31 09:49:38,310 [527] [DEBUG] [app] Ending request: urn:request:fc3d2a31-9b71-48ec-8aaa-6bb7b3ed1c6b (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:38,310 [527] [DEBUG] [app] Ending request: urn:request:fc3d2a31-9b71-48ec-8aaa-6bb7b3ed1c6b (/_internal_ping) 2020-03-31 09:49:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:38,311 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:38,311 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:38,311 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:38,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:38,314 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:38,315 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:38,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:38,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:38,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:38,323 [537] [DEBUG] [app] Starting request: urn:request:78898387-2f39-4e20-84be-e437100eb050 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:38,323 [537] [DEBUG] [app] Starting request: urn:request:78898387-2f39-4e20-84be-e437100eb050 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:38,324 [537] [DEBUG] [app] Ending request: urn:request:78898387-2f39-4e20-84be-e437100eb050 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:38,325 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:49:38,324 [537] [DEBUG] [app] Ending request: urn:request:78898387-2f39-4e20-84be-e437100eb050 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:38,326 [527] [DEBUG] [app] Ending request: urn:request:63767b2f-2aee-4a8d-89fe-d9bec4de1044 (/health/instance) 2020-03-31 09:49:38,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:49:38,325 [537] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:38,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:49:38,326 [527] [DEBUG] [app] Ending request: urn:request:63767b2f-2aee-4a8d-89fe-d9bec4de1044 (/health/instance) 2020-03-31 09:49:38,326 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:49:38,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:49:39,157 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2020-03-31 09:49:39,157 [165] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:39,157 [165] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:44:39.155593+00:00 (in 3299.998271 seconds) 2020-03-31 09:49:39,158 [165] [INFO] [apscheduler.executors.default] Running job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 10:44:39 UTC)" (scheduled at 2020-03-31 09:49:39.155593+00:00) 2020-03-31 09:49:39,158 [165] [DEBUG] [__main__] Starting automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 to new expiration 2020-03-31 11:49:39.158403 servicekey stdout | 2020-03-31 09:49:39,157 [165] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 10:44:39.155593+00:00 (in 3299.998271 seconds) 2020-03-31 09:49:39,158 [165] [INFO] [apscheduler.executors.default] Running job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 10:44:39 UTC)" (scheduled at 2020-03-31 09:49:39.155593+00:00) 2020-03-31 09:49:39,158 [165] [DEBUG] [__main__] Starting automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 to new expiration 2020-03-31 11:49:39.158403 2020-03-31 09:49:39,159 [165] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', [u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', datetime.datetime(2020, 3, 24, 9, 49, 39, 158777), None, 1, 0]) servicekey stdout | 2020-03-31 09:49:39,159 [165] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE (("t1"."kid" = %s) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s))) LIMIT %s OFFSET %s', [u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', datetime.datetime(2020, 3, 24, 9, 49, 39, 158777), None, 1, 0]) 2020-03-31 09:49:39,174 [165] [DEBUG] [peewee] ('UPDATE "servicekey" SET "name" = %s, "kid" = %s, "service" = %s, "jwk" = %s, "metadata" = %s, "created_date" = %s, "expiration_date" = %s, "rotation_duration" = %s, "approval_id" = %s WHERE ("servicekey"."id" = %s)', [u'https://quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com:443', u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', u'quay', '{"e": "AQAB", "kty": "RSA", "n": "x2cjbBkjdqblm4sOgP83bzbi9iWUQh70dRViEYlT_NPxaj0tU4s1IVaiU2gADr4n0LORlbpr6aaSOh5QWeDu4RkWFgbCmDGqBfnMioSyt2wMpcmBunBUDGh8Lw9t86_XTm-KzDZ9OdcRpwaA7qKabZjH-kD8U6txl_rPkkzC5xoEhjr_l4U-pXq8dW65pB8iecP0h8mbr6xflR1TlR20QDvB7nzm59rj1ZtvbvqOyOQo5XHvT-8JBK7FSfBi7GQgSc4zZG7UBUcSnBR6l6nxKbuPtceesAm1PGy6xV2wVpIHGfn6dIHJ4uEwCRVqEumR2qZonmZzmSgROCgooVOnkw"}', '{"created_by": "CLI tool"}', datetime.datetime(2020, 3, 31, 8, 49, 54, 471685), datetime.datetime(2020, 3, 31, 11, 49, 39, 158403), None, 1, 1]) servicekey stdout | 2020-03-31 09:49:39,174 [165] [DEBUG] [peewee] ('UPDATE "servicekey" SET "name" = %s, "kid" = %s, "service" = %s, "jwk" = %s, "metadata" = %s, "created_date" = %s, "expiration_date" = %s, "rotation_duration" = %s, "approval_id" = %s WHERE ("servicekey"."id" = %s)', [u'https://quay-enterprise.apps.bmwus.fabric-ocpv4-poc.com:443', u'29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4', u'quay', '{"e": "AQAB", "kty": "RSA", "n": "x2cjbBkjdqblm4sOgP83bzbi9iWUQh70dRViEYlT_NPxaj0tU4s1IVaiU2gADr4n0LORlbpr6aaSOh5QWeDu4RkWFgbCmDGqBfnMioSyt2wMpcmBunBUDGh8Lw9t86_XTm-KzDZ9OdcRpwaA7qKabZjH-kD8U6txl_rPkkzC5xoEhjr_l4U-pXq8dW65pB8iecP0h8mbr6xflR1TlR20QDvB7nzm59rj1ZtvbvqOyOQo5XHvT-8JBK7FSfBi7GQgSc4zZG7UBUcSnBR6l6nxKbuPtceesAm1PGy6xV2wVpIHGfn6dIHJ4uEwCRVqEumR2qZonmZzmSgROCgooVOnkw"}', '{"created_by": "CLI tool"}', datetime.datetime(2020, 3, 31, 8, 49, 54, 471685), datetime.datetime(2020, 3, 31, 11, 49, 39, 158403), None, 1, 1]) 2020-03-31 09:49:39,180 [165] [DEBUG] [__main__] Finished automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 with new expiration 2020-03-31 11:49:39.158403 servicekey stdout | 2020-03-31 09:49:39,180 [165] [DEBUG] [__main__] Finished automatic refresh of service key 29afbd28640625bb232bed790f9f98e033c2bdda20cd3eb537f33145f24c39c4 with new expiration 2020-03-31 11:49:39.158403 2020-03-31 09:49:39,181 [165] [DEBUG] [data.database] Disconnecting from database. servicekey stdout | 2020-03-31 09:49:39,181 [165] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:39,183 [165] [INFO] [apscheduler.executors.default] Job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 10:44:39 UTC)" executed successfully servicekey stdout | 2020-03-31 09:49:39,183 [165] [INFO] [apscheduler.executors.default] Job "_refresh_service_key (trigger: interval[0:55:00], next run at: 2020-03-31 10:44:39 UTC)" executed successfully 2020-03-31 09:49:39,184 [165] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 servicekey stdout | 2020-03-31 09:49:39,184 [165] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:39,186 [165] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 servicekey stdout | 2020-03-31 09:49:39,186 [165] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:39,186 [165] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics servicekey stdout | 2020-03-31 09:49:39,186 [165] [DEBUG] [util.metrics.prometheus] Sent 1 prometheus metrics 2020-03-31 09:49:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:49:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:41 UTC)" (scheduled at 2020-03-31 09:49:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:49:41,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:41 UTC)" (scheduled at 2020-03-31 09:49:41.440707+00:00) 2020-03-31 09:49:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:51.440707+00:00 (in 9.999127 seconds) repositoryactioncounter stdout | 2020-03-31 09:49:41,443 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:51.440707+00:00 (in 9.999127 seconds) 2020-03-31 09:49:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:49:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:49:41,468 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:49:41,468 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:49:41,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:41,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:49:41,468 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:41,468 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:51 UTC)" executed successfully 2020-03-31 09:49:43,138 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:f273a42a-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:43 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 storagereplication stdout | 2020-03-31 09:49:43,138 [191] [DEBUG] [azure.storage.common._auth] String_to_sign=HEAD x-ms-client-request-id:f273a42a-7334-11ea-a15c-0a580a830015 x-ms-date:Tue, 31 Mar 2020 09:49:43 GMT x-ms-version:2017-07-29 /ukquaybackendbmwuskq7s7/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97 timeout:20 2020-03-31 09:49:43,138 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f273a42a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:loN+vrzUTyh1L2QCUR7NP1VMFjm6nf945mi9Vu6F1mQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:43 GMT'}. storagereplication stdout | 2020-03-31 09:49:43,138 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Outgoing request: Method=HEAD, Path=/quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97, Query={'snapshot': None, 'timeout': '20'}, Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f273a42a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:loN+vrzUTyh1L2QCUR7NP1VMFjm6nf945mi9Vu6F1mQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:43 GMT'}. 2020-03-31 09:49:43,219 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 storagereplication stdout | 2020-03-31 09:49:43,219 [191] [DEBUG] [urllib3.connectionpool] https://ukquaybackendbmwuskq7s7.blob.core.windows.net:443 "HEAD /quayregistry/datastorage/registry/sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97?timeout=20 HTTP/1.1" 404 0 2020-03-31 09:49:43,220 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:43 GMT, Server-Request-ID=47791ad8-501e-0041-0141-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f273a42a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:loN+vrzUTyh1L2QCUR7NP1VMFjm6nf945mi9Vu6F1mQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:43 GMT'}. 2020-03-31 09:49:43,220 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:43 GMT, Server-Request-ID=47791ad8-501e-0041-0141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. storagereplication stdout | 2020-03-31 09:49:43,220 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Receiving Response: Server-Timestamp=Tue, 31 Mar 2020 09:49:43 GMT, Server-Request-ID=47791ad8-501e-0041-0141-078056000000, HTTP Status Code=404, Message=The specified blob does not exist., Headers={'x-ms-version': '2017-07-29', 'x-ms-client-request-id': 'f273a42a-7334-11ea-a15c-0a580a830015', 'x-ms-lease-id': None, 'If-Match': None, 'If-Modified-Since': None, 'If-None-Match': None, 'User-Agent': 'Azure-Storage/1.1.0-1.1.0 (Python CPython 2.7.17; Linux 4.18.0-147.5.1.el8_1.x86_64)', 'If-Unmodified-Since': None, 'Authorization': u'SharedKey ukquaybackendbmwuskq7s7:loN+vrzUTyh1L2QCUR7NP1VMFjm6nf945mi9Vu6F1mQ=', 'x-ms-date': 'Tue, 31 Mar 2020 09:49:43 GMT'}. 2020-03-31 09:49:43,220 [191] [INFO] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Tue, 31 Mar 2020 09:49:43 GMT, Server-Request-ID=47791ad8-501e-0041-0141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:43,220 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:43 GMT, Server-Request-ID=47791ad8-501e-0041-0141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:43,221 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds storagereplication stdout | 2020-03-31 09:49:43,220 [191] [ERROR] [azure.storage.common.storageclient] Client-Request-ID=f273a42a-7334-11ea-a15c-0a580a830015 Retry policy did not allow for a retry: Server-Timestamp=Tue, 31 Mar 2020 09:49:43 GMT, Server-Request-ID=47791ad8-501e-0041-0141-078056000000, HTTP status code=404, Exception=The specified blob does not exist.ErrorCode: BlobNotFound. 2020-03-31 09:49:43,221 [191] [DEBUG] [__main__] Cannot find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location azure-uk (try #3). Sleeping for 16 seconds 2020-03-31 09:49:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:49:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:14.539408+00:00 (in 30.998554 seconds) 2020-03-31 09:49:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:43 UTC)" (scheduled at 2020-03-31 09:49:43.539999+00:00) 2020-03-31 09:49:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:49:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:14.539408+00:00 (in 30.998554 seconds) 2020-03-31 09:49:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:43 UTC)" (scheduled at 2020-03-31 09:49:43.539999+00:00) 2020-03-31 09:49:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:43 UTC)" executed successfully 2020-03-31 09:49:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:49:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:49.498966+00:00 (in 5.174530 seconds) 2020-03-31 09:49:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:54 UTC)" (scheduled at 2020-03-31 09:49:44.323352+00:00) 2020-03-31 09:49:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:49:44,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:49.498966+00:00 (in 5.174530 seconds) 2020-03-31 09:49:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:54 UTC)" (scheduled at 2020-03-31 09:49:44.323352+00:00) 2020-03-31 09:49:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 44, 325687), True, datetime.datetime(2020, 3, 31, 9, 49, 44, 325687), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:49:44,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 44, 325687), True, datetime.datetime(2020, 3, 31, 9, 49, 44, 325687), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:49:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:49:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:49:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:44,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:54 UTC)" executed successfully 2020-03-31 09:49:44,446 [527] [DEBUG] [app] Starting request: urn:request:a7fc64ed-ef2d-402a-9505-0d97b0c110cc (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:44,446 [527] [DEBUG] [app] Starting request: urn:request:a7fc64ed-ef2d-402a-9505-0d97b0c110cc (/health/instance) 2020-03-31 09:49:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:44,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:44,453 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:44,454 [551] [DEBUG] [app] Starting request: urn:request:fdfd65a9-4d68-46cd-b2b1-4b207c3b543e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:44,454 [551] [DEBUG] [app] Starting request: urn:request:fdfd65a9-4d68-46cd-b2b1-4b207c3b543e (/v1/_internal_ping) 2020-03-31 09:49:44,455 [551] [DEBUG] [app] Ending request: urn:request:fdfd65a9-4d68-46cd-b2b1-4b207c3b543e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:44,455 [551] [DEBUG] [app] Ending request: urn:request:fdfd65a9-4d68-46cd-b2b1-4b207c3b543e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:49:44,456 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:44,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:44,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:49:44,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:49:44,458 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:44,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:44,460 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:44,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:44,461 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:44,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:44,471 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:44,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:44,484 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:44,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:44,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:44,489 [527] [DEBUG] [app] Starting request: urn:request:0474b32c-2e31-4957-b193-82b9a7896a60 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:44,489 [527] [DEBUG] [app] Starting request: urn:request:0474b32c-2e31-4957-b193-82b9a7896a60 (/_internal_ping) 2020-03-31 09:49:44,490 [527] [DEBUG] [app] Ending request: urn:request:0474b32c-2e31-4957-b193-82b9a7896a60 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:44,490 [527] [DEBUG] [app] Ending request: urn:request:0474b32c-2e31-4957-b193-82b9a7896a60 (/_internal_ping) 2020-03-31 09:49:44,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:44,490 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:44,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:44,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:44,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:44,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:44,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:44,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:49:44,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:49:44,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:44,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:44,502 [543] [DEBUG] [app] Starting request: urn:request:a469a41e-1e74-4ac2-8a02-4f47ab3f6fa5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:44,502 [543] [DEBUG] [app] Starting request: urn:request:a469a41e-1e74-4ac2-8a02-4f47ab3f6fa5 (/c1/_internal_ping) 2020-03-31 09:49:44,503 [543] [DEBUG] [app] Ending request: urn:request:a469a41e-1e74-4ac2-8a02-4f47ab3f6fa5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:44,503 [543] [DEBUG] [app] Ending request: urn:request:a469a41e-1e74-4ac2-8a02-4f47ab3f6fa5 (/c1/_internal_ping) 2020-03-31 09:49:44,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:44,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:44,504 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:44,505 [527] [DEBUG] [app] Ending request: urn:request:a7fc64ed-ef2d-402a-9505-0d97b0c110cc (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:49:44,504 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:44,505 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:44,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:49:44,505 [527] [DEBUG] [app] Ending request: urn:request:a7fc64ed-ef2d-402a-9505-0d97b0c110cc (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:44,505 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:44,506 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:49:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:49:45,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:45 UTC)" (scheduled at 2020-03-31 09:49:45.064514+00:00) gcworker stdout | 2020-03-31 09:49:45,068 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:49:45 UTC)" (scheduled at 2020-03-31 09:49:45.064514+00:00) 2020-03-31 09:49:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438585, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:49:45,071 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438585, 1209600, 500, 1, 0]) 2020-03-31 09:49:45,072 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:15.064514+00:00 (in 29.995900 seconds) gcworker stdout | 2020-03-31 09:49:45,072 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:15.064514+00:00 (in 29.995900 seconds) 2020-03-31 09:49:45,088 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:49:45,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:49:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:49:45,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:45,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:15 UTC)" executed successfully 2020-03-31 09:49:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:49:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:56.924497+00:00 (in 9.998994 seconds) storagereplication stdout | 2020-03-31 09:49:46,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:46 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:46,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:56.924497+00:00 (in 9.998994 seconds) 2020-03-31 09:49:48,273 [518] [DEBUG] [app] Starting request: urn:request:d1c27209-5520-47d4-8e3f-7f831a8a9b5d (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:48,273 [518] [DEBUG] [app] Starting request: urn:request:d1c27209-5520-47d4-8e3f-7f831a8a9b5d (/health/instance) 2020-03-31 09:49:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:48,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:48,279 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:48,281 [544] [DEBUG] [app] Starting request: urn:request:8b50e333-ea3e-4715-af83-e176612d1caf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:48,281 [544] [DEBUG] [app] Starting request: urn:request:8b50e333-ea3e-4715-af83-e176612d1caf (/v1/_internal_ping) 2020-03-31 09:49:48,282 [544] [DEBUG] [app] Ending request: urn:request:8b50e333-ea3e-4715-af83-e176612d1caf (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:48,282 [544] [DEBUG] [app] Ending request: urn:request:8b50e333-ea3e-4715-af83-e176612d1caf (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:48,283 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:49:48,283 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:48,283 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:49:48,284 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:49:48,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:49:48,285 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:48,286 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:48,286 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:48,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:48,287 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:48,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:48,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:48,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:48,309 [523] [DEBUG] [app] Starting request: urn:request:2e230614-932c-4507-8245-f8743c8a8b21 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:48,309 [523] [DEBUG] [app] Starting request: urn:request:2e230614-932c-4507-8245-f8743c8a8b21 (/_internal_ping) 2020-03-31 09:49:48,310 [523] [DEBUG] [app] Ending request: urn:request:2e230614-932c-4507-8245-f8743c8a8b21 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:48,310 [523] [DEBUG] [app] Ending request: urn:request:2e230614-932c-4507-8245-f8743c8a8b21 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:49:48,311 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:48,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:49:48,311 [523] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:48,311 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:48,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:48,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:48,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:49:48,319 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:49:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:48,324 [543] [DEBUG] [app] Starting request: urn:request:2f68fa40-f222-43d9-9bac-6baf733d2769 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:48,324 [543] [DEBUG] [app] Starting request: urn:request:2f68fa40-f222-43d9-9bac-6baf733d2769 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:48,325 [543] [DEBUG] [app] Ending request: urn:request:2f68fa40-f222-43d9-9bac-6baf733d2769 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:48,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:48,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:49:48,325 [543] [DEBUG] [app] Ending request: urn:request:2f68fa40-f222-43d9-9bac-6baf733d2769 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:48,325 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:48,327 [518] [DEBUG] [app] Ending request: urn:request:d1c27209-5520-47d4-8e3f-7f831a8a9b5d (/health/instance) 2020-03-31 09:49:48,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:49:48,325 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:48,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) gunicorn-web stdout | 2020-03-31 09:49:48,327 [518] [DEBUG] [app] Ending request: urn:request:d1c27209-5520-47d4-8e3f-7f831a8a9b5d (/health/instance) 2020-03-31 09:49:48,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:49:48,327 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.055 49 0.055) 2020-03-31 09:49:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:49:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:54.323352+00:00 (in 4.822567 seconds) 2020-03-31 09:49:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:49 UTC)" (scheduled at 2020-03-31 09:49:49.498966+00:00) notificationworker stdout | 2020-03-31 09:49:49,501 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:54.323352+00:00 (in 4.822567 seconds) 2020-03-31 09:49:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:49 UTC)" (scheduled at 2020-03-31 09:49:49.498966+00:00) 2020-03-31 09:49:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:49:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:49 UTC)" executed successfully 2020-03-31 09:49:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:49:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:53.556282+00:00 (in 2.444877 seconds) 2020-03-31 09:49:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:51 UTC)" (scheduled at 2020-03-31 09:49:51.110610+00:00) 2020-03-31 09:49:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:49:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:53.556282+00:00 (in 2.444877 seconds) 2020-03-31 09:49:51,111 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:51 UTC)" (scheduled at 2020-03-31 09:49:51.110610+00:00) 2020-03-31 09:49:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 51, 112245), True, datetime.datetime(2020, 3, 31, 9, 49, 51, 112245), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:49:51,115 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 51, 112245), True, datetime.datetime(2020, 3, 31, 9, 49, 51, 112245), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:49:51,129 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:49:51,129 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:49:51,129 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:51,129 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:51 UTC)" executed successfully 2020-03-31 09:49:51,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:49:51,443 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:01.440707+00:00 (in 9.996853 seconds) 2020-03-31 09:49:51,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:01 UTC)" (scheduled at 2020-03-31 09:49:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:49:51,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:01.440707+00:00 (in 9.996853 seconds) 2020-03-31 09:49:51,444 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:01 UTC)" (scheduled at 2020-03-31 09:49:51.440707+00:00) 2020-03-31 09:49:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:49:51,446 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:49:51,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:49:51,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:49:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:49:51,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:51,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:01 UTC)" executed successfully 2020-03-31 09:49:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:49:52,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:22 UTC)" (scheduled at 2020-03-31 09:49:52.113221+00:00) securityworker stdout | 2020-03-31 09:49:52,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:22 UTC)" (scheduled at 2020-03-31 09:49:52.113221+00:00) 2020-03-31 09:49:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:49:52,117 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:49:52,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:22.113221+00:00 (in 29.997639 seconds) securityworker stdout | 2020-03-31 09:49:52,117 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:22.113221+00:00 (in 29.997639 seconds) 2020-03-31 09:49:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:49:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:49:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:49:52,133 [187] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2020-03-31 09:49:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:49:52,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:49:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:22 UTC)" executed successfully 2020-03-31 09:49:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:49:53,556 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:53 UTC)" (scheduled at 2020-03-31 09:49:53.556282+00:00) namespacegcworker stdout | 2020-03-31 09:49:53,557 [192] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:53 UTC)" (scheduled at 2020-03-31 09:49:53.556282+00:00) 2020-03-31 09:49:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:59.560326+00:00 (in 6.003294 seconds) namespacegcworker stdout | 2020-03-31 09:49:53,557 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:49:59.560326+00:00 (in 6.003294 seconds) 2020-03-31 09:49:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:49:53,559 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), u'namespacegc/%']) 2020-03-31 09:49:53,573 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), True, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), 0, u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:49:53,573 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), True, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), 0, u'namespacegc/%']) 2020-03-31 09:49:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), True, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), u'namespacegc/%']) namespacegcworker stdout | 2020-03-31 09:49:53,578 [192] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), True, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), 0, u'namespacegc/%', False, datetime.datetime(2020, 3, 31, 9, 49, 53, 557834), u'namespacegc/%']) 2020-03-31 09:49:53,581 [192] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2020-03-31 09:49:53,581 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:53,581 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:53 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:49:53,581 [192] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:54:53 UTC)" executed successfully 2020-03-31 09:49:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 namespacegcworker stdout | 2020-03-31 09:49:53,583 [192] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:53,584 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 namespacegcworker stdout | 2020-03-31 09:49:53,584 [192] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:53,585 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics namespacegcworker stdout | 2020-03-31 09:49:53,585 [192] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:49:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:49:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:04 UTC)" (scheduled at 2020-03-31 09:49:54.323352+00:00) 2020-03-31 09:49:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:04.323352+00:00 (in 9.998488 seconds) 2020-03-31 09:49:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:49:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:04 UTC)" (scheduled at 2020-03-31 09:49:54.323352+00:00) 2020-03-31 09:49:54,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:04.323352+00:00 (in 9.998488 seconds) 2020-03-31 09:49:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:49:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 54, 325877), True, datetime.datetime(2020, 3, 31, 9, 49, 54, 325877), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:49:54,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 54, 325877), True, datetime.datetime(2020, 3, 31, 9, 49, 54, 325877), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:49:54,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:49:54,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:49:54,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:54,341 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:04 UTC)" executed successfully 2020-03-31 09:49:54,446 [518] [DEBUG] [app] Starting request: urn:request:6beecc47-1b6b-49f1-8b9d-221386336139 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:54,446 [518] [DEBUG] [app] Starting request: urn:request:6beecc47-1b6b-49f1-8b9d-221386336139 (/health/instance) 2020-03-31 09:49:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:54,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:54,451 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:54,453 [542] [DEBUG] [app] Starting request: urn:request:335d2363-466c-4eac-a162-87d2d10ff030 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:54,453 [542] [DEBUG] [app] Starting request: urn:request:335d2363-466c-4eac-a162-87d2d10ff030 (/v1/_internal_ping) 2020-03-31 09:49:54,454 [542] [DEBUG] [app] Ending request: urn:request:335d2363-466c-4eac-a162-87d2d10ff030 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:54,454 [542] [DEBUG] [app] Ending request: urn:request:335d2363-466c-4eac-a162-87d2d10ff030 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:54,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:49:54,455 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:54,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:49:54,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:49:54,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:49:54,457 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:54,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:54,459 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:54,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:54,459 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:54,467 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:54,479 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:54,483 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:54,484 [518] [DEBUG] [app] Starting request: urn:request:785c0db9-9556-4abb-8000-2d2b4edd5e34 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:54,484 [518] [DEBUG] [app] Starting request: urn:request:785c0db9-9556-4abb-8000-2d2b4edd5e34 (/_internal_ping) 2020-03-31 09:49:54,485 [518] [DEBUG] [app] Ending request: urn:request:785c0db9-9556-4abb-8000-2d2b4edd5e34 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:54,485 [518] [DEBUG] [app] Ending request: urn:request:785c0db9-9556-4abb-8000-2d2b4edd5e34 (/_internal_ping) 2020-03-31 09:49:54,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:54,485 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:49:54,486 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:54,489 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:54,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:54,492 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:49:54,493 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:49:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:54,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:54,498 [541] [DEBUG] [app] Starting request: urn:request:02ad75d0-cf39-4fed-b60c-25fd7e7c1f78 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:54,498 [541] [DEBUG] [app] Starting request: urn:request:02ad75d0-cf39-4fed-b60c-25fd7e7c1f78 (/c1/_internal_ping) 2020-03-31 09:49:54,499 [541] [DEBUG] [app] Ending request: urn:request:02ad75d0-cf39-4fed-b60c-25fd7e7c1f78 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:54,499 [541] [DEBUG] [app] Ending request: urn:request:02ad75d0-cf39-4fed-b60c-25fd7e7c1f78 (/c1/_internal_ping) 2020-03-31 09:49:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:54,500 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:54,500 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:54,502 [518] [DEBUG] [app] Ending request: urn:request:6beecc47-1b6b-49f1-8b9d-221386336139 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:49:54,500 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:54,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:54,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) gunicorn-web stdout | 2020-03-31 09:49:54,502 [518] [DEBUG] [app] Ending request: urn:request:6beecc47-1b6b-49f1-8b9d-221386336139 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:54,502 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:54,503 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.058 49 0.058) 2020-03-31 09:49:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:49:56,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:06.924497+00:00 (in 9.999271 seconds) storagereplication stdout | 2020-03-31 09:49:56,925 [191] [WARNING] [apscheduler.scheduler] Execution of job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:49:56 UTC)" skipped: maximum number of running instances reached (1) 2020-03-31 09:49:56,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:06.924497+00:00 (in 9.999271 seconds) 2020-03-31 09:49:58,273 [527] [DEBUG] [app] Starting request: urn:request:a5342102-40a9-43d0-94a8-2f93815720c3 (/health/instance) gunicorn-web stdout | 2020-03-31 09:49:58,273 [527] [DEBUG] [app] Starting request: urn:request:a5342102-40a9-43d0-94a8-2f93815720c3 (/health/instance) 2020-03-31 09:49:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:58,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:58,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:58,280 [547] [DEBUG] [app] Starting request: urn:request:4e7cb0a3-ba5f-407d-aab5-72f5b64a5e38 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:58,280 [547] [DEBUG] [app] Starting request: urn:request:4e7cb0a3-ba5f-407d-aab5-72f5b64a5e38 (/v1/_internal_ping) 2020-03-31 09:49:58,281 [547] [DEBUG] [app] Ending request: urn:request:4e7cb0a3-ba5f-407d-aab5-72f5b64a5e38 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:49:58,281 [547] [DEBUG] [app] Ending request: urn:request:4e7cb0a3-ba5f-407d-aab5-72f5b64a5e38 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:49:58,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:49:58,282 [547] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:49:58,282 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:49:58,283 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:49:58,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:49:58,285 [547] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:49:58,287 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:49:58,287 [547] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:49:58,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:49:58,287 [547] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:49:58,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:49:58,292 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:49:58,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:49:58,305 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:49:58,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:58,309 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:58,311 [525] [DEBUG] [app] Starting request: urn:request:282eb9a7-0088-467d-8dc8-5d905b73bbd5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:58,311 [525] [DEBUG] [app] Starting request: urn:request:282eb9a7-0088-467d-8dc8-5d905b73bbd5 (/_internal_ping) 2020-03-31 09:49:58,311 [525] [DEBUG] [app] Ending request: urn:request:282eb9a7-0088-467d-8dc8-5d905b73bbd5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:49:58,311 [525] [DEBUG] [app] Ending request: urn:request:282eb9a7-0088-467d-8dc8-5d905b73bbd5 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:49:58,312 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:58,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:49:58,312 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:58,312 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:58,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:58,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:49:58,316 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:49:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:58,318 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:58,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:49:58,319 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:49:58,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:49:58,322 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:49:58,324 [541] [DEBUG] [app] Starting request: urn:request:e3ed7d4f-2790-47a6-ab68-0e006655ef93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:58,324 [541] [DEBUG] [app] Starting request: urn:request:e3ed7d4f-2790-47a6-ab68-0e006655ef93 (/c1/_internal_ping) 2020-03-31 09:49:58,325 [541] [DEBUG] [app] Ending request: urn:request:e3ed7d4f-2790-47a6-ab68-0e006655ef93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:49:58,325 [541] [DEBUG] [app] Ending request: urn:request:e3ed7d4f-2790-47a6-ab68-0e006655ef93 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:49:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:49:58,325 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:49:58,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:49:58,325 [541] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:49:58,327 [527] [DEBUG] [app] Ending request: urn:request:a5342102-40a9-43d0-94a8-2f93815720c3 (/health/instance) 2020-03-31 09:49:58,327 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:49:58,327 [527] [DEBUG] [app] Ending request: urn:request:a5342102-40a9-43d0-94a8-2f93815720c3 (/health/instance) 2020-03-31 09:49:58,327 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:58,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:49:58,327 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:49:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:49:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:49:59,233 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy storagereplication stdout | 2020-03-31 09:49:59,233 [191] [WARNING] [__main__] Failed to find path `sha256/bf/bf8192a67ca1d7a9973948a37dec6423f86468992bfeb1461578c587f36ded97` in location `azure-uk` after copy 2020-03-31 09:49:59,233 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:49:59,233 [191] [WARNING] [workers.queueworker] Job exception: storagereplication stdout | 2020-03-31 09:49:59,233 [191] [WARNING] [workers.queueworker] An error occurred processing request: {"namespace_user_id": 1, "storage_id": "7a10c766-d963-4234-89c4-998a563408cc"} 2020-03-31 09:49:59,233 [191] [WARNING] [workers.queueworker] Job exception: 2020-03-31 09:49:59,244 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) storagereplication stdout | 2020-03-31 09:49:59,244 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s FOR UPDATE', [1, 1, 0]) 2020-03-31 09:49:59,247 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 54, 59, 243830), True, u'4c75bc59-a23a-4466-8236-9f1622bf59a1', 1]) storagereplication stdout | 2020-03-31 09:49:59,247 [191] [DEBUG] [peewee] ('UPDATE "queueitem" SET "available_after" = %s, "available" = %s, "state_id" = %s WHERE ("queueitem"."id" = %s)', [datetime.datetime(2020, 3, 31, 9, 54, 59, 243830), True, u'4c75bc59-a23a-4466-8236-9f1622bf59a1', 1]) 2020-03-31 09:49:59,254 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 59, 253085), True, datetime.datetime(2020, 3, 31, 9, 49, 59, 253085), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:49:59,254 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 49, 59, 253085), True, datetime.datetime(2020, 3, 31, 9, 49, 59, 253085), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:49:59,259 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:49:59,259 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:49:59,259 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:59,259 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:49:59,259 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:49:59,259 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:06 UTC)" executed successfully 2020-03-31 09:49:59,563 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:49:59,563 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:49:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:51.110610+00:00 (in 51.547221 seconds) 2020-03-31 09:49:59,564 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:59 UTC)" (scheduled at 2020-03-31 09:49:59.560326+00:00) 2020-03-31 09:49:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:49:59,563 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:51.110610+00:00 (in 51.547221 seconds) 2020-03-31 09:49:59,564 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:59 UTC)" (scheduled at 2020-03-31 09:49:59.560326+00:00) 2020-03-31 09:49:59,564 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:49:59,564 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:50:59 UTC)" executed successfully 2020-03-31 09:50:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:50:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:30.337699+00:00 (in 29.999172 seconds) 2020-03-31 09:50:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:30 UTC)" (scheduled at 2020-03-31 09:50:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:50:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:30.337699+00:00 (in 29.999172 seconds) 2020-03-31 09:50:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:30 UTC)" (scheduled at 2020-03-31 09:50:00.337699+00:00) 2020-03-31 09:50:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 50, 0, 339369), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:50:00,340 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 50, 0, 339369), False, 50, 1, 0]) 2020-03-31 09:50:00,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:50:00,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:50:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:50:00,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:00,356 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:30 UTC)" executed successfully 2020-03-31 09:50:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:50:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:11.440707+00:00 (in 9.999084 seconds) 2020-03-31 09:50:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:11 UTC)" (scheduled at 2020-03-31 09:50:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:50:01,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:11.440707+00:00 (in 9.999084 seconds) 2020-03-31 09:50:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:11 UTC)" (scheduled at 2020-03-31 09:50:01.440707+00:00) 2020-03-31 09:50:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:50:01,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:50:01,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:50:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:50:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:50:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:11 UTC)" executed successfully 2020-03-31 09:50:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:50:03,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:33 UTC)" (scheduled at 2020-03-31 09:50:03.330112+00:00) 2020-03-31 09:50:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:24.410927+00:00 (in 21.079871 seconds) security_notification_worker stdout | 2020-03-31 09:50:03,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:33 UTC)" (scheduled at 2020-03-31 09:50:03.330112+00:00) 2020-03-31 09:50:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:24.410927+00:00 (in 21.079871 seconds) 2020-03-31 09:50:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:50:03,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 3, 332118), True, datetime.datetime(2020, 3, 31, 9, 50, 3, 332118), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:50:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 3, 332118), True, datetime.datetime(2020, 3, 31, 9, 50, 3, 332118), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:50:03,349 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:50:03,349 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:50:03,349 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:33 UTC)" executed successfully 2020-03-31 09:50:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:50:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:14.323352+00:00 (in 9.998640 seconds) 2020-03-31 09:50:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:14 UTC)" (scheduled at 2020-03-31 09:50:04.323352+00:00) 2020-03-31 09:50:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:50:04,325 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:14.323352+00:00 (in 9.998640 seconds) 2020-03-31 09:50:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:14 UTC)" (scheduled at 2020-03-31 09:50:04.323352+00:00) 2020-03-31 09:50:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 4, 325769), True, datetime.datetime(2020, 3, 31, 9, 50, 4, 325769), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:50:04,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 4, 325769), True, datetime.datetime(2020, 3, 31, 9, 50, 4, 325769), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:50:04,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:50:04,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:50:04,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:04,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:14 UTC)" executed successfully 2020-03-31 09:50:04,445 [527] [DEBUG] [app] Starting request: urn:request:f811e988-9f84-41b8-b496-efcf6a2a9c5d (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:04,445 [527] [DEBUG] [app] Starting request: urn:request:f811e988-9f84-41b8-b496-efcf6a2a9c5d (/health/instance) 2020-03-31 09:50:04,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:04,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:04,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:04,453 [553] [DEBUG] [app] Starting request: urn:request:09fe58a7-364e-4ad9-aa30-92ab70cd693e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:04,453 [553] [DEBUG] [app] Starting request: urn:request:09fe58a7-364e-4ad9-aa30-92ab70cd693e (/v1/_internal_ping) 2020-03-31 09:50:04,454 [553] [DEBUG] [app] Ending request: urn:request:09fe58a7-364e-4ad9-aa30-92ab70cd693e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:04,454 [553] [DEBUG] [app] Ending request: urn:request:09fe58a7-364e-4ad9-aa30-92ab70cd693e (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:50:04,455 [553] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:04,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:50:04,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:50:04,457 [553] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:04,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:04,458 [553] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:04,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:04,459 [553] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:04,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:04,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:04,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:04,479 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:04,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:04,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:04,484 [527] [DEBUG] [app] Starting request: urn:request:318719c2-8134-4782-b251-0fa956725290 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:04,484 [527] [DEBUG] [app] Starting request: urn:request:318719c2-8134-4782-b251-0fa956725290 (/_internal_ping) 2020-03-31 09:50:04,485 [527] [DEBUG] [app] Ending request: urn:request:318719c2-8134-4782-b251-0fa956725290 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:04,485 [527] [DEBUG] [app] Ending request: urn:request:318719c2-8134-4782-b251-0fa956725290 (/_internal_ping) 2020-03-31 09:50:04,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:04,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:04,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:04,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:04,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:04,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:04,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:04,491 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:04,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:04,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:50:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:04,497 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:04,501 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:04,503 [535] [DEBUG] [app] Starting request: urn:request:83b5626e-86d9-4fb5-9f9a-a745d77d6776 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:04,503 [535] [DEBUG] [app] Starting request: urn:request:83b5626e-86d9-4fb5-9f9a-a745d77d6776 (/c1/_internal_ping) 2020-03-31 09:50:04,504 [535] [DEBUG] [app] Ending request: urn:request:83b5626e-86d9-4fb5-9f9a-a745d77d6776 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:04,504 [535] [DEBUG] [app] Ending request: urn:request:83b5626e-86d9-4fb5-9f9a-a745d77d6776 (/c1/_internal_ping) 2020-03-31 09:50:04,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:50:04,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:04,507 [527] [DEBUG] [app] Ending request: urn:request:f811e988-9f84-41b8-b496-efcf6a2a9c5d (/health/instance) 2020-03-31 09:50:04,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:04,507 [527] [DEBUG] [app] Ending request: urn:request:f811e988-9f84-41b8-b496-efcf6a2a9c5d (/health/instance) 2020-03-31 09:50:04,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:04,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:50:04,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 2020-03-31 09:50:04,508 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-verbs stdout | 2020-03-31 09:50:04,508 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:50:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:16.924497+00:00 (in 9.999176 seconds) 2020-03-31 09:50:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:16 UTC)" (scheduled at 2020-03-31 09:50:06.924497+00:00) 2020-03-31 09:50:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:50:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:16.924497+00:00 (in 9.999176 seconds) 2020-03-31 09:50:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:16 UTC)" (scheduled at 2020-03-31 09:50:06.924497+00:00) 2020-03-31 09:50:06,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 6, 926091), True, datetime.datetime(2020, 3, 31, 9, 50, 6, 926091), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:50:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 6, 926091), True, datetime.datetime(2020, 3, 31, 9, 50, 6, 926091), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:50:06,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:50:06,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:50:06,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:06,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:16 UTC)" executed successfully 2020-03-31 09:50:08,272 [518] [DEBUG] [app] Starting request: urn:request:5bd29134-6a65-48ee-b08b-4b4e2f3b668a (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:08,272 [518] [DEBUG] [app] Starting request: urn:request:5bd29134-6a65-48ee-b08b-4b4e2f3b668a (/health/instance) 2020-03-31 09:50:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:08,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:08,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:08,280 [552] [DEBUG] [app] Starting request: urn:request:022a8664-bd8d-4421-b52b-4276c408f0b4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:08,280 [552] [DEBUG] [app] Starting request: urn:request:022a8664-bd8d-4421-b52b-4276c408f0b4 (/v1/_internal_ping) 2020-03-31 09:50:08,281 [552] [DEBUG] [app] Ending request: urn:request:022a8664-bd8d-4421-b52b-4276c408f0b4 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:08,281 [552] [DEBUG] [app] Ending request: urn:request:022a8664-bd8d-4421-b52b-4276c408f0b4 (/v1/_internal_ping) 2020-03-31 09:50:08,281 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:50:08,281 [552] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:08,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:08,283 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:50:08,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:50:08,283 [552] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:08,285 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:08,285 [552] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:08,285 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:08,285 [552] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:08,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:08,308 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:08,312 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:08,313 [518] [DEBUG] [app] Starting request: urn:request:2af128c2-5860-49f6-b9fc-0d45f88a7c01 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:08,313 [518] [DEBUG] [app] Starting request: urn:request:2af128c2-5860-49f6-b9fc-0d45f88a7c01 (/_internal_ping) 2020-03-31 09:50:08,314 [518] [DEBUG] [app] Ending request: urn:request:2af128c2-5860-49f6-b9fc-0d45f88a7c01 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:08,314 [518] [DEBUG] [app] Ending request: urn:request:2af128c2-5860-49f6-b9fc-0d45f88a7c01 (/_internal_ping) 2020-03-31 09:50:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:08,315 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:08,315 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:08,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:08,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:08,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:08,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:08,337 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:08,337 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:08,338 [543] [DEBUG] [app] Starting request: urn:request:827126d0-b6c2-4117-bd39-42d589972925 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:08,338 [543] [DEBUG] [app] Starting request: urn:request:827126d0-b6c2-4117-bd39-42d589972925 (/c1/_internal_ping) 2020-03-31 09:50:08,339 [543] [DEBUG] [app] Ending request: urn:request:827126d0-b6c2-4117-bd39-42d589972925 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:08,339 [543] [DEBUG] [app] Ending request: urn:request:827126d0-b6c2-4117-bd39-42d589972925 (/c1/_internal_ping) 2020-03-31 09:50:08,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:08,340 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:08,340 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:08,341 [518] [DEBUG] [app] Ending request: urn:request:5bd29134-6a65-48ee-b08b-4b4e2f3b668a (/health/instance) 2020-03-31 09:50:08,341 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:50:08,340 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:08,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) gunicorn-web stdout | 2020-03-31 09:50:08,341 [518] [DEBUG] [app] Ending request: urn:request:5bd29134-6a65-48ee-b08b-4b4e2f3b668a (/health/instance) 2020-03-31 09:50:08,341 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:08,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.071 49 0.071) 2020-03-31 09:50:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:50:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:21.440707+00:00 (in 9.999243 seconds) 2020-03-31 09:50:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:21 UTC)" (scheduled at 2020-03-31 09:50:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:50:11,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:21.440707+00:00 (in 9.999243 seconds) 2020-03-31 09:50:11,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:21 UTC)" (scheduled at 2020-03-31 09:50:11.440707+00:00) 2020-03-31 09:50:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:50:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:50:11,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:50:11,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:50:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:50:11,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:11,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:21 UTC)" executed successfully 2020-03-31 09:50:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:50:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:24.323352+00:00 (in 9.998854 seconds) 2020-03-31 09:50:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:24 UTC)" (scheduled at 2020-03-31 09:50:14.323352+00:00) 2020-03-31 09:50:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:50:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:24.323352+00:00 (in 9.998854 seconds) 2020-03-31 09:50:14,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:24 UTC)" (scheduled at 2020-03-31 09:50:14.323352+00:00) 2020-03-31 09:50:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 14, 325505), True, datetime.datetime(2020, 3, 31, 9, 50, 14, 325505), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:50:14,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 14, 325505), True, datetime.datetime(2020, 3, 31, 9, 50, 14, 325505), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:50:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:14,341 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:50:14,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:14,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:50:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:24 UTC)" executed successfully 2020-03-31 09:50:14,446 [526] [DEBUG] [app] Starting request: urn:request:db1ef072-0065-4cef-a47c-785b00d9d7ce (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:14,446 [526] [DEBUG] [app] Starting request: urn:request:db1ef072-0065-4cef-a47c-785b00d9d7ce (/health/instance) 2020-03-31 09:50:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:14,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:14,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:14,453 [544] [DEBUG] [app] Starting request: urn:request:82fc823b-f6ff-4243-9cdb-df0ee4aa9c01 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:14,453 [544] [DEBUG] [app] Starting request: urn:request:82fc823b-f6ff-4243-9cdb-df0ee4aa9c01 (/v1/_internal_ping) 2020-03-31 09:50:14,454 [544] [DEBUG] [app] Ending request: urn:request:82fc823b-f6ff-4243-9cdb-df0ee4aa9c01 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:14,454 [544] [DEBUG] [app] Ending request: urn:request:82fc823b-f6ff-4243-9cdb-df0ee4aa9c01 (/v1/_internal_ping) 2020-03-31 09:50:14,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:50:14,455 [544] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:14,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:50:14,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:14,457 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:14,457 [544] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:14,458 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:14,458 [544] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:14,459 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:14,459 [544] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:14,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:14,471 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:14,484 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:14,484 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:14,488 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:14,488 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:14,489 [526] [DEBUG] [app] Starting request: urn:request:88cea3b1-a5b6-450f-82c5-992ec2de50e1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:14,489 [526] [DEBUG] [app] Starting request: urn:request:88cea3b1-a5b6-450f-82c5-992ec2de50e1 (/_internal_ping) 2020-03-31 09:50:14,490 [526] [DEBUG] [app] Ending request: urn:request:88cea3b1-a5b6-450f-82c5-992ec2de50e1 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:14,490 [526] [DEBUG] [app] Ending request: urn:request:88cea3b1-a5b6-450f-82c5-992ec2de50e1 (/_internal_ping) 2020-03-31 09:50:14,491 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:50:14,491 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:14,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:50:14,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:14,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:14,494 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:14,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:14,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:14,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:14,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) gunicorn-web stdout | 2020-03-31 09:50:14,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:50:14,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:14,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:14,503 [539] [DEBUG] [app] Starting request: urn:request:f0ac8752-f926-4ff6-aba0-ca188c088ddf (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:14,503 [539] [DEBUG] [app] Starting request: urn:request:f0ac8752-f926-4ff6-aba0-ca188c088ddf (/c1/_internal_ping) 2020-03-31 09:50:14,504 [539] [DEBUG] [app] Ending request: urn:request:f0ac8752-f926-4ff6-aba0-ca188c088ddf (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:14,504 [539] [DEBUG] [app] Ending request: urn:request:f0ac8752-f926-4ff6-aba0-ca188c088ddf (/c1/_internal_ping) 2020-03-31 09:50:14,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:14,504 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:14,504 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:50:14,504 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:14,506 [526] [DEBUG] [app] Ending request: urn:request:db1ef072-0065-4cef-a47c-785b00d9d7ce (/health/instance) 2020-03-31 09:50:14,506 [526] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:14,506 [526] [DEBUG] [app] Ending request: urn:request:db1ef072-0065-4cef-a47c-785b00d9d7ce (/health/instance) 2020-03-31 09:50:14,506 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:14,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:50:14,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:50:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:50:14,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:55:14 UTC)" (scheduled at 2020-03-31 09:50:14.539408+00:00) exportactionlogsworker stdout | 2020-03-31 09:50:14,540 [166] [INFO] [apscheduler.executors.default] Running job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:55:14 UTC)" (scheduled at 2020-03-31 09:50:14.539408+00:00) 2020-03-31 09:50:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:33.366791+00:00 (in 18.826561 seconds) exportactionlogsworker stdout | 2020-03-31 09:50:14,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:33.366791+00:00 (in 18.826561 seconds) 2020-03-31 09:50:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:50:14,542 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), u'exportactionlogs/%']) 2020-03-31 09:50:14,554 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), True, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), 0, u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:50:14,554 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), True, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), 0, u'exportactionlogs/%']) 2020-03-31 09:50:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), True, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), u'exportactionlogs/%']) exportactionlogsworker stdout | 2020-03-31 09:50:14,559 [166] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), True, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), 0, u'exportactionlogs/%', False, datetime.datetime(2020, 3, 31, 9, 50, 14, 540866), u'exportactionlogs/%']) 2020-03-31 09:50:14,562 [166] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2020-03-31 09:50:14,562 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:14,565 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:55:14 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:50:14,565 [166] [INFO] [apscheduler.executors.default] Job "update_queue_metrics (trigger: interval[0:05:00], next run at: 2020-03-31 09:55:14 UTC)" executed successfully 2020-03-31 09:50:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 exportactionlogsworker stdout | 2020-03-31 09:50:14,566 [166] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:14,566 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 exportactionlogsworker stdout | 2020-03-31 09:50:14,566 [166] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:14,567 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics exportactionlogsworker stdout | 2020-03-31 09:50:14,567 [166] [DEBUG] [util.metrics.prometheus] Sent 3 prometheus metrics 2020-03-31 09:50:15,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:50:15,068 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:45.064514+00:00 (in 29.995801 seconds) 2020-03-31 09:50:15,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:45 UTC)" (scheduled at 2020-03-31 09:50:15.064514+00:00) gcworker stdout | 2020-03-31 09:50:15,069 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:45.064514+00:00 (in 29.995801 seconds) 2020-03-31 09:50:15,069 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:45 UTC)" (scheduled at 2020-03-31 09:50:15.064514+00:00) 2020-03-31 09:50:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438615, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:50:15,070 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438615, 1209600, 500, 1, 0]) 2020-03-31 09:50:15,088 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:50:15,088 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:50:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:50:15,088 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:15,088 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:45 UTC)" executed successfully 2020-03-31 09:50:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:26.924497+00:00 (in 9.999158 seconds) 2020-03-31 09:50:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:26 UTC)" (scheduled at 2020-03-31 09:50:16.924497+00:00) 2020-03-31 09:50:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:50:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:50:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:26.924497+00:00 (in 9.999158 seconds) 2020-03-31 09:50:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:26 UTC)" (scheduled at 2020-03-31 09:50:16.924497+00:00) 2020-03-31 09:50:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 16, 926277), True, datetime.datetime(2020, 3, 31, 9, 50, 16, 926277), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:50:16,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:50:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 16, 926277), True, datetime.datetime(2020, 3, 31, 9, 50, 16, 926277), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:50:16,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:50:16,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:16,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:16,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:50:16,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:16,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:26 UTC)" executed successfully 2020-03-31 09:50:18,273 [518] [DEBUG] [app] Starting request: urn:request:938638de-d8af-490e-b438-d7544289979b (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:18,273 [518] [DEBUG] [app] Starting request: urn:request:938638de-d8af-490e-b438-d7544289979b (/health/instance) 2020-03-31 09:50:18,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:18,276 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:18,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:18,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:18,282 [545] [DEBUG] [app] Starting request: urn:request:3c66583c-5786-421a-ab5c-cf6db2b8891e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:18,282 [545] [DEBUG] [app] Starting request: urn:request:3c66583c-5786-421a-ab5c-cf6db2b8891e (/v1/_internal_ping) 2020-03-31 09:50:18,283 [545] [DEBUG] [app] Ending request: urn:request:3c66583c-5786-421a-ab5c-cf6db2b8891e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:18,283 [545] [DEBUG] [app] Ending request: urn:request:3c66583c-5786-421a-ab5c-cf6db2b8891e (/v1/_internal_ping) 2020-03-31 09:50:18,284 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2020-03-31 09:50:18,284 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:18,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2020-03-31 09:50:18,284 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:18,285 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:18,286 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:18,286 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:18,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:18,287 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:18,288 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:18,288 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:18,295 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:18,310 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:18,313 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:18,315 [527] [DEBUG] [app] Starting request: urn:request:4169eeba-b881-48c0-9203-a66f116657ca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:18,315 [527] [DEBUG] [app] Starting request: urn:request:4169eeba-b881-48c0-9203-a66f116657ca (/_internal_ping) 2020-03-31 09:50:18,316 [527] [DEBUG] [app] Ending request: urn:request:4169eeba-b881-48c0-9203-a66f116657ca (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:18,316 [527] [DEBUG] [app] Ending request: urn:request:4169eeba-b881-48c0-9203-a66f116657ca (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:50:18,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:50:18,316 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:50:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:50:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:18,322 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:18,323 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:18,326 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:18,328 [539] [DEBUG] [app] Starting request: urn:request:d2f66d9f-c1ec-4bc8-9f73-195315daaa5e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:18,328 [539] [DEBUG] [app] Starting request: urn:request:d2f66d9f-c1ec-4bc8-9f73-195315daaa5e (/c1/_internal_ping) 2020-03-31 09:50:18,328 [539] [DEBUG] [app] Ending request: urn:request:d2f66d9f-c1ec-4bc8-9f73-195315daaa5e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:18,328 [539] [DEBUG] [app] Ending request: urn:request:d2f66d9f-c1ec-4bc8-9f73-195315daaa5e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:18,329 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:50:18,329 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:18,329 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:18,331 [518] [DEBUG] [app] Ending request: urn:request:938638de-d8af-490e-b438-d7544289979b (/health/instance) 2020-03-31 09:50:18,331 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:18,331 [518] [DEBUG] [app] Ending request: urn:request:938638de-d8af-490e-b438-d7544289979b (/health/instance) 2020-03-31 09:50:18,331 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:18,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) gunicorn-web stdout | 2020-03-31 09:50:18,332 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.060 49 0.060) 2020-03-31 09:50:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:50:21,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:31.440707+00:00 (in 9.999138 seconds) 2020-03-31 09:50:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:31 UTC)" (scheduled at 2020-03-31 09:50:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:50:21,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:31.440707+00:00 (in 9.999138 seconds) 2020-03-31 09:50:21,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:31 UTC)" (scheduled at 2020-03-31 09:50:21.440707+00:00) 2020-03-31 09:50:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:50:21,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:50:21,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:50:21,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:50:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:50:21,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:21,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:31 UTC)" executed successfully 2020-03-31 09:50:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:50:22,115 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:52.113221+00:00 (in 29.997493 seconds) 2020-03-31 09:50:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:52 UTC)" (scheduled at 2020-03-31 09:50:22.113221+00:00) securityworker stdout | 2020-03-31 09:50:22,116 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:52.113221+00:00 (in 29.997493 seconds) 2020-03-31 09:50:22,116 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:52 UTC)" (scheduled at 2020-03-31 09:50:22.113221+00:00) 2020-03-31 09:50:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:50:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:50:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:50:22,130 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:50:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:50:22,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:50:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:50:22,133 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:22,133 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:50:52 UTC)" executed successfully 2020-03-31 09:50:24,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:50:24,325 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:24,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:34.323352+00:00 (in 9.997210 seconds) 2020-03-31 09:50:24,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:34 UTC)" (scheduled at 2020-03-31 09:50:24.323352+00:00) 2020-03-31 09:50:24,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:50:24,326 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:34.323352+00:00 (in 9.997210 seconds) 2020-03-31 09:50:24,326 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:34 UTC)" (scheduled at 2020-03-31 09:50:24.323352+00:00) 2020-03-31 09:50:24,327 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 24, 327060), True, datetime.datetime(2020, 3, 31, 9, 50, 24, 327060), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:50:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 24, 327060), True, datetime.datetime(2020, 3, 31, 9, 50, 24, 327060), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:50:24,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:50:24,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:50:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:34 UTC)" executed successfully 2020-03-31 09:50:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:50:24,412 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:33.330112+00:00 (in 8.917895 seconds) 2020-03-31 09:50:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:24 UTC)" (scheduled at 2020-03-31 09:50:24.410927+00:00) 2020-03-31 09:50:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:50:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:33.330112+00:00 (in 8.917895 seconds) 2020-03-31 09:50:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:24 UTC)" (scheduled at 2020-03-31 09:50:24.410927+00:00) 2020-03-31 09:50:24,413 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:24,413 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:24 UTC)" executed successfully 2020-03-31 09:50:24,446 [527] [DEBUG] [app] Starting request: urn:request:f1f65f99-ef9c-4f83-b358-d390ea6cc920 (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:24,446 [527] [DEBUG] [app] Starting request: urn:request:f1f65f99-ef9c-4f83-b358-d390ea6cc920 (/health/instance) 2020-03-31 09:50:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:24,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:24,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:24,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:24,454 [554] [DEBUG] [app] Starting request: urn:request:cc2ea75d-5f9a-443a-af9a-c2b5410b9ed7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:24,454 [554] [DEBUG] [app] Starting request: urn:request:cc2ea75d-5f9a-443a-af9a-c2b5410b9ed7 (/v1/_internal_ping) 2020-03-31 09:50:24,455 [554] [DEBUG] [app] Ending request: urn:request:cc2ea75d-5f9a-443a-af9a-c2b5410b9ed7 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:24,455 [554] [DEBUG] [app] Ending request: urn:request:cc2ea75d-5f9a-443a-af9a-c2b5410b9ed7 (/v1/_internal_ping) 2020-03-31 09:50:24,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:50:24,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:24,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:50:24,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:24,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:24,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:24,459 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:24,459 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:24,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:24,460 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:24,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:24,461 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:24,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:24,468 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:24,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:24,480 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:24,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:24,483 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:24,485 [527] [DEBUG] [app] Starting request: urn:request:ed3c60dd-9103-47a0-b566-5b6bd24b551d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:24,485 [527] [DEBUG] [app] Starting request: urn:request:ed3c60dd-9103-47a0-b566-5b6bd24b551d (/_internal_ping) 2020-03-31 09:50:24,486 [527] [DEBUG] [app] Ending request: urn:request:ed3c60dd-9103-47a0-b566-5b6bd24b551d (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:24,486 [527] [DEBUG] [app] Ending request: urn:request:ed3c60dd-9103-47a0-b566-5b6bd24b551d (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:50:24,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:24,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:50:24,486 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:24,487 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:24,490 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:24,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:24,493 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:24,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:24,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:24,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:24,499 [543] [DEBUG] [app] Starting request: urn:request:6797390a-9424-4cc9-b975-23a5b600aa8e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:24,499 [543] [DEBUG] [app] Starting request: urn:request:6797390a-9424-4cc9-b975-23a5b600aa8e (/c1/_internal_ping) 2020-03-31 09:50:24,500 [543] [DEBUG] [app] Ending request: urn:request:6797390a-9424-4cc9-b975-23a5b600aa8e (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:24,500 [543] [DEBUG] [app] Ending request: urn:request:6797390a-9424-4cc9-b975-23a5b600aa8e (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:24,500 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:24,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:50:24,500 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:24,500 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:24,502 [527] [DEBUG] [app] Ending request: urn:request:f1f65f99-ef9c-4f83-b358-d390ea6cc920 (/health/instance) 2020-03-31 09:50:24,502 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:24,502 [527] [DEBUG] [app] Ending request: urn:request:f1f65f99-ef9c-4f83-b358-d390ea6cc920 (/health/instance) 2020-03-31 09:50:24,502 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:24,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) gunicorn-web stdout | 2020-03-31 09:50:24,502 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.057 49 0.057) 2020-03-31 09:50:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:36.924497+00:00 (in 9.998928 seconds) 2020-03-31 09:50:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:36 UTC)" (scheduled at 2020-03-31 09:50:26.924497+00:00) 2020-03-31 09:50:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 26, 926489), True, datetime.datetime(2020, 3, 31, 9, 50, 26, 926489), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:50:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:36.924497+00:00 (in 9.998928 seconds) 2020-03-31 09:50:26,926 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:36 UTC)" (scheduled at 2020-03-31 09:50:26.924497+00:00) 2020-03-31 09:50:26,926 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:26,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 26, 926489), True, datetime.datetime(2020, 3, 31, 9, 50, 26, 926489), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:50:26,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:50:26,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:50:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:26,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:36 UTC)" executed successfully 2020-03-31 09:50:28,273 [518] [DEBUG] [app] Starting request: urn:request:d0e0d8a7-2178-4122-b839-08f357a3719f (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:28,273 [518] [DEBUG] [app] Starting request: urn:request:d0e0d8a7-2178-4122-b839-08f357a3719f (/health/instance) 2020-03-31 09:50:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:28,275 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:28,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:28,281 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:28,283 [555] [DEBUG] [app] Starting request: urn:request:3df7a3d4-a5c7-4db5-8b67-6ce5e9ae21a2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:28,283 [555] [DEBUG] [app] Starting request: urn:request:3df7a3d4-a5c7-4db5-8b67-6ce5e9ae21a2 (/v1/_internal_ping) 2020-03-31 09:50:28,284 [555] [DEBUG] [app] Ending request: urn:request:3df7a3d4-a5c7-4db5-8b67-6ce5e9ae21a2 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:28,284 [555] [DEBUG] [app] Ending request: urn:request:3df7a3d4-a5c7-4db5-8b67-6ce5e9ae21a2 (/v1/_internal_ping) 2020-03-31 09:50:28,285 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:50:28,285 [555] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:28,285 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:28,286 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:50:28,285 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:28,286 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:28,288 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:28,288 [555] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:28,289 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:28,289 [555] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:28,290 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:28,290 [555] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:28,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:28,298 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:28,312 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:28,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:28,319 [518] [DEBUG] [app] Starting request: urn:request:8378187c-3580-45b4-8749-2cb338792b80 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:28,319 [518] [DEBUG] [app] Starting request: urn:request:8378187c-3580-45b4-8749-2cb338792b80 (/_internal_ping) 2020-03-31 09:50:28,320 [518] [DEBUG] [app] Ending request: urn:request:8378187c-3580-45b4-8749-2cb338792b80 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:28,320 [518] [DEBUG] [app] Ending request: urn:request:8378187c-3580-45b4-8749-2cb338792b80 (/_internal_ping) 2020-03-31 09:50:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:50:28,321 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.004) gunicorn-web stdout | 2020-03-31 09:50:28,321 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:28,325 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:50:28,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:50:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:28,329 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:28,330 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:28,335 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:28,336 [543] [DEBUG] [app] Starting request: urn:request:f02b25f5-ac3f-4bb2-a3d2-ea8d9acd5eb4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:28,336 [543] [DEBUG] [app] Starting request: urn:request:f02b25f5-ac3f-4bb2-a3d2-ea8d9acd5eb4 (/c1/_internal_ping) 2020-03-31 09:50:28,338 [543] [DEBUG] [app] Ending request: urn:request:f02b25f5-ac3f-4bb2-a3d2-ea8d9acd5eb4 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:28,338 [543] [DEBUG] [app] Ending request: urn:request:f02b25f5-ac3f-4bb2-a3d2-ea8d9acd5eb4 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:28,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:28,338 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:28,338 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:50:28,338 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:28,341 [518] [DEBUG] [app] Ending request: urn:request:d0e0d8a7-2178-4122-b839-08f357a3719f (/health/instance) 2020-03-31 09:50:28,341 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:28,341 [518] [DEBUG] [app] Ending request: urn:request:d0e0d8a7-2178-4122-b839-08f357a3719f (/health/instance) 2020-03-31 09:50:28,341 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:28,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) gunicorn-web stdout | 2020-03-31 09:50:28,342 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.070 49 0.070) 2020-03-31 09:50:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:50:30,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:00.337699+00:00 (in 29.999130 seconds) 2020-03-31 09:50:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:00 UTC)" (scheduled at 2020-03-31 09:50:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:50:30,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:00.337699+00:00 (in 29.999130 seconds) 2020-03-31 09:50:30,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:00 UTC)" (scheduled at 2020-03-31 09:50:30.337699+00:00) 2020-03-31 09:50:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 50, 30, 339469), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:50:30,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 50, 30, 339469), False, 50, 1, 0]) 2020-03-31 09:50:30,356 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:50:30,356 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:50:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:50:30,356 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:30,357 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:00 UTC)" executed successfully 2020-03-31 09:50:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:50:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:41.440707+00:00 (in 9.999181 seconds) 2020-03-31 09:50:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:41 UTC)" (scheduled at 2020-03-31 09:50:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:50:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:41.440707+00:00 (in 9.999181 seconds) 2020-03-31 09:50:31,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:41 UTC)" (scheduled at 2020-03-31 09:50:31.440707+00:00) 2020-03-31 09:50:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:50:31,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:50:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:50:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:50:31,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:50:31,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:31,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:41 UTC)" executed successfully 2020-03-31 09:50:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:50:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:03.330112+00:00 (in 29.999214 seconds) 2020-03-31 09:50:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:03 UTC)" (scheduled at 2020-03-31 09:50:33.330112+00:00) 2020-03-31 09:50:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:50:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:03.330112+00:00 (in 29.999214 seconds) 2020-03-31 09:50:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:03 UTC)" (scheduled at 2020-03-31 09:50:33.330112+00:00) 2020-03-31 09:50:33,331 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 33, 331920), True, datetime.datetime(2020, 3, 31, 9, 50, 33, 331920), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:50:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 33, 331920), True, datetime.datetime(2020, 3, 31, 9, 50, 33, 331920), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:50:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:50:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:50:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:03 UTC)" executed successfully 2020-03-31 09:50:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:50:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:43.539999+00:00 (in 10.171640 seconds) 2020-03-31 09:50:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:33 UTC)" (scheduled at 2020-03-31 09:50:33.366791+00:00) exportactionlogsworker stdout | 2020-03-31 09:50:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:43.539999+00:00 (in 10.171640 seconds) 2020-03-31 09:50:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:33 UTC)" (scheduled at 2020-03-31 09:50:33.366791+00:00) 2020-03-31 09:50:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:50:33,369 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 33, 369185), True, datetime.datetime(2020, 3, 31, 9, 50, 33, 369185), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:50:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 33, 369185), True, datetime.datetime(2020, 3, 31, 9, 50, 33, 369185), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:50:33,383 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:50:33,383 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:50:33,383 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:33,383 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:33 UTC)" executed successfully 2020-03-31 09:50:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:50:34,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:44.323352+00:00 (in 9.999037 seconds) 2020-03-31 09:50:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:44 UTC)" (scheduled at 2020-03-31 09:50:34.323352+00:00) 2020-03-31 09:50:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:50:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:44.323352+00:00 (in 9.999037 seconds) 2020-03-31 09:50:34,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:44 UTC)" (scheduled at 2020-03-31 09:50:34.323352+00:00) 2020-03-31 09:50:34,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 34, 325554), True, datetime.datetime(2020, 3, 31, 9, 50, 34, 325554), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:50:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 34, 325554), True, datetime.datetime(2020, 3, 31, 9, 50, 34, 325554), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:50:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:34,341 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:50:34,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:34,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:50:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:44 UTC)" executed successfully 2020-03-31 09:50:34,446 [527] [DEBUG] [app] Starting request: urn:request:066768d1-5b7b-475b-8a82-5e8bd6af65bb (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:34,446 [527] [DEBUG] [app] Starting request: urn:request:066768d1-5b7b-475b-8a82-5e8bd6af65bb (/health/instance) 2020-03-31 09:50:34,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:34,447 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:34,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:34,453 [554] [DEBUG] [app] Starting request: urn:request:260e5eb3-dbf7-489c-8e20-0386412d697a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:34,453 [554] [DEBUG] [app] Starting request: urn:request:260e5eb3-dbf7-489c-8e20-0386412d697a (/v1/_internal_ping) 2020-03-31 09:50:34,454 [554] [DEBUG] [app] Ending request: urn:request:260e5eb3-dbf7-489c-8e20-0386412d697a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:34,454 [554] [DEBUG] [app] Ending request: urn:request:260e5eb3-dbf7-489c-8e20-0386412d697a (/v1/_internal_ping) 2020-03-31 09:50:34,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:50:34,455 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:34,455 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:34,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:50:34,456 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:50:34,457 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:34,458 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:34,458 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:34,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:34,459 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:34,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:34,469 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:34,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:34,485 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:34,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:34,489 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:34,490 [525] [DEBUG] [app] Starting request: urn:request:9a555053-8ca1-49b2-af3e-6d497271591e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:34,490 [525] [DEBUG] [app] Starting request: urn:request:9a555053-8ca1-49b2-af3e-6d497271591e (/_internal_ping) 2020-03-31 09:50:34,491 [525] [DEBUG] [app] Ending request: urn:request:9a555053-8ca1-49b2-af3e-6d497271591e (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:34,491 [525] [DEBUG] [app] Ending request: urn:request:9a555053-8ca1-49b2-af3e-6d497271591e (/_internal_ping) 2020-03-31 09:50:34,491 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:34,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:50:34,491 [525] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:34,492 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:50:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:34,495 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:34,495 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:34,498 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:50:34,499 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:34,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:34,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:34,503 [532] [DEBUG] [app] Starting request: urn:request:85f31c3f-3357-4ec0-b297-cd46465d7172 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:34,503 [532] [DEBUG] [app] Starting request: urn:request:85f31c3f-3357-4ec0-b297-cd46465d7172 (/c1/_internal_ping) 2020-03-31 09:50:34,504 [532] [DEBUG] [app] Ending request: urn:request:85f31c3f-3357-4ec0-b297-cd46465d7172 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:34,504 [532] [DEBUG] [app] Ending request: urn:request:85f31c3f-3357-4ec0-b297-cd46465d7172 (/c1/_internal_ping) 2020-03-31 09:50:34,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:34,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:34,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-verbs stdout | 2020-03-31 09:50:34,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:34,507 [527] [DEBUG] [app] Ending request: urn:request:066768d1-5b7b-475b-8a82-5e8bd6af65bb (/health/instance) 2020-03-31 09:50:34,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:34,507 [527] [DEBUG] [app] Ending request: urn:request:066768d1-5b7b-475b-8a82-5e8bd6af65bb (/health/instance) 2020-03-31 09:50:34,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:34,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:50:34,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:50:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:38.010934+00:00 (in 1.085651 seconds) 2020-03-31 09:50:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:46 UTC)" (scheduled at 2020-03-31 09:50:36.924497+00:00) 2020-03-31 09:50:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:50:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:38.010934+00:00 (in 1.085651 seconds) 2020-03-31 09:50:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:46 UTC)" (scheduled at 2020-03-31 09:50:36.924497+00:00) 2020-03-31 09:50:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 36, 925967), True, datetime.datetime(2020, 3, 31, 9, 50, 36, 925967), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:50:36,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 36, 925967), True, datetime.datetime(2020, 3, 31, 9, 50, 36, 925967), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:50:36,943 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:50:36,943 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:50:36,943 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:36,943 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:46 UTC)" executed successfully 2020-03-31 09:50:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:50:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:46.924497+00:00 (in 8.912656 seconds) 2020-03-31 09:50:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:38 UTC)" (scheduled at 2020-03-31 09:50:38.010934+00:00) 2020-03-31 09:50:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:50:38,012 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:46.924497+00:00 (in 8.912656 seconds) 2020-03-31 09:50:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:38 UTC)" (scheduled at 2020-03-31 09:50:38.010934+00:00) 2020-03-31 09:50:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:38 UTC)" executed successfully 2020-03-31 09:50:38,273 [527] [DEBUG] [app] Starting request: urn:request:261bf209-7e89-48e3-a1e8-d54168e537cc (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:38,273 [527] [DEBUG] [app] Starting request: urn:request:261bf209-7e89-48e3-a1e8-d54168e537cc (/health/instance) 2020-03-31 09:50:38,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:38,276 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:38,278 [554] [DEBUG] [app] Starting request: urn:request:0ef8c71f-c751-4165-84d8-c30f3cfbab37 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:38,278 [554] [DEBUG] [app] Starting request: urn:request:0ef8c71f-c751-4165-84d8-c30f3cfbab37 (/v1/_internal_ping) 2020-03-31 09:50:38,279 [554] [DEBUG] [app] Ending request: urn:request:0ef8c71f-c751-4165-84d8-c30f3cfbab37 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:38,279 [554] [DEBUG] [app] Ending request: urn:request:0ef8c71f-c751-4165-84d8-c30f3cfbab37 (/v1/_internal_ping) 2020-03-31 09:50:38,280 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:50:38,280 [554] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:38,280 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:50:38,281 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:38,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:38,283 [554] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:38,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:38,284 [554] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:38,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:38,285 [554] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:38,291 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:38,291 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:38,306 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:38,310 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:38,311 [521] [DEBUG] [app] Starting request: urn:request:10c4ade0-4ec6-4113-ac14-359df01fde4f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:38,311 [521] [DEBUG] [app] Starting request: urn:request:10c4ade0-4ec6-4113-ac14-359df01fde4f (/_internal_ping) 2020-03-31 09:50:38,312 [521] [DEBUG] [app] Ending request: urn:request:10c4ade0-4ec6-4113-ac14-359df01fde4f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:38,312 [521] [DEBUG] [app] Ending request: urn:request:10c4ade0-4ec6-4113-ac14-359df01fde4f (/_internal_ping) 2020-03-31 09:50:38,313 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:38,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:38,313 [521] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:38,313 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:50:38,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:38,317 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:38,317 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:38,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:38,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:38,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:38,327 [543] [DEBUG] [app] Starting request: urn:request:c4b3d002-db4f-4090-a47a-7dc7332a7618 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:38,327 [543] [DEBUG] [app] Starting request: urn:request:c4b3d002-db4f-4090-a47a-7dc7332a7618 (/c1/_internal_ping) 2020-03-31 09:50:38,328 [543] [DEBUG] [app] Ending request: urn:request:c4b3d002-db4f-4090-a47a-7dc7332a7618 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:38,328 [543] [DEBUG] [app] Ending request: urn:request:c4b3d002-db4f-4090-a47a-7dc7332a7618 (/c1/_internal_ping) 2020-03-31 09:50:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:38,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:38,328 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:50:38,328 [543] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:38,330 [527] [DEBUG] [app] Ending request: urn:request:261bf209-7e89-48e3-a1e8-d54168e537cc (/health/instance) 2020-03-31 09:50:38,330 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:38,330 [527] [DEBUG] [app] Ending request: urn:request:261bf209-7e89-48e3-a1e8-d54168e537cc (/health/instance) 2020-03-31 09:50:38,330 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) gunicorn-web stdout | 2020-03-31 09:50:38,331 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.059 49 0.059) 2020-03-31 09:50:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:50:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:51.440707+00:00 (in 9.998801 seconds) 2020-03-31 09:50:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:51 UTC)" (scheduled at 2020-03-31 09:50:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:50:41,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:51.440707+00:00 (in 9.998801 seconds) 2020-03-31 09:50:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:51 UTC)" (scheduled at 2020-03-31 09:50:41.440707+00:00) 2020-03-31 09:50:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:50:41,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:50:41,461 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:50:41,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:50:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:50:41,462 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:41,462 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:51 UTC)" executed successfully 2020-03-31 09:50:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:50:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:33.366791+00:00 (in 49.826011 seconds) 2020-03-31 09:50:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:43 UTC)" (scheduled at 2020-03-31 09:50:43.539999+00:00) 2020-03-31 09:50:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:50:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:33.366791+00:00 (in 49.826011 seconds) 2020-03-31 09:50:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:43 UTC)" (scheduled at 2020-03-31 09:50:43.539999+00:00) 2020-03-31 09:50:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:43 UTC)" executed successfully 2020-03-31 09:50:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:50:44,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:49.498966+00:00 (in 5.174331 seconds) 2020-03-31 09:50:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:54 UTC)" (scheduled at 2020-03-31 09:50:44.323352+00:00) 2020-03-31 09:50:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:50:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:49.498966+00:00 (in 5.174331 seconds) 2020-03-31 09:50:44,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:54 UTC)" (scheduled at 2020-03-31 09:50:44.323352+00:00) 2020-03-31 09:50:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 44, 325715), True, datetime.datetime(2020, 3, 31, 9, 50, 44, 325715), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:50:44,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 44, 325715), True, datetime.datetime(2020, 3, 31, 9, 50, 44, 325715), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:50:44,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:50:44,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:50:44,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:54 UTC)" executed successfully 2020-03-31 09:50:44,446 [518] [DEBUG] [app] Starting request: urn:request:ee528f34-e7d4-4484-92b9-f3da5e436903 (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:44,446 [518] [DEBUG] [app] Starting request: urn:request:ee528f34-e7d4-4484-92b9-f3da5e436903 (/health/instance) 2020-03-31 09:50:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:44,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:44,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:44,453 [550] [DEBUG] [app] Starting request: urn:request:cd443539-c5ed-4c36-8b6d-9f78189d526f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:44,453 [550] [DEBUG] [app] Starting request: urn:request:cd443539-c5ed-4c36-8b6d-9f78189d526f (/v1/_internal_ping) 2020-03-31 09:50:44,454 [550] [DEBUG] [app] Ending request: urn:request:cd443539-c5ed-4c36-8b6d-9f78189d526f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:44,454 [550] [DEBUG] [app] Ending request: urn:request:cd443539-c5ed-4c36-8b6d-9f78189d526f (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:44,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:50:44,455 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:44,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:50:44,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:44,457 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:44,457 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:44,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:44,459 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:44,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:44,460 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:44,466 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:44,481 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:44,485 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:44,487 [518] [DEBUG] [app] Starting request: urn:request:b339c218-b4e0-47a7-8b8a-30c883384df9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:44,487 [518] [DEBUG] [app] Starting request: urn:request:b339c218-b4e0-47a7-8b8a-30c883384df9 (/_internal_ping) 2020-03-31 09:50:44,488 [518] [DEBUG] [app] Ending request: urn:request:b339c218-b4e0-47a7-8b8a-30c883384df9 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:44,488 [518] [DEBUG] [app] Ending request: urn:request:b339c218-b4e0-47a7-8b8a-30c883384df9 (/_internal_ping) 2020-03-31 09:50:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:44,488 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:44,489 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:44,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:44,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:44,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:44,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:44,502 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:44,503 [532] [DEBUG] [app] Starting request: urn:request:b40bc956-b70d-4d14-a08b-6737c9779d28 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:44,503 [532] [DEBUG] [app] Starting request: urn:request:b40bc956-b70d-4d14-a08b-6737c9779d28 (/c1/_internal_ping) 2020-03-31 09:50:44,504 [532] [DEBUG] [app] Ending request: urn:request:b40bc956-b70d-4d14-a08b-6737c9779d28 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:44,504 [532] [DEBUG] [app] Ending request: urn:request:b40bc956-b70d-4d14-a08b-6737c9779d28 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:44,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:44,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:50:44,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:44,507 [518] [DEBUG] [app] Ending request: urn:request:ee528f34-e7d4-4484-92b9-f3da5e436903 (/health/instance) 2020-03-31 09:50:44,507 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:44,507 [518] [DEBUG] [app] Ending request: urn:request:ee528f34-e7d4-4484-92b9-f3da5e436903 (/health/instance) 2020-03-31 09:50:44,507 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:50:44,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:50:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:50:45,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:15.064514+00:00 (in 29.999257 seconds) 2020-03-31 09:50:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:15 UTC)" (scheduled at 2020-03-31 09:50:45.064514+00:00) gcworker stdout | 2020-03-31 09:50:45,065 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:15.064514+00:00 (in 29.999257 seconds) 2020-03-31 09:50:45,065 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:15 UTC)" (scheduled at 2020-03-31 09:50:45.064514+00:00) 2020-03-31 09:50:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438645, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:50:45,067 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438645, 1209600, 500, 1, 0]) 2020-03-31 09:50:45,082 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:50:45,082 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:50:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:15 UTC)" executed successfully gcworker stdout | 2020-03-31 09:50:45,083 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:45,083 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:15 UTC)" executed successfully 2020-03-31 09:50:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:56 UTC)" (scheduled at 2020-03-31 09:50:46.924497+00:00) 2020-03-31 09:50:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:50:46,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:50:46,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:56 UTC)" (scheduled at 2020-03-31 09:50:46.924497+00:00) 2020-03-31 09:50:46,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 46, 925973), True, datetime.datetime(2020, 3, 31, 9, 50, 46, 925973), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:50:46,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 46, 925973), True, datetime.datetime(2020, 3, 31, 9, 50, 46, 925973), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:50:46,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:56.924497+00:00 (in 9.999156 seconds) storagereplication stdout | 2020-03-31 09:50:46,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:56.924497+00:00 (in 9.999156 seconds) 2020-03-31 09:50:46,942 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:50:46,942 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:46,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:46,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:56 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:50:46,942 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:46,942 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:50:56 UTC)" executed successfully 2020-03-31 09:50:48,273 [518] [DEBUG] [app] Starting request: urn:request:9e485586-66b1-4316-b3a5-dad600884b57 (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:48,273 [518] [DEBUG] [app] Starting request: urn:request:9e485586-66b1-4316-b3a5-dad600884b57 (/health/instance) 2020-03-31 09:50:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:48,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:48,279 [538] [DEBUG] [app] Starting request: urn:request:06d87aa8-6d8b-41af-b6e2-78af57f5df3f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:48,279 [538] [DEBUG] [app] Starting request: urn:request:06d87aa8-6d8b-41af-b6e2-78af57f5df3f (/v1/_internal_ping) 2020-03-31 09:50:48,280 [538] [DEBUG] [app] Ending request: urn:request:06d87aa8-6d8b-41af-b6e2-78af57f5df3f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:48,280 [538] [DEBUG] [app] Ending request: urn:request:06d87aa8-6d8b-41af-b6e2-78af57f5df3f (/v1/_internal_ping) 2020-03-31 09:50:48,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:50:48,281 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:50:48,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:48,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:48,284 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:48,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:48,285 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:48,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:48,286 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:48,293 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:48,306 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:48,310 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:48,311 [518] [DEBUG] [app] Starting request: urn:request:e6f84bdd-9482-4539-94d7-c39cbafa723a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:48,311 [518] [DEBUG] [app] Starting request: urn:request:e6f84bdd-9482-4539-94d7-c39cbafa723a (/_internal_ping) 2020-03-31 09:50:48,312 [518] [DEBUG] [app] Ending request: urn:request:e6f84bdd-9482-4539-94d7-c39cbafa723a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:48,312 [518] [DEBUG] [app] Ending request: urn:request:e6f84bdd-9482-4539-94d7-c39cbafa723a (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) 2020-03-31 09:50:48,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:50:48,313 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:48,313 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:48,316 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:50:48,317 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:50:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:48,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:48,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:48,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:48,324 [532] [DEBUG] [app] Starting request: urn:request:b11afe42-9e53-4fd4-a870-27d1b40be558 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:48,324 [532] [DEBUG] [app] Starting request: urn:request:b11afe42-9e53-4fd4-a870-27d1b40be558 (/c1/_internal_ping) 2020-03-31 09:50:48,325 [532] [DEBUG] [app] Ending request: urn:request:b11afe42-9e53-4fd4-a870-27d1b40be558 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:48,325 [532] [DEBUG] [app] Ending request: urn:request:b11afe42-9e53-4fd4-a870-27d1b40be558 (/c1/_internal_ping) 2020-03-31 09:50:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:48,326 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:48,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:50:48,326 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:48,327 [518] [DEBUG] [app] Ending request: urn:request:9e485586-66b1-4316-b3a5-dad600884b57 (/health/instance) 2020-03-31 09:50:48,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:48,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:50:48,327 [518] [DEBUG] [app] Ending request: urn:request:9e485586-66b1-4316-b3a5-dad600884b57 (/health/instance) 2020-03-31 09:50:48,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:48,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:48 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:48 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:54.323352+00:00 (in 4.822891 seconds) 2020-03-31 09:50:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:49 UTC)" (scheduled at 2020-03-31 09:50:49.498966+00:00) 2020-03-31 09:50:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:49 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:50:49,500 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:54.323352+00:00 (in 4.822891 seconds) 2020-03-31 09:50:49,501 [175] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:49 UTC)" (scheduled at 2020-03-31 09:50:49.498966+00:00) 2020-03-31 09:50:49,501 [175] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:49,501 [175] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:49 UTC)" executed successfully 2020-03-31 09:50:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:50:51,111 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:59.560326+00:00 (in 8.448894 seconds) 2020-03-31 09:50:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:51 UTC)" (scheduled at 2020-03-31 09:50:51.110610+00:00) 2020-03-31 09:50:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2020-03-31 09:50:51,111 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:50:59.560326+00:00 (in 8.448894 seconds) 2020-03-31 09:50:51,112 [192] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:51 UTC)" (scheduled at 2020-03-31 09:50:51.110610+00:00) 2020-03-31 09:50:51,112 [192] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 51, 112400), True, datetime.datetime(2020, 3, 31, 9, 50, 51, 112400), 0, u'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2020-03-31 09:50:51,114 [192] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 51, 112400), True, datetime.datetime(2020, 3, 31, 9, 50, 51, 112400), 0, u'namespacegc/%', 50, 1, 0]) 2020-03-31 09:50:51,132 [192] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2020-03-31 09:50:51,132 [192] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:51 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:50:51,132 [192] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:51,132 [192] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:51 UTC)" executed successfully 2020-03-31 09:50:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:50:51,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:01.440707+00:00 (in 9.999011 seconds) 2020-03-31 09:50:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:01 UTC)" (scheduled at 2020-03-31 09:50:51.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:50:51,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:01.440707+00:00 (in 9.999011 seconds) 2020-03-31 09:50:51,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:01 UTC)" (scheduled at 2020-03-31 09:50:51.440707+00:00) 2020-03-31 09:50:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:50:51,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:50:51,460 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:50:51,460 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:50:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:01 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:50:51,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:51,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:01 UTC)" executed successfully 2020-03-31 09:50:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:50:52,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:22.113221+00:00 (in 29.998784 seconds) 2020-03-31 09:50:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:22 UTC)" (scheduled at 2020-03-31 09:50:52.113221+00:00) securityworker stdout | 2020-03-31 09:50:52,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:22.113221+00:00 (in 29.998784 seconds) 2020-03-31 09:50:52,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:22 UTC)" (scheduled at 2020-03-31 09:50:52.113221+00:00) 2020-03-31 09:50:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:50:52,115 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:50:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:50:52,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:50:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:50:52,133 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:50:52,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:22 UTC)" executed successfully securityworker stdout | 2020-03-31 09:50:52,134 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:52,134 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:22 UTC)" executed successfully 2020-03-31 09:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:04.323352+00:00 (in 9.999027 seconds) 2020-03-31 09:50:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:04 UTC)" (scheduled at 2020-03-31 09:50:54.323352+00:00) 2020-03-31 09:50:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:50:54,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:04.323352+00:00 (in 9.999027 seconds) 2020-03-31 09:50:54,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:04 UTC)" (scheduled at 2020-03-31 09:50:54.323352+00:00) 2020-03-31 09:50:54,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 54, 325398), True, datetime.datetime(2020, 3, 31, 9, 50, 54, 325398), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:50:54,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 54, 325398), True, datetime.datetime(2020, 3, 31, 9, 50, 54, 325398), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:50:54,340 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:50:54,340 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:54,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:54,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:04 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:50:54,340 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:54,340 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:04 UTC)" executed successfully 2020-03-31 09:50:54,446 [527] [DEBUG] [app] Starting request: urn:request:b868ff13-62f7-445d-a2ee-99928c9da05c (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:54,446 [527] [DEBUG] [app] Starting request: urn:request:b868ff13-62f7-445d-a2ee-99928c9da05c (/health/instance) 2020-03-31 09:50:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:54,448 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:54,453 [556] [DEBUG] [app] Starting request: urn:request:de83fd19-889c-4e4c-b258-92b150a44b16 (/v1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:54,452 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-registry stdout | 2020-03-31 09:50:54,453 [556] [DEBUG] [app] Starting request: urn:request:de83fd19-889c-4e4c-b258-92b150a44b16 (/v1/_internal_ping) 2020-03-31 09:50:54,455 [556] [DEBUG] [app] Ending request: urn:request:de83fd19-889c-4e4c-b258-92b150a44b16 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:54,455 [556] [DEBUG] [app] Ending request: urn:request:de83fd19-889c-4e4c-b258-92b150a44b16 (/v1/_internal_ping) 2020-03-31 09:50:54,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:50:54,456 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:54,456 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:50:54,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:50:54,457 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:50:54,459 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:54,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:54,460 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:54,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:54,461 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:54,467 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:54,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:54,483 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:54,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:54,488 [518] [DEBUG] [app] Starting request: urn:request:3a5355f2-cdfb-4337-a836-d9cba70ee161 (/_internal_ping) 2020-03-31 09:50:54,489 [518] [DEBUG] [app] Ending request: urn:request:3a5355f2-cdfb-4337-a836-d9cba70ee161 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:54,487 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:54,488 [518] [DEBUG] [app] Starting request: urn:request:3a5355f2-cdfb-4337-a836-d9cba70ee161 (/_internal_ping) 2020-03-31 09:50:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:54,489 [518] [DEBUG] [app] Ending request: urn:request:3a5355f2-cdfb-4337-a836-d9cba70ee161 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:50:54,490 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:54,490 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:54,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:54,494 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:54,494 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:54,497 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:50:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:54,498 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:54,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:54,502 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:54,504 [535] [DEBUG] [app] Starting request: urn:request:ec38d907-a9e4-46df-a6a9-4dde9bb21d93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:54,504 [535] [DEBUG] [app] Starting request: urn:request:ec38d907-a9e4-46df-a6a9-4dde9bb21d93 (/c1/_internal_ping) 2020-03-31 09:50:54,504 [535] [DEBUG] [app] Ending request: urn:request:ec38d907-a9e4-46df-a6a9-4dde9bb21d93 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:54,504 [535] [DEBUG] [app] Ending request: urn:request:ec38d907-a9e4-46df-a6a9-4dde9bb21d93 (/c1/_internal_ping) 2020-03-31 09:50:54,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:54,505 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:54,505 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-verbs stdout | 2020-03-31 09:50:54,505 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:54,507 [527] [DEBUG] [app] Ending request: urn:request:b868ff13-62f7-445d-a2ee-99928c9da05c (/health/instance) 2020-03-31 09:50:54,507 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:54,507 [527] [DEBUG] [app] Ending request: urn:request:b868ff13-62f7-445d-a2ee-99928c9da05c (/health/instance) 2020-03-31 09:50:54,507 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) gunicorn-web stdout | 2020-03-31 09:50:54,508 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:54 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:54 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.064 49 0.064) 2020-03-31 09:50:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:06.924497+00:00 (in 9.996276 seconds) 2020-03-31 09:50:56,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:06 UTC)" (scheduled at 2020-03-31 09:50:56.924497+00:00) 2020-03-31 09:50:56,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:50:56,928 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:50:56,928 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:06.924497+00:00 (in 9.996276 seconds) 2020-03-31 09:50:56,928 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:06 UTC)" (scheduled at 2020-03-31 09:50:56.924497+00:00) 2020-03-31 09:50:56,928 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:50:56,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 56, 928916), True, datetime.datetime(2020, 3, 31, 9, 50, 56, 928916), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:50:56,930 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 50, 56, 928916), True, datetime.datetime(2020, 3, 31, 9, 50, 56, 928916), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:50:56,946 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:50:56,946 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:50:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:06 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:50:56,946 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:56,946 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:06 UTC)" executed successfully 2020-03-31 09:50:58,273 [518] [DEBUG] [app] Starting request: urn:request:9e5562fe-8f76-4a88-bf63-5aeacd277128 (/health/instance) gunicorn-web stdout | 2020-03-31 09:50:58,273 [518] [DEBUG] [app] Starting request: urn:request:9e5562fe-8f76-4a88-bf63-5aeacd277128 (/health/instance) 2020-03-31 09:50:58,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:58,274 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:58,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:58,278 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:58,280 [550] [DEBUG] [app] Starting request: urn:request:55f392fa-a19a-4b4d-a3a8-2d095aa7f05b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:58,280 [550] [DEBUG] [app] Starting request: urn:request:55f392fa-a19a-4b4d-a3a8-2d095aa7f05b (/v1/_internal_ping) 2020-03-31 09:50:58,281 [550] [DEBUG] [app] Ending request: urn:request:55f392fa-a19a-4b4d-a3a8-2d095aa7f05b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:50:58,281 [550] [DEBUG] [app] Ending request: urn:request:55f392fa-a19a-4b4d-a3a8-2d095aa7f05b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:50:58,281 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:50:58,281 [550] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:50:58,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:50:58,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:50:58,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:50:58,284 [550] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:50:58,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:50:58,285 [550] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:50:58,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:50:58,286 [550] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:50:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:50:58,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:50:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:50:58,304 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:50:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:58,308 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:58,309 [518] [DEBUG] [app] Starting request: urn:request:9ab56712-b815-446b-acd0-c60e818ed672 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:58,309 [518] [DEBUG] [app] Starting request: urn:request:9ab56712-b815-446b-acd0-c60e818ed672 (/_internal_ping) 2020-03-31 09:50:58,310 [518] [DEBUG] [app] Ending request: urn:request:9ab56712-b815-446b-acd0-c60e818ed672 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:50:58,310 [518] [DEBUG] [app] Ending request: urn:request:9ab56712-b815-446b-acd0-c60e818ed672 (/_internal_ping) 2020-03-31 09:50:58,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:50:58,310 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:50:58,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:50:58,310 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:58,314 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:50:58,314 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:50:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:58,317 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:50:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:50:58,318 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:50:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:50:58,321 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:50:58,322 [532] [DEBUG] [app] Starting request: urn:request:063bd4a5-47b7-40cf-a121-adf84c58e537 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:58,322 [532] [DEBUG] [app] Starting request: urn:request:063bd4a5-47b7-40cf-a121-adf84c58e537 (/c1/_internal_ping) 2020-03-31 09:50:58,323 [532] [DEBUG] [app] Ending request: urn:request:063bd4a5-47b7-40cf-a121-adf84c58e537 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:50:58,323 [532] [DEBUG] [app] Ending request: urn:request:063bd4a5-47b7-40cf-a121-adf84c58e537 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:58,323 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:50:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:50:58,323 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:50:58,324 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:50:58,325 [518] [DEBUG] [app] Ending request: urn:request:9e5562fe-8f76-4a88-bf63-5aeacd277128 (/health/instance) 2020-03-31 09:50:58,325 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:50:58,325 [518] [DEBUG] [app] Ending request: urn:request:9e5562fe-8f76-4a88-bf63-5aeacd277128 (/health/instance) 2020-03-31 09:50:58,325 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:50:58,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) gunicorn-web stdout | 2020-03-31 09:50:58,326 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:50:58 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:50:58 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.054 49 0.054) 2020-03-31 09:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2020-03-31 09:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:51.110610+00:00 (in 51.549368 seconds) 2020-03-31 09:50:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:59 UTC)" (scheduled at 2020-03-31 09:50:59.560326+00:00) 2020-03-31 09:50:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2020-03-31 09:50:59,561 [192] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:51.110610+00:00 (in 51.549368 seconds) 2020-03-31 09:50:59,561 [192] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:59 UTC)" (scheduled at 2020-03-31 09:50:59.560326+00:00) 2020-03-31 09:50:59,562 [192] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:50:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:59 UTC)" executed successfully namespacegcworker stdout | 2020-03-31 09:50:59,562 [192] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:51:59 UTC)" executed successfully 2020-03-31 09:51:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:51:00,338 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:30.337699+00:00 (in 29.999095 seconds) 2020-03-31 09:51:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:30 UTC)" (scheduled at 2020-03-31 09:51:00.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:51:00,338 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:30.337699+00:00 (in 29.999095 seconds) 2020-03-31 09:51:00,339 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:30 UTC)" (scheduled at 2020-03-31 09:51:00.337699+00:00) 2020-03-31 09:51:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 51, 0, 339512), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:51:00,341 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 51, 0, 339512), False, 50, 1, 0]) 2020-03-31 09:51:00,552 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:51:00,552 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:51:00,552 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:00,553 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:30 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:51:00,552 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:00,553 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:30 UTC)" executed successfully 2020-03-31 09:51:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:51:01,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:11.440707+00:00 (in 9.998868 seconds) 2020-03-31 09:51:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:11 UTC)" (scheduled at 2020-03-31 09:51:01.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:51:01,442 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:11.440707+00:00 (in 9.998868 seconds) 2020-03-31 09:51:01,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:11 UTC)" (scheduled at 2020-03-31 09:51:01.440707+00:00) 2020-03-31 09:51:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:51:01,445 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:51:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:51:01,461 [172] [DEBUG] [data.database] Disconnecting from database. repositoryactioncounter stdout | 2020-03-31 09:51:01,461 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:51:01,461 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:11 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:51:01,461 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:11 UTC)" executed successfully 2020-03-31 09:51:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:51:03,331 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:24.410927+00:00 (in 21.079606 seconds) 2020-03-31 09:51:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:33 UTC)" (scheduled at 2020-03-31 09:51:03.330112+00:00) 2020-03-31 09:51:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:51:03,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:24.410927+00:00 (in 21.079606 seconds) 2020-03-31 09:51:03,332 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:33 UTC)" (scheduled at 2020-03-31 09:51:03.330112+00:00) 2020-03-31 09:51:03,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 3, 332341), True, datetime.datetime(2020, 3, 31, 9, 51, 3, 332341), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:51:03,334 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 3, 332341), True, datetime.datetime(2020, 3, 31, 9, 51, 3, 332341), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:51:03,350 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:51:03,350 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:33 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:51:03,350 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:03,350 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:33 UTC)" executed successfully 2020-03-31 09:51:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:51:04,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:14.323352+00:00 (in 9.998787 seconds) 2020-03-31 09:51:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:14 UTC)" (scheduled at 2020-03-31 09:51:04.323352+00:00) 2020-03-31 09:51:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:51:04,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:14.323352+00:00 (in 9.998787 seconds) 2020-03-31 09:51:04,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:14 UTC)" (scheduled at 2020-03-31 09:51:04.323352+00:00) 2020-03-31 09:51:04,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 4, 325691), True, datetime.datetime(2020, 3, 31, 9, 51, 4, 325691), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:51:04,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 4, 325691), True, datetime.datetime(2020, 3, 31, 9, 51, 4, 325691), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:51:04,352 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:51:04,352 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:04,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:04,353 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:14 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:51:04,352 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:04,353 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:14 UTC)" executed successfully 2020-03-31 09:51:04,446 [526] [DEBUG] [app] Starting request: urn:request:538df827-15b9-4873-a32a-b307de095a0b (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:04,446 [526] [DEBUG] [app] Starting request: urn:request:538df827-15b9-4873-a32a-b307de095a0b (/health/instance) 2020-03-31 09:51:04,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:04,448 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:04,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:04,452 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:04,454 [538] [DEBUG] [app] Starting request: urn:request:04aefec9-df76-496a-a3c5-cf9fb66a6ab6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:04,454 [538] [DEBUG] [app] Starting request: urn:request:04aefec9-df76-496a-a3c5-cf9fb66a6ab6 (/v1/_internal_ping) 2020-03-31 09:51:04,455 [538] [DEBUG] [app] Ending request: urn:request:04aefec9-df76-496a-a3c5-cf9fb66a6ab6 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:04,455 [538] [DEBUG] [app] Ending request: urn:request:04aefec9-df76-496a-a3c5-cf9fb66a6ab6 (/v1/_internal_ping) 2020-03-31 09:51:04,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-registry stdout | 2020-03-31 09:51:04,455 [538] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:04,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:04,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2020-03-31 09:51:04,455 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:04,456 [526] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:04,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:04,457 [538] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:04,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:04,459 [538] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:04,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:04,459 [538] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:04,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:04,466 [526] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:04,484 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:04,484 [526] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:04,487 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:04,487 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:04,489 [526] [DEBUG] [app] Starting request: urn:request:e0b03e57-87ac-42d8-a875-8ccd54fde826 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:04,489 [526] [DEBUG] [app] Starting request: urn:request:e0b03e57-87ac-42d8-a875-8ccd54fde826 (/_internal_ping) 2020-03-31 09:51:04,490 [526] [DEBUG] [app] Ending request: urn:request:e0b03e57-87ac-42d8-a875-8ccd54fde826 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:04,490 [526] [DEBUG] [app] Ending request: urn:request:e0b03e57-87ac-42d8-a875-8ccd54fde826 (/_internal_ping) 2020-03-31 09:51:04,490 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:51:04,490 [526] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:04,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:51:04,491 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:04,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:04,493 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:51:04,494 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:51:04,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:04,497 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:51:04,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:04,498 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:04,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:04,501 [526] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:04,502 [539] [DEBUG] [app] Starting request: urn:request:0d6bae5b-f030-47a8-9520-56dbdfdf8a5a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:04,502 [539] [DEBUG] [app] Starting request: urn:request:0d6bae5b-f030-47a8-9520-56dbdfdf8a5a (/c1/_internal_ping) 2020-03-31 09:51:04,503 [539] [DEBUG] [app] Ending request: urn:request:0d6bae5b-f030-47a8-9520-56dbdfdf8a5a (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:04,503 [539] [DEBUG] [app] Ending request: urn:request:0d6bae5b-f030-47a8-9520-56dbdfdf8a5a (/c1/_internal_ping) 2020-03-31 09:51:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:04,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:51:04,503 [526] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:04,505 [526] [DEBUG] [app] Ending request: urn:request:538df827-15b9-4873-a32a-b307de095a0b (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:51:04,503 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:04,505 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:04,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) gunicorn-web stdout | 2020-03-31 09:51:04,505 [526] [DEBUG] [app] Ending request: urn:request:538df827-15b9-4873-a32a-b307de095a0b (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:04,505 [526] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:04,506 [526] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:04 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:04 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.061 49 0.061) 2020-03-31 09:51:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:51:06,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:16.924497+00:00 (in 9.999222 seconds) 2020-03-31 09:51:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:16 UTC)" (scheduled at 2020-03-31 09:51:06.924497+00:00) 2020-03-31 09:51:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:51:06,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:16.924497+00:00 (in 9.999222 seconds) 2020-03-31 09:51:06,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:16 UTC)" (scheduled at 2020-03-31 09:51:06.924497+00:00) 2020-03-31 09:51:06,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 6, 926025), True, datetime.datetime(2020, 3, 31, 9, 51, 6, 926025), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:51:06,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 6, 926025), True, datetime.datetime(2020, 3, 31, 9, 51, 6, 926025), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:51:06,952 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:51:06,952 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:06,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:06,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:16 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:51:06,953 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:06,953 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:16 UTC)" executed successfully 2020-03-31 09:51:08,273 [527] [DEBUG] [app] Starting request: urn:request:c2a6cba9-5f8d-4e20-b8c9-2cc365b7d2a0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:08,273 [527] [DEBUG] [app] Starting request: urn:request:c2a6cba9-5f8d-4e20-b8c9-2cc365b7d2a0 (/health/instance) 2020-03-31 09:51:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:08,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:08,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:08,281 [556] [DEBUG] [app] Starting request: urn:request:1985a68f-93be-4097-8a54-2890b386033a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:08,281 [556] [DEBUG] [app] Starting request: urn:request:1985a68f-93be-4097-8a54-2890b386033a (/v1/_internal_ping) 2020-03-31 09:51:08,282 [556] [DEBUG] [app] Ending request: urn:request:1985a68f-93be-4097-8a54-2890b386033a (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:08,282 [556] [DEBUG] [app] Ending request: urn:request:1985a68f-93be-4097-8a54-2890b386033a (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:51:08,282 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:51:08,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:08,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:08,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:08,285 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:08,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:08,287 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:08,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:08,287 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:08,294 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:08,349 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:08,349 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:08,353 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:08,353 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:08,355 [527] [DEBUG] [app] Starting request: urn:request:54693af1-fbdc-4286-9bd6-eb61c83ff913 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:08,355 [527] [DEBUG] [app] Starting request: urn:request:54693af1-fbdc-4286-9bd6-eb61c83ff913 (/_internal_ping) 2020-03-31 09:51:08,356 [527] [DEBUG] [app] Ending request: urn:request:54693af1-fbdc-4286-9bd6-eb61c83ff913 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:08,356 [527] [DEBUG] [app] Ending request: urn:request:54693af1-fbdc-4286-9bd6-eb61c83ff913 (/_internal_ping) 2020-03-31 09:51:08,356 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:51:08,356 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:08,357 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:51:08,357 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:08,360 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:08,360 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:08,361 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:51:08,361 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:08,363 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:08,363 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:51:08,364 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:08,364 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:08,367 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:08,367 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:08,369 [532] [DEBUG] [app] Starting request: urn:request:d8279005-06b7-499b-9191-bd5c2d368313 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:08,369 [532] [DEBUG] [app] Starting request: urn:request:d8279005-06b7-499b-9191-bd5c2d368313 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:08,370 [532] [DEBUG] [app] Ending request: urn:request:d8279005-06b7-499b-9191-bd5c2d368313 (/c1/_internal_ping) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:08,370 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:08,370 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:51:08,370 [532] [DEBUG] [app] Ending request: urn:request:d8279005-06b7-499b-9191-bd5c2d368313 (/c1/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:08,370 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:08,372 [527] [DEBUG] [app] Ending request: urn:request:c2a6cba9-5f8d-4e20-b8c9-2cc365b7d2a0 (/health/instance) 2020-03-31 09:51:08,372 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:51:08,370 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:08,373 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.100 49 0.100) gunicorn-web stdout | 2020-03-31 09:51:08,372 [527] [DEBUG] [app] Ending request: urn:request:c2a6cba9-5f8d-4e20-b8c9-2cc365b7d2a0 (/health/instance) 2020-03-31 09:51:08,372 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:08,373 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:08 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:08 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.100 49 0.100) 2020-03-31 09:51:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:51:11,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:21 UTC)" (scheduled at 2020-03-31 09:51:11.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:51:11,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:21 UTC)" (scheduled at 2020-03-31 09:51:11.440707+00:00) 2020-03-31 09:51:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:51:11,444 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:51:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:21.440707+00:00 (in 9.999145 seconds) repositoryactioncounter stdout | 2020-03-31 09:51:11,444 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:21.440707+00:00 (in 9.999145 seconds) 2020-03-31 09:51:11,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:51:11,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:51:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:11,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:21 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:51:11,459 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:11,459 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:21 UTC)" executed successfully 2020-03-31 09:51:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:51:14,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:24.323352+00:00 (in 9.998923 seconds) 2020-03-31 09:51:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:24 UTC)" (scheduled at 2020-03-31 09:51:14.323352+00:00) 2020-03-31 09:51:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:51:14,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:24.323352+00:00 (in 9.998923 seconds) 2020-03-31 09:51:14,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:24 UTC)" (scheduled at 2020-03-31 09:51:14.323352+00:00) 2020-03-31 09:51:14,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 14, 325544), True, datetime.datetime(2020, 3, 31, 9, 51, 14, 325544), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:51:14,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 14, 325544), True, datetime.datetime(2020, 3, 31, 9, 51, 14, 325544), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:51:14,342 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:51:14,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:24 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:51:14,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:14,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:24 UTC)" executed successfully 2020-03-31 09:51:14,446 [518] [DEBUG] [app] Starting request: urn:request:64999e93-439c-4a01-a3c0-bfec1df3c685 (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:14,446 [518] [DEBUG] [app] Starting request: urn:request:64999e93-439c-4a01-a3c0-bfec1df3c685 (/health/instance) 2020-03-31 09:51:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:14,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:14,452 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:14,453 [545] [DEBUG] [app] Starting request: urn:request:a3f88376-1d12-4441-ae5c-7726076f1156 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:14,453 [545] [DEBUG] [app] Starting request: urn:request:a3f88376-1d12-4441-ae5c-7726076f1156 (/v1/_internal_ping) 2020-03-31 09:51:14,454 [545] [DEBUG] [app] Ending request: urn:request:a3f88376-1d12-4441-ae5c-7726076f1156 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:14,454 [545] [DEBUG] [app] Ending request: urn:request:a3f88376-1d12-4441-ae5c-7726076f1156 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:51:14,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:51:14,455 [545] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:51:14,455 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:14,456 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:14,458 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:14,458 [545] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:14,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:14,459 [545] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:14,460 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:14,460 [545] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:14,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:14,468 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:14,483 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:14,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:14,486 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:14,488 [527] [DEBUG] [app] Starting request: urn:request:31e018cb-586a-4534-a61b-93c9b3f8e10a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:14,488 [527] [DEBUG] [app] Starting request: urn:request:31e018cb-586a-4534-a61b-93c9b3f8e10a (/_internal_ping) 2020-03-31 09:51:14,489 [527] [DEBUG] [app] Ending request: urn:request:31e018cb-586a-4534-a61b-93c9b3f8e10a (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:14,489 [527] [DEBUG] [app] Ending request: urn:request:31e018cb-586a-4534-a61b-93c9b3f8e10a (/_internal_ping) 2020-03-31 09:51:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:51:14,489 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:51:14,490 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:14,493 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:51:14,494 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:14,497 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:51:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:14,498 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:14,501 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:14,503 [532] [DEBUG] [app] Starting request: urn:request:65588b13-c82b-438d-845f-5c5a81a5686d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:14,503 [532] [DEBUG] [app] Starting request: urn:request:65588b13-c82b-438d-845f-5c5a81a5686d (/c1/_internal_ping) 2020-03-31 09:51:14,504 [532] [DEBUG] [app] Ending request: urn:request:65588b13-c82b-438d-845f-5c5a81a5686d (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:14,504 [532] [DEBUG] [app] Ending request: urn:request:65588b13-c82b-438d-845f-5c5a81a5686d (/c1/_internal_ping) 2020-03-31 09:51:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:14,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:51:14,505 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:14,506 [518] [DEBUG] [app] Ending request: urn:request:64999e93-439c-4a01-a3c0-bfec1df3c685 (/health/instance) gunicorn-verbs stdout | 2020-03-31 09:51:14,505 [532] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:14,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:14,506 [518] [DEBUG] [app] Ending request: urn:request:64999e93-439c-4a01-a3c0-bfec1df3c685 (/health/instance) 2020-03-31 09:51:14,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) gunicorn-web stdout | 2020-03-31 09:51:14,506 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:14,507 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:14 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:14 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.063 49 0.063) 2020-03-31 09:51:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2020-03-31 09:51:15,065 [174] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:45.064514+00:00 (in 29.998950 seconds) 2020-03-31 09:51:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:45 UTC)" (scheduled at 2020-03-31 09:51:15.064514+00:00) gcworker stdout | 2020-03-31 09:51:15,066 [174] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:45.064514+00:00 (in 29.998950 seconds) 2020-03-31 09:51:15,066 [174] [INFO] [apscheduler.executors.default] Running job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:45 UTC)" (scheduled at 2020-03-31 09:51:15.064514+00:00) 2020-03-31 09:51:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438675, 1209600, 500, 1, 0]) gcworker stdout | 2020-03-31 09:51:15,068 [174] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "repositorytag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((NOT ("t1"."lifetime_end_ts" IS %s) AND ("t1"."lifetime_end_ts" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1584438675, 1209600, 500, 1, 0]) 2020-03-31 09:51:15,085 [174] [DEBUG] [__main__] No repository with garbage found 2020-03-31 09:51:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:45 UTC)" executed successfully gcworker stdout | 2020-03-31 09:51:15,085 [174] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2020-03-31 09:51:15,085 [174] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:15,086 [174] [INFO] [apscheduler.executors.default] Job "_garbage_collection_repos (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:45 UTC)" executed successfully 2020-03-31 09:51:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:26.924497+00:00 (in 9.999256 seconds) 2020-03-31 09:51:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:26 UTC)" (scheduled at 2020-03-31 09:51:16.924497+00:00) 2020-03-31 09:51:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 16, 925925), True, datetime.datetime(2020, 3, 31, 9, 51, 16, 925925), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:51:16,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:16,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:26.924497+00:00 (in 9.999256 seconds) 2020-03-31 09:51:16,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:26 UTC)" (scheduled at 2020-03-31 09:51:16.924497+00:00) 2020-03-31 09:51:16,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:16,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 16, 925925), True, datetime.datetime(2020, 3, 31, 9, 51, 16, 925925), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:51:16,952 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:51:16,952 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:16,952 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:16,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:26 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:51:16,952 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:16,952 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:26 UTC)" executed successfully 2020-03-31 09:51:18,273 [518] [DEBUG] [app] Starting request: urn:request:4925a58b-418d-4523-9ce1-a512c7b43ba0 (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:18,273 [518] [DEBUG] [app] Starting request: urn:request:4925a58b-418d-4523-9ce1-a512c7b43ba0 (/health/instance) 2020-03-31 09:51:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:18,277 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:18,279 [540] [DEBUG] [app] Starting request: urn:request:f45e985f-b908-4e9e-8896-9191c2c61c91 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:18,279 [540] [DEBUG] [app] Starting request: urn:request:f45e985f-b908-4e9e-8896-9191c2c61c91 (/v1/_internal_ping) 2020-03-31 09:51:18,280 [540] [DEBUG] [app] Ending request: urn:request:f45e985f-b908-4e9e-8896-9191c2c61c91 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:18,280 [540] [DEBUG] [app] Ending request: urn:request:f45e985f-b908-4e9e-8896-9191c2c61c91 (/v1/_internal_ping) 2020-03-31 09:51:18,281 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:51:18,281 [540] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:51:18,281 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:51:18,282 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:18,283 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:18,283 [540] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:18,285 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:18,285 [540] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:18,286 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:18,286 [540] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:18,292 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:18,305 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:18,309 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:18,310 [520] [DEBUG] [app] Starting request: urn:request:46315a64-87a7-418d-bdb0-bba8a282adbe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:18,310 [520] [DEBUG] [app] Starting request: urn:request:46315a64-87a7-418d-bdb0-bba8a282adbe (/_internal_ping) 2020-03-31 09:51:18,311 [520] [DEBUG] [app] Ending request: urn:request:46315a64-87a7-418d-bdb0-bba8a282adbe (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:18,311 [520] [DEBUG] [app] Ending request: urn:request:46315a64-87a7-418d-bdb0-bba8a282adbe (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:51:18,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:18,312 [520] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:18,312 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:18,315 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:51:18,316 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:18,319 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:51:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:18,320 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:18,323 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:18,324 [539] [DEBUG] [app] Starting request: urn:request:4a959306-c8ab-4bcf-9d43-e0e1c16db411 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:18,324 [539] [DEBUG] [app] Starting request: urn:request:4a959306-c8ab-4bcf-9d43-e0e1c16db411 (/c1/_internal_ping) 2020-03-31 09:51:18,325 [539] [DEBUG] [app] Ending request: urn:request:4a959306-c8ab-4bcf-9d43-e0e1c16db411 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:18,325 [539] [DEBUG] [app] Ending request: urn:request:4a959306-c8ab-4bcf-9d43-e0e1c16db411 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:18,326 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:51:18,326 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:51:18,326 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:18,327 [518] [DEBUG] [app] Ending request: urn:request:4925a58b-418d-4523-9ce1-a512c7b43ba0 (/health/instance) 2020-03-31 09:51:18,327 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:18,327 [518] [DEBUG] [app] Ending request: urn:request:4925a58b-418d-4523-9ce1-a512c7b43ba0 (/health/instance) 2020-03-31 09:51:18,327 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) gunicorn-web stdout | 2020-03-31 09:51:18,328 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:18 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:18 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.056 49 0.056) 2020-03-31 09:51:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:51:21,444 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:31.440707+00:00 (in 9.995634 seconds) 2020-03-31 09:51:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:31 UTC)" (scheduled at 2020-03-31 09:51:21.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:51:21,445 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:31.440707+00:00 (in 9.995634 seconds) 2020-03-31 09:51:21,445 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:31 UTC)" (scheduled at 2020-03-31 09:51:21.440707+00:00) 2020-03-31 09:51:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:51:21,447 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:51:21,465 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:51:21,465 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:51:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:31 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:51:21,465 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:21,465 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:31 UTC)" executed successfully 2020-03-31 09:51:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2020-03-31 09:51:22,114 [187] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:52.113221+00:00 (in 29.998715 seconds) 2020-03-31 09:51:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:52 UTC)" (scheduled at 2020-03-31 09:51:22.113221+00:00) securityworker stdout | 2020-03-31 09:51:22,114 [187] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:52.113221+00:00 (in 29.998715 seconds) 2020-03-31 09:51:22,115 [187] [INFO] [apscheduler.executors.default] Running job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:52 UTC)" (scheduled at 2020-03-31 09:51:22.113221+00:00) 2020-03-31 09:51:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) securityworker stdout | 2020-03-31 09:51:22,116 [187] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "image" AS "t1" WHERE ("t1"."security_indexed_engine" < %s)', [3]) 2020-03-31 09:51:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) securityworker stdout | 2020-03-31 09:51:22,131 [187] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "image" AS "t1"', []) 2020-03-31 09:51:22,138 [187] [DEBUG] [workers.securityworker] Found no additional images to scan securityworker stdout | 2020-03-31 09:51:22,138 [187] [DEBUG] [workers.securityworker] Found no additional images to scan 2020-03-31 09:51:22,138 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:22,138 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:52 UTC)" executed successfully securityworker stdout | 2020-03-31 09:51:22,138 [187] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:22,138 [187] [INFO] [apscheduler.executors.default] Job "_index_images (trigger: interval[0:00:30], next run at: 2020-03-31 09:51:52 UTC)" executed successfully 2020-03-31 09:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:34.323352+00:00 (in 9.998883 seconds) 2020-03-31 09:51:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:34 UTC)" (scheduled at 2020-03-31 09:51:24.323352+00:00) 2020-03-31 09:51:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:51:24,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:34.323352+00:00 (in 9.998883 seconds) 2020-03-31 09:51:24,325 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:34 UTC)" (scheduled at 2020-03-31 09:51:24.323352+00:00) notificationworker stdout | 2020-03-31 09:51:24,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 24, 325597), True, datetime.datetime(2020, 3, 31, 9, 51, 24, 325597), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:51:24,328 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 24, 325597), True, datetime.datetime(2020, 3, 31, 9, 51, 24, 325597), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:51:24,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:51:24,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:34 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:51:24,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:24,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:34 UTC)" executed successfully 2020-03-31 09:51:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:51:24,411 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:33.330112+00:00 (in 8.918321 seconds) 2020-03-31 09:51:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:24 UTC)" (scheduled at 2020-03-31 09:51:24.410927+00:00) 2020-03-31 09:51:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:51:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:24 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:51:24,412 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:33.330112+00:00 (in 8.918321 seconds) 2020-03-31 09:51:24,412 [168] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:24 UTC)" (scheduled at 2020-03-31 09:51:24.410927+00:00) 2020-03-31 09:51:24,412 [168] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:51:24,412 [168] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:24 UTC)" executed successfully 2020-03-31 09:51:24,446 [518] [DEBUG] [app] Starting request: urn:request:9bc62dd5-5702-49b1-945f-a0e493c75028 (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:24,446 [518] [DEBUG] [app] Starting request: urn:request:9bc62dd5-5702-49b1-945f-a0e493c75028 (/health/instance) 2020-03-31 09:51:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:24,448 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:24,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:24,456 [556] [DEBUG] [app] Starting request: urn:request:f98dbaad-905a-4814-a7a7-17cfa42d66d0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:24,456 [556] [DEBUG] [app] Starting request: urn:request:f98dbaad-905a-4814-a7a7-17cfa42d66d0 (/v1/_internal_ping) 2020-03-31 09:51:24,457 [556] [DEBUG] [app] Ending request: urn:request:f98dbaad-905a-4814-a7a7-17cfa42d66d0 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:24,457 [556] [DEBUG] [app] Ending request: urn:request:f98dbaad-905a-4814-a7a7-17cfa42d66d0 (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:51:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:51:24,458 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:24,459 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:51:24,460 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:24,463 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:24,463 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:24,465 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:24,465 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:24,466 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:24,466 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:24,471 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:24,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:24,490 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:24,494 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:24,497 [518] [DEBUG] [app] Starting request: urn:request:f2b1ed0e-2c74-4ec7-a5ed-e719787ef88f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:24,497 [518] [DEBUG] [app] Starting request: urn:request:f2b1ed0e-2c74-4ec7-a5ed-e719787ef88f (/_internal_ping) 2020-03-31 09:51:24,498 [518] [DEBUG] [app] Ending request: urn:request:f2b1ed0e-2c74-4ec7-a5ed-e719787ef88f (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:24,498 [518] [DEBUG] [app] Ending request: urn:request:f2b1ed0e-2c74-4ec7-a5ed-e719787ef88f (/_internal_ping) 2020-03-31 09:51:24,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:51:24,498 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:51:24,499 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:24,503 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:51:24,504 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:51:24,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:24,507 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) gunicorn-web stdout | 2020-03-31 09:51:24,508 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.000 153 0.000) 2020-03-31 09:51:24,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:24,512 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:24,513 [539] [DEBUG] [app] Starting request: urn:request:73244d3a-1c12-43d5-84d4-80aa1cf5c8d5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:24,513 [539] [DEBUG] [app] Starting request: urn:request:73244d3a-1c12-43d5-84d4-80aa1cf5c8d5 (/c1/_internal_ping) 2020-03-31 09:51:24,514 [539] [DEBUG] [app] Ending request: urn:request:73244d3a-1c12-43d5-84d4-80aa1cf5c8d5 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:24,514 [539] [DEBUG] [app] Ending request: urn:request:73244d3a-1c12-43d5-84d4-80aa1cf5c8d5 (/c1/_internal_ping) 2020-03-31 09:51:24,515 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-verbs stdout | 2020-03-31 09:51:24,515 [539] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:24,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:51:24,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:51:24,518 [518] [DEBUG] [app] Ending request: urn:request:9bc62dd5-5702-49b1-945f-a0e493c75028 (/health/instance) 2020-03-31 09:51:24,518 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:24,518 [518] [DEBUG] [app] Ending request: urn:request:9bc62dd5-5702-49b1-945f-a0e493c75028 (/health/instance) 2020-03-31 09:51:24,518 [518] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:24,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" gunicorn-web stdout | 2020-03-31 09:51:24,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:24 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:24 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.075 49 0.075) 2020-03-31 09:51:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:51:26,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:36.924497+00:00 (in 9.999271 seconds) 2020-03-31 09:51:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:36 UTC)" (scheduled at 2020-03-31 09:51:26.924497+00:00) 2020-03-31 09:51:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:51:26,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:36.924497+00:00 (in 9.999271 seconds) 2020-03-31 09:51:26,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:36 UTC)" (scheduled at 2020-03-31 09:51:26.924497+00:00) 2020-03-31 09:51:26,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 26, 925954), True, datetime.datetime(2020, 3, 31, 9, 51, 26, 925954), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:51:26,927 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 26, 925954), True, datetime.datetime(2020, 3, 31, 9, 51, 26, 925954), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:51:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:26,947 [191] [DEBUG] [data.database] Disconnecting from database. storagereplication stdout | 2020-03-31 09:51:26,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:26,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:36 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:51:26,948 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:36 UTC)" executed successfully 2020-03-31 09:51:28,273 [527] [DEBUG] [app] Starting request: urn:request:8f6a1811-8957-449e-9601-2865438f6b98 (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:28,273 [527] [DEBUG] [app] Starting request: urn:request:8f6a1811-8957-449e-9601-2865438f6b98 (/health/instance) 2020-03-31 09:51:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:28,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:28,279 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:28,281 [542] [DEBUG] [app] Starting request: urn:request:fd3a6160-4bec-401a-98f9-d19b8300d93e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:28,281 [542] [DEBUG] [app] Starting request: urn:request:fd3a6160-4bec-401a-98f9-d19b8300d93e (/v1/_internal_ping) 2020-03-31 09:51:28,282 [542] [DEBUG] [app] Ending request: urn:request:fd3a6160-4bec-401a-98f9-d19b8300d93e (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:28,282 [542] [DEBUG] [app] Ending request: urn:request:fd3a6160-4bec-401a-98f9-d19b8300d93e (/v1/_internal_ping) 2020-03-31 09:51:28,283 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:51:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:51:28,283 [542] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:28,286 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:51:28,283 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:51:28,288 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-web stdout | 2020-03-31 09:51:28,284 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:28,289 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:28,286 [542] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:28,288 [542] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:28,289 [542] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:28,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:28,295 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:28,312 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:28,312 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:28,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:28,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:28,317 [527] [DEBUG] [app] Starting request: urn:request:b4f36a27-734c-4c91-bc58-03363f7c9a51 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:28,317 [527] [DEBUG] [app] Starting request: urn:request:b4f36a27-734c-4c91-bc58-03363f7c9a51 (/_internal_ping) 2020-03-31 09:51:28,318 [527] [DEBUG] [app] Ending request: urn:request:b4f36a27-734c-4c91-bc58-03363f7c9a51 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:28,318 [527] [DEBUG] [app] Ending request: urn:request:b4f36a27-734c-4c91-bc58-03363f7c9a51 (/_internal_ping) 2020-03-31 09:51:28,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:51:28,318 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:28,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2020-03-31 09:51:28,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:28,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:28,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 gunicorn-web stdout | 2020-03-31 09:51:28,322 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 2020-03-31 09:51:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:28,325 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:51:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:28,326 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:28,329 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:28,330 [535] [DEBUG] [app] Starting request: urn:request:8b73e92a-ad7b-4e68-a142-b34b6cb66020 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:28,330 [535] [DEBUG] [app] Starting request: urn:request:8b73e92a-ad7b-4e68-a142-b34b6cb66020 (/c1/_internal_ping) 2020-03-31 09:51:28,331 [535] [DEBUG] [app] Ending request: urn:request:8b73e92a-ad7b-4e68-a142-b34b6cb66020 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:28,331 [535] [DEBUG] [app] Ending request: urn:request:8b73e92a-ad7b-4e68-a142-b34b6cb66020 (/c1/_internal_ping) 2020-03-31 09:51:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:28,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:51:28,331 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:28,333 [527] [DEBUG] [app] Ending request: urn:request:8f6a1811-8957-449e-9601-2865438f6b98 (/health/instance) 2020-03-31 09:51:28,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:51:28,331 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:28,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) gunicorn-web stdout | 2020-03-31 09:51:28,333 [527] [DEBUG] [app] Ending request: urn:request:8f6a1811-8957-449e-9601-2865438f6b98 (/health/instance) 2020-03-31 09:51:28,333 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:28,333 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:28 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:28 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.062 49 0.062) 2020-03-31 09:51:30,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2020-03-31 09:51:30,341 [173] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:00 UTC)" (scheduled at 2020-03-31 09:51:30.337699+00:00) buildlogsarchiver stdout | 2020-03-31 09:51:30,341 [173] [INFO] [apscheduler.executors.default] Running job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:00 UTC)" (scheduled at 2020-03-31 09:51:30.337699+00:00) 2020-03-31 09:51:30,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:52:00.337699+00:00 (in 29.996170 seconds) buildlogsarchiver stdout | 2020-03-31 09:51:30,342 [173] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:52:00.337699+00:00 (in 29.996170 seconds) 2020-03-31 09:51:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 51, 30, 342230), False, 50, 1, 0]) buildlogsarchiver stdout | 2020-03-31 09:51:30,343 [173] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [u'complete', u'error', u'cancelled', datetime.datetime(2020, 3, 16, 9, 51, 30, 342230), False, 50, 1, 0]) 2020-03-31 09:51:30,362 [173] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2020-03-31 09:51:30,362 [173] [DEBUG] [__main__] No more builds to archive 2020-03-31 09:51:30,362 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:30,363 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:00 UTC)" executed successfully buildlogsarchiver stdout | 2020-03-31 09:51:30,362 [173] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:30,363 [173] [INFO] [apscheduler.executors.default] Job "_archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:00 UTC)" executed successfully 2020-03-31 09:51:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:51:31,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:41.440707+00:00 (in 9.999263 seconds) 2020-03-31 09:51:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:41 UTC)" (scheduled at 2020-03-31 09:51:31.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:51:31,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:41.440707+00:00 (in 9.999263 seconds) 2020-03-31 09:51:31,441 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:41 UTC)" (scheduled at 2020-03-31 09:51:31.440707+00:00) 2020-03-31 09:51:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:51:31,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:51:31,459 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:51:31,459 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:51:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:41 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:51:31,460 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:31,460 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:41 UTC)" executed successfully 2020-03-31 09:51:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run security_notification_worker stdout | 2020-03-31 09:51:33,330 [168] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:52:03.330112+00:00 (in 29.999139 seconds) 2020-03-31 09:51:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:03 UTC)" (scheduled at 2020-03-31 09:51:33.330112+00:00) security_notification_worker stdout | 2020-03-31 09:51:33,331 [168] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:52:03.330112+00:00 (in 29.999139 seconds) 2020-03-31 09:51:33,331 [168] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:03 UTC)" (scheduled at 2020-03-31 09:51:33.330112+00:00) 2020-03-31 09:51:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. security_notification_worker stdout | 2020-03-31 09:51:33,332 [168] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 33, 332162), True, datetime.datetime(2020, 3, 31, 9, 51, 33, 332162), 0, u'security_notification/%', 50, 1, 0]) security_notification_worker stdout | 2020-03-31 09:51:33,335 [168] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 33, 332162), True, datetime.datetime(2020, 3, 31, 9, 51, 33, 332162), 0, u'security_notification/%', 50, 1, 0]) 2020-03-31 09:51:33,351 [168] [DEBUG] [workers.queueworker] No more work. security_notification_worker stdout | 2020-03-31 09:51:33,351 [168] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:03 UTC)" executed successfully security_notification_worker stdout | 2020-03-31 09:51:33,351 [168] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:33,351 [168] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:30], next run at: 2020-03-31 09:52:03 UTC)" executed successfully 2020-03-31 09:51:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:51:33,368 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:43.539999+00:00 (in 10.171839 seconds) exportactionlogsworker stdout | 2020-03-31 09:51:33,368 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:43.539999+00:00 (in 10.171839 seconds) 2020-03-31 09:51:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:33 UTC)" (scheduled at 2020-03-31 09:51:33.366791+00:00) 2020-03-31 09:51:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2020-03-31 09:51:33,368 [166] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:33 UTC)" (scheduled at 2020-03-31 09:51:33.366791+00:00) 2020-03-31 09:51:33,368 [166] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 33, 368953), True, datetime.datetime(2020, 3, 31, 9, 51, 33, 368953), 0, u'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2020-03-31 09:51:33,370 [166] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 33, 368953), True, datetime.datetime(2020, 3, 31, 9, 51, 33, 368953), 0, u'exportactionlogs/%', 50, 1, 0]) 2020-03-31 09:51:33,384 [166] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2020-03-31 09:51:33,384 [166] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:33 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:51:33,384 [166] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:33,384 [166] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:33 UTC)" executed successfully 2020-03-31 09:51:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:51:34,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:44.323352+00:00 (in 9.999272 seconds) 2020-03-31 09:51:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:44 UTC)" (scheduled at 2020-03-31 09:51:34.323352+00:00) 2020-03-31 09:51:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:51:34,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:44.323352+00:00 (in 9.999272 seconds) 2020-03-31 09:51:34,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:44 UTC)" (scheduled at 2020-03-31 09:51:34.323352+00:00) 2020-03-31 09:51:34,324 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 34, 324854), True, datetime.datetime(2020, 3, 31, 9, 51, 34, 324854), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:51:34,327 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 34, 324854), True, datetime.datetime(2020, 3, 31, 9, 51, 34, 324854), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:51:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:34,342 [175] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2020-03-31 09:51:34,342 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:34,342 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:44 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:51:34,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:44 UTC)" executed successfully 2020-03-31 09:51:34,446 [518] [DEBUG] [app] Starting request: urn:request:6d77b2e2-e3e3-4c68-8b9c-871539763daa (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:34,446 [518] [DEBUG] [app] Starting request: urn:request:6d77b2e2-e3e3-4c68-8b9c-871539763daa (/health/instance) 2020-03-31 09:51:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:34,449 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:34,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:34,454 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:34,456 [556] [DEBUG] [app] Starting request: urn:request:0765bd7a-c0c6-424e-bb1a-890002f4138b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:34,456 [556] [DEBUG] [app] Starting request: urn:request:0765bd7a-c0c6-424e-bb1a-890002f4138b (/v1/_internal_ping) 2020-03-31 09:51:34,457 [556] [DEBUG] [app] Ending request: urn:request:0765bd7a-c0c6-424e-bb1a-890002f4138b (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:34,457 [556] [DEBUG] [app] Ending request: urn:request:0765bd7a-c0c6-424e-bb1a-890002f4138b (/v1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:51:34,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:34,458 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:34,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:51:34,459 [518] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:34,460 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2020-03-31 09:51:34,460 [556] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:34,462 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:34,462 [556] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:34,464 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:34,464 [556] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:34,465 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:34,465 [556] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:34,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:34,472 [518] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:34,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:34,494 [518] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:34,499 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:34,500 [518] [DEBUG] [app] Starting request: urn:request:eef2378d-041c-4ba4-8c9b-bd0aa44e86c5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:34,500 [518] [DEBUG] [app] Starting request: urn:request:eef2378d-041c-4ba4-8c9b-bd0aa44e86c5 (/_internal_ping) 2020-03-31 09:51:34,501 [518] [DEBUG] [app] Ending request: urn:request:eef2378d-041c-4ba4-8c9b-bd0aa44e86c5 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:34,501 [518] [DEBUG] [app] Ending request: urn:request:eef2378d-041c-4ba4-8c9b-bd0aa44e86c5 (/_internal_ping) 2020-03-31 09:51:34,502 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:51:34,502 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:34,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2020-03-31 09:51:34,502 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:34,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:34,505 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:34,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:51:34,506 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:34,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:34,508 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:51:34,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:34,509 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:34,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:34,513 [518] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:34,514 [535] [DEBUG] [app] Starting request: urn:request:1fc085be-a9d5-42b8-94d0-894057eec296 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:34,514 [535] [DEBUG] [app] Starting request: urn:request:1fc085be-a9d5-42b8-94d0-894057eec296 (/c1/_internal_ping) 2020-03-31 09:51:34,515 [535] [DEBUG] [app] Ending request: urn:request:1fc085be-a9d5-42b8-94d0-894057eec296 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:34,515 [535] [DEBUG] [app] Ending request: urn:request:1fc085be-a9d5-42b8-94d0-894057eec296 (/c1/_internal_ping) 2020-03-31 09:51:34,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:34,515 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:51:34,516 [518] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) 2020-03-31 09:51:34,517 [518] [DEBUG] [app] Ending request: urn:request:6d77b2e2-e3e3-4c68-8b9c-871539763daa (/health/instance) 2020-03-31 09:51:34,517 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-verbs stdout | 2020-03-31 09:51:34,515 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:34,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) gunicorn-web stdout | 2020-03-31 09:51:34,517 [518] [DEBUG] [app] Ending request: urn:request:6d77b2e2-e3e3-4c68-8b9c-871539763daa (/health/instance) 2020-03-31 09:51:34,517 [518] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:34,518 [518] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:34 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:34 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.072 49 0.072) 2020-03-31 09:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:38.010934+00:00 (in 1.085569 seconds) 2020-03-31 09:51:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:46 UTC)" (scheduled at 2020-03-31 09:51:36.924497+00:00) 2020-03-31 09:51:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 36, 926048), True, datetime.datetime(2020, 3, 31, 9, 51, 36, 926048), 0, u'imagestoragereplication/%', 50, 1, 0]) storagereplication stdout | 2020-03-31 09:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:51:36,925 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:38.010934+00:00 (in 1.085569 seconds) 2020-03-31 09:51:36,925 [191] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:46 UTC)" (scheduled at 2020-03-31 09:51:36.924497+00:00) 2020-03-31 09:51:36,925 [191] [DEBUG] [workers.queueworker] Getting work item from queue. storagereplication stdout | 2020-03-31 09:51:36,928 [191] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 36, 926048), True, datetime.datetime(2020, 3, 31, 9, 51, 36, 926048), 0, u'imagestoragereplication/%', 50, 1, 0]) 2020-03-31 09:51:36,947 [191] [DEBUG] [workers.queueworker] No more work. storagereplication stdout | 2020-03-31 09:51:36,947 [191] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:46 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:51:36,947 [191] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:36,947 [191] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:46 UTC)" executed successfully 2020-03-31 09:51:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run storagereplication stdout | 2020-03-31 09:51:38,011 [191] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:46.924497+00:00 (in 8.912818 seconds) 2020-03-31 09:51:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:38 UTC)" (scheduled at 2020-03-31 09:51:38.010934+00:00) 2020-03-31 09:51:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:51:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:38 UTC)" executed successfully storagereplication stdout | 2020-03-31 09:51:38,011 [191] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:46.924497+00:00 (in 8.912818 seconds) 2020-03-31 09:51:38,012 [191] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:38 UTC)" (scheduled at 2020-03-31 09:51:38.010934+00:00) 2020-03-31 09:51:38,012 [191] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:51:38,012 [191] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:38 UTC)" executed successfully 2020-03-31 09:51:38,273 [527] [DEBUG] [app] Starting request: urn:request:ea35c78a-2097-47b8-b252-b3a3e13bdab7 (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:38,273 [527] [DEBUG] [app] Starting request: urn:request:ea35c78a-2097-47b8-b252-b3a3e13bdab7 (/health/instance) 2020-03-31 09:51:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:38,275 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:38,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:38,280 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:38,282 [551] [DEBUG] [app] Starting request: urn:request:a71b8cb1-ef26-4d23-9a9b-315229128c74 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:38,282 [551] [DEBUG] [app] Starting request: urn:request:a71b8cb1-ef26-4d23-9a9b-315229128c74 (/v1/_internal_ping) 2020-03-31 09:51:38,283 [551] [DEBUG] [app] Ending request: urn:request:a71b8cb1-ef26-4d23-9a9b-315229128c74 (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:38,283 [551] [DEBUG] [app] Ending request: urn:request:a71b8cb1-ef26-4d23-9a9b-315229128c74 (/v1/_internal_ping) 2020-03-31 09:51:38,284 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:38,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-registry stdout | 2020-03-31 09:51:38,284 [551] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:38,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2020-03-31 09:51:38,284 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:51:38,287 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-web stdout | 2020-03-31 09:51:38,285 [527] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2020-03-31 09:51:38,287 [551] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:38,288 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:38,288 [551] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:38,289 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:38,289 [551] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:38,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:38,296 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:38,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:38,311 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:38,315 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:38,316 [518] [DEBUG] [app] Starting request: urn:request:2e8432d0-1856-4483-bb68-ffb71e224134 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:38,316 [518] [DEBUG] [app] Starting request: urn:request:2e8432d0-1856-4483-bb68-ffb71e224134 (/_internal_ping) 2020-03-31 09:51:38,317 [518] [DEBUG] [app] Ending request: urn:request:2e8432d0-1856-4483-bb68-ffb71e224134 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:38,317 [518] [DEBUG] [app] Ending request: urn:request:2e8432d0-1856-4483-bb68-ffb71e224134 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:51:38,318 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:38,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:38,318 [518] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:38,318 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:38,321 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:51:38,321 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:38,324 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:51:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:38,325 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:38,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:38,328 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:38,330 [535] [DEBUG] [app] Starting request: urn:request:27fd6d30-35b1-45a4-8dfb-fab1ec55aa12 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:38,330 [535] [DEBUG] [app] Starting request: urn:request:27fd6d30-35b1-45a4-8dfb-fab1ec55aa12 (/c1/_internal_ping) 2020-03-31 09:51:38,331 [535] [DEBUG] [app] Ending request: urn:request:27fd6d30-35b1-45a4-8dfb-fab1ec55aa12 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:38,331 [535] [DEBUG] [app] Ending request: urn:request:27fd6d30-35b1-45a4-8dfb-fab1ec55aa12 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:51:38,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:38,333 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:51:38,333 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:51:38,333 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:38,335 [527] [DEBUG] [app] Ending request: urn:request:ea35c78a-2097-47b8-b252-b3a3e13bdab7 (/health/instance) 2020-03-31 09:51:38,335 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:38,335 [527] [DEBUG] [app] Ending request: urn:request:ea35c78a-2097-47b8-b252-b3a3e13bdab7 (/health/instance) 2020-03-31 09:51:38,335 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:38,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) gunicorn-web stdout | 2020-03-31 09:51:38,336 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:38 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:38 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.065 49 0.065) 2020-03-31 09:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2020-03-31 09:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:51.440707+00:00 (in 9.999130 seconds) 2020-03-31 09:51:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:51 UTC)" (scheduled at 2020-03-31 09:51:41.440707+00:00) repositoryactioncounter stdout | 2020-03-31 09:51:41,441 [172] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:51.440707+00:00 (in 9.999130 seconds) 2020-03-31 09:51:41,442 [172] [INFO] [apscheduler.executors.default] Running job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:51 UTC)" (scheduled at 2020-03-31 09:51:41.440707+00:00) 2020-03-31 09:51:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) repositoryactioncounter stdout | 2020-03-31 09:51:41,443 [172] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE NOT ("t1"."id" IN (SELECT "t2"."repository_id" FROM "repositoryactioncount" AS "t2" WHERE ("t2"."date" = %s))) ORDER BY Random() LIMIT %s OFFSET %s', [datetime.date(2020, 3, 30), 1, 0]) 2020-03-31 09:51:41,458 [172] [DEBUG] [__main__] No further repositories to count repositoryactioncounter stdout | 2020-03-31 09:51:41,458 [172] [DEBUG] [__main__] No further repositories to count 2020-03-31 09:51:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:41,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:51 UTC)" executed successfully repositoryactioncounter stdout | 2020-03-31 09:51:41,458 [172] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:41,458 [172] [INFO] [apscheduler.executors.default] Job "wrapper (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:51 UTC)" executed successfully 2020-03-31 09:51:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2020-03-31 09:51:43,540 [166] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:52:33.366791+00:00 (in 49.825817 seconds) 2020-03-31 09:51:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:43 UTC)" (scheduled at 2020-03-31 09:51:43.539999+00:00) 2020-03-31 09:51:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:51:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:43 UTC)" executed successfully exportactionlogsworker stdout | 2020-03-31 09:51:43,541 [166] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:52:33.366791+00:00 (in 49.825817 seconds) 2020-03-31 09:51:43,541 [166] [INFO] [apscheduler.executors.default] Running job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:43 UTC)" (scheduled at 2020-03-31 09:51:43.539999+00:00) 2020-03-31 09:51:43,541 [166] [DEBUG] [workers.queueworker] Running watchdog. 2020-03-31 09:51:43,541 [166] [INFO] [apscheduler.executors.default] Job "run_watchdog (trigger: interval[0:01:00], next run at: 2020-03-31 09:52:43 UTC)" executed successfully 2020-03-31 09:51:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2020-03-31 09:51:44,323 [175] [DEBUG] [apscheduler.scheduler] Looking for jobs to run 2020-03-31 09:51:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:49.498966+00:00 (in 5.174797 seconds) 2020-03-31 09:51:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:54 UTC)" (scheduled at 2020-03-31 09:51:44.323352+00:00) 2020-03-31 09:51:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2020-03-31 09:51:44,324 [175] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2020-03-31 09:51:49.498966+00:00 (in 5.174797 seconds) 2020-03-31 09:51:44,324 [175] [INFO] [apscheduler.executors.default] Running job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:54 UTC)" (scheduled at 2020-03-31 09:51:44.323352+00:00) 2020-03-31 09:51:44,325 [175] [DEBUG] [workers.queueworker] Getting work item from queue. 2020-03-31 09:51:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 44, 325116), True, datetime.datetime(2020, 3, 31, 9, 51, 44, 325116), 0, u'notification/%', 50, 1, 0]) notificationworker stdout | 2020-03-31 09:51:44,326 [175] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2020, 3, 31, 9, 51, 44, 325116), True, datetime.datetime(2020, 3, 31, 9, 51, 44, 325116), 0, u'notification/%', 50, 1, 0]) 2020-03-31 09:51:44,341 [175] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2020-03-31 09:51:44,341 [175] [DEBUG] [workers.queueworker] No more work. 2020-03-31 09:51:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:54 UTC)" executed successfully notificationworker stdout | 2020-03-31 09:51:44,341 [175] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:44,342 [175] [INFO] [apscheduler.executors.default] Job "poll_queue (trigger: interval[0:00:10], next run at: 2020-03-31 09:51:54 UTC)" executed successfully 2020-03-31 09:51:44,446 [527] [DEBUG] [app] Starting request: urn:request:bb8db0ab-0399-4dd6-a87e-2e836b0e3852 (/health/instance) gunicorn-web stdout | 2020-03-31 09:51:44,446 [527] [DEBUG] [app] Starting request: urn:request:bb8db0ab-0399-4dd6-a87e-2e836b0e3852 (/health/instance) 2020-03-31 09:51:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:44,449 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:44,454 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:44,454 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:44,456 [549] [DEBUG] [app] Starting request: urn:request:1debab0d-d781-4622-8e58-152a5e99aa3f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:44,456 [549] [DEBUG] [app] Starting request: urn:request:1debab0d-d781-4622-8e58-152a5e99aa3f (/v1/_internal_ping) 2020-03-31 09:51:44,458 [549] [DEBUG] [app] Ending request: urn:request:1debab0d-d781-4622-8e58-152a5e99aa3f (/v1/_internal_ping) 2020-03-31 09:51:44,458 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) 2020-03-31 09:51:44,459 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2020-03-31 09:51:44,458 [549] [DEBUG] [app] Ending request: urn:request:1debab0d-d781-4622-8e58-152a5e99aa3f (/v1/_internal_ping) gunicorn-registry stdout | 2020-03-31 09:51:44,458 [549] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:44,460 [527] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2020-03-31 09:51:44,459 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:44,460 [527] [INFO] [data.database] Connection pooling disabled for postgresql 2020-03-31 09:51:44,463 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 gunicorn-registry stdout | 2020-03-31 09:51:44,463 [549] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:9092 2020-03-31 09:51:44,466 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 gunicorn-registry stdout | 2020-03-31 09:51:44,466 [549] [DEBUG] [urllib3.connectionpool] http://localhost:9092 "POST /call HTTP/1.1" 200 0 2020-03-31 09:51:44,467 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics gunicorn-registry stdout | 2020-03-31 09:51:44,467 [549] [DEBUG] [util.metrics.prometheus] Sent 2 prometheus metrics 2020-03-31 09:51:44,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2020-03-31 09:51:44,472 [527] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) 2020-03-31 09:51:44,500 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2020-03-31 09:51:44,500 [527] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost 2020-03-31 09:51:44,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:44,505 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:44,507 [527] [DEBUG] [app] Starting request: urn:request:200c1146-244f-4eb1-aca0-e91463f4e097 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:44,507 [527] [DEBUG] [app] Starting request: urn:request:200c1146-244f-4eb1-aca0-e91463f4e097 (/_internal_ping) 2020-03-31 09:51:44,508 [527] [DEBUG] [app] Ending request: urn:request:200c1146-244f-4eb1-aca0-e91463f4e097 (/_internal_ping) gunicorn-web stdout | 2020-03-31 09:51:44,508 [527] [DEBUG] [app] Ending request: urn:request:200c1146-244f-4eb1-aca0-e91463f4e097 (/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) 2020-03-31 09:51:44,509 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:44,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2020-03-31 09:51:44,509 [527] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" 2020-03-31 09:51:44,509 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:44,513 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:44,513 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:44,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) gunicorn-web stdout | 2020-03-31 09:51:44,513 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan HTTP/1.1" 301 185 nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /secscan HTTP/1.1" 301 185 "-" "python-requests/2.22.0" (0.000 152 -) 2020-03-31 09:51:44,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:44,516 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /secscan/ HTTP/1.1" 403 55 "-" "python-requests/2.22.0" (0.001 153 0.001) 2020-03-31 09:51:44,518 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 gunicorn-web stdout | 2020-03-31 09:51:44,518 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /secscan/ HTTP/1.1" 403 55 2020-03-31 09:51:44,523 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) gunicorn-web stdout | 2020-03-31 09:51:44,523 [527] [WARNING] [py.warnings] /opt/rh/python27/root/usr/lib/python2.7/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 2020-03-31 09:51:44,525 [535] [DEBUG] [app] Starting request: urn:request:3c4e3e1f-10a9-43ce-bb99-00b9c8fb5a53 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:44,525 [535] [DEBUG] [app] Starting request: urn:request:3c4e3e1f-10a9-43ce-bb99-00b9c8fb5a53 (/c1/_internal_ping) 2020-03-31 09:51:44,526 [535] [DEBUG] [app] Ending request: urn:request:3c4e3e1f-10a9-43ce-bb99-00b9c8fb5a53 (/c1/_internal_ping) gunicorn-verbs stdout | 2020-03-31 09:51:44,526 [535] [DEBUG] [app] Ending request: urn:request:3c4e3e1f-10a9-43ce-bb99-00b9c8fb5a53 (/c1/_internal_ping) 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:51:44,526 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /c1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) 2020-03-31 09:51:44,527 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 gunicorn-verbs stdout | 2020-03-31 09:51:44,526 [535] [INFO] [gunicorn.access] 127.0.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /c1/_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2020-03-31 09:51:44,527 [527] [DEBUG] [urllib3.connectionpool] https://localhost:8443 "GET /c1/_internal_ping HTTP/1.1" 200 4 2020-03-31 09:51:44,529 [527] [DEBUG] [app] Ending request: urn:request:bb8db0ab-0399-4dd6-a87e-2e836b0e3852 (/health/instance) 2020-03-31 09:51:44,529 [527] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2020-03-31 09:51:44,529 [527] [DEBUG] [app] Ending request: urn:request:bb8db0ab-0399-4dd6-a87e-2e836b0e3852 (/health/instance) 2020-03-31 09:51:44,529 [527] [DEBUG] [data.database] Disconnecting from database. 2020-03-31 09:51:44,530 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" 10.131.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.085 49 0.085) gunicorn-web stdout | 2020-03-31 09:51:44,530 [527] [INFO] [gunicorn.access] 10.131.0.1 - - [31/Mar/2020:09:51:44 +0000] "GET /health/instance HTTP/1.0" 200 190 "-" "kube-probe/1.16+" nginx stdout | 10.131.0.1 () - - [31/Mar/2020:09:51:44 +0000] "GET /health/instance HTTP/2.0" 200 190 "-" "kube-probe/1.16+" (0.085 49 0.085)